Commit Graph

1069 Commits

Author SHA1 Message Date
Christian Decker f1c29aa3bd keysend: Do not strip even TLV types that were allowlisted
Changelog-Fixed: keysend: Keysend would strip even allowed extra TLV types before resolving, this is no longer the case.
2023-02-28 15:00:44 -06:00
Christian Decker 5dc85d185a keysend: Extract `accept-extra-tlv-types` from `listconfigs`
Since it is an optional field in the `listconfigs` output we can't use
the `rpc_scan` mechanism (doesn't handle optionals yet). We'll use
that list of accepted types later to avoid stripping them.
2023-02-28 15:00:44 -06:00
Christian Decker 29031c02ca libplugin: Expose the `jsonrpc_request_sync` method
This one was mostly used in libplugin, but not available outside. It
is rather useful, so let's expose it.
2023-02-28 15:00:44 -06:00
Rusty Russell f3baa3e510 sql: fix crash on fresh node_announcment.
Missing quotes when we delete the old one!

Reported-by: Alex Myers
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Vincenzo Palazzo bcc94b2d43 fix: do not send send peerstorage msg when disabled
This commit will disable the peerstorage plugins
when the feature is not enabled.

I found this issue with lnprototest, and I guess
we did not find it with normal run because
other the unknown messages are ingored?

Changelog-Fixed: Disable the protocol messages when peerstorage is disabled.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-02-27 11:18:56 -06:00
Rusty Russell 38d90b2505 autoclean: fix timer crash when we're cleaning two things at once.
If we had two things to clean, we fired off two requests (eg.
listforwards and listinvoices) and both marked the timer as finished,
triggering an assert.

We already have a refcount for outstanding requests to avoid this
for e.g. outstanding del commands, so use it here too!

```
Jan 19 19:20:00  lightningd[748044]: autoclean: plugins/libplugin.c:445: timer_complete: Assertion `p->in_timer > 0' failed.
Jan 19 19:20:00  lightningd[748044]: autoclean: FATAL SIGNAL 6 (version v22.11.1)
Jan 19 19:20:00  lightningd[748044]: 0x562c388136e4 send_backtrace
Jan 19 19:20:00  lightningd[748044]:         common/daemon.c:33
Jan 19 19:20:00  lightningd[748044]: 0x562c3881376c crashdump
Jan 19 19:20:00  lightningd[748044]:         common/daemon.c:46
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0898d5f ???
Jan 19 19:20:00  lightningd[748044]:         ./signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0898ce1 __GI_raise
Jan 19 19:20:00  lightningd[748044]:         ../sysdeps/unix/sysv/linux/raise.c:51
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0882536 __GI_abort
Jan 19 19:20:00  lightningd[748044]:         ./stdlib/abort.c:79
Jan 19 19:20:00  lightningd[748044]: 0x7f26d088240e __assert_fail_base
Jan 19 19:20:00  lightningd[748044]:         ./assert/assert.c:92
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0891661 __GI___assert_fail
Jan 19 19:20:00  lightningd[748044]:         ./assert/assert.c:101
Jan 19 19:20:00  lightningd[748044]: 0x562c3880355d timer_complete
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:445
Jan 19 19:20:00  lightningd[748044]: 0x562c38800b54 clean_finished
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:122
Jan 19 19:20:00  lightningd[748044]: 0x562c388010ed clean_finished_one
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:132
Jan 19 19:20:00  lightningd[748044]: 0x562c388011b6 del_done
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:149
Jan 19 19:20:00  lightningd[748044]: 0x562c388058b5 handle_rpc_reply
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:768
Jan 19 19:20:00  lightningd[748044]: 0x562c38805a39 rpc_read_response_one
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:944
Jan 19 19:20:00  lightningd[748044]: 0x562c38805ad7 rpc_conn_read_response
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:968
Jan 19 19:20:00  lightningd[748044]: 0x562c38876b60 next_plan
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/io.c:59
Jan 19 19:20:00  lightningd[748044]: 0x562c38876fe7 do_plan
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/io.c:407
Jan 19 19:20:00  lightningd[748044]: 0x562c38877080 io_ready
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/io.c:417
Jan 19 19:20:00  lightningd[748044]: 0x562c3887823c io_loop
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/poll.c:453
Jan 19 19:20:00  lightningd[748044]: 0x562c38805d11 plugin_main
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:1801
Jan 19 19:20:00  lightningd[748044]: 0x562c38801c7a main
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:613
```

Fixes: #5912
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-27 11:05:36 -06:00
Rusty Russell 15a744be8c commando: don't try putting an integer as the 'string' parameter to "datastore".
This only worked because we handled the JSON raw: next patch prohibits this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-11 12:22:40 -06:00
Rusty Russell 70aee52903 libplugin: don't spew datastore errors to LOG_DEBUG.
People get upset, especially as our "not found" error can be a bit
hard to read!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
See-also: #5990
2023-02-11 12:22:40 -06:00
Vincenzo Palazzo ca1fa84458 ignore sql binary plugin
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-02-08 14:38:58 -06:00
Rusty Russell a71bd3ea37 options: create enable/disable option for peer storage.
Since it's not spec-final yet (hell, it's not even properly specified
yet!) we need to put it behind an experimental flag.

Unfortunately, we don't have support for doing this in a plugin; a
plugin must present features before parsing options.  So we need to do
it in core.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
Rusty Russell c60ea5bcbb plugins/chanbackup: make get_file_data take ctx.
When you return an allocated pointer, you should always hand in the
context you want it allocated from.  This is more explicit, because it may
really matter to the caller!

This also folds some simple operations, and avoids doing too much
variable assignment in the declarations themselves: some coding styles
prohibit such initializers, but that's a bit exteme.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
Rusty Russell 17c35819d8 plugins/chanbackup: neaten a little.
node_id can be on the stack, avoiding a tal call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
Rusty Russell f1fed40ac2 features: make name of peer storage features match spec.
And we should always represent them as is, not as optional: it's
possible in future we could *require* "WANT_PEER_BACKUP_STORAGE".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
Rusty Russell 68d9b21aec plugins/chanbackup: switch to normal indentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
adi2011 e637e843e7 Plugins/chanbackup: Add RPC for recovering from the latestscb received from peers. 2023-02-08 08:37:59 -06:00
adi2011 7affaff728 Plugins/chanbackup: Add hook for exchanging msgs on connect with a peer 2023-02-08 08:37:59 -06:00
adi2011 d205f489bd Plugins/chanbackup: Add hook for receiving custommsg 2023-02-08 08:37:59 -06:00
adi2011 01cafe478b Plugins/chanbackup: Add SCB on CHANNELD_AWAITING_LOCKING stage 2023-02-08 08:37:59 -06:00
adi2011 33f0c4ec0b plugins/chanbackup: use grab_file. 2023-02-08 08:37:59 -06:00
adi2011 ff777e3238 plugins/chanbackup: Define FILENAME globally (Good Manners) 2023-02-08 08:37:59 -06:00
adi2011 2b1867aca3 Plugins/chanbackup: Add featurebit Peerstrg and YourPeerStrg. 2023-02-08 08:37:59 -06:00
adi2011 93d03bf9e8 plugins/chanbackup: PLUGIN_RESTARTABLE to PLUGIN_STATIC... 2023-02-08 08:37:59 -06:00
Christian Decker 640edf3955 grpc: Silence a warning about `nonnumericids` being unused 2023-02-07 08:23:17 -06:00
Rusty Russell bcab3f7e83 Makefile: don't try to build sql plugin if there's no sqlite3 support.
Reported-by: @whitslack
Fixes: #5940
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-06 09:20:28 +01:00
niftynei 278fa7a0a4 v2 opens: don't use p2sh inputs for v2 opens
They're not allowed!
2023-02-04 15:31:16 +10:30
niftynei 35f12b4ca1 upgradewallet: JSONRPC call to update p2sh outputs to a native segwit
v2 opens require you to use native segwit inputs

Changelog-Added: JSONRPC: `upgradewallet` command, sweeps all p2sh-wrapped outputs to a native segwit output
2023-02-04 15:31:16 +10:30
Rusty Russell 35acdc112f offers: fix pay where we are using deprecated apis.
In this case, "fee_base_msat" from `listincoming` has `msat` appended.

Fixes: #5850
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 16:59:51 +01:00
niftynei ff0a63a0d7 valgrind-fix: patch valgrind error on log statement in pay plugin
The htlc_budget only exists iff the hint is a 'local' one; we were
failing to write to the htlc_budget field for non-local cases.

To avoid this, we make `local` into a struct that contains the fields
that pertain to local-only payments (in this case, `htlc_budget`).

Valgrind error file: valgrind-errors.1813487
==1813487== Conditional jump or move depends on uninitialised value(s)
==1813487==    at 0x4A9C958: __vfprintf_internal (vfprintf-internal.c:1687)
==1813487==    by 0x4AB0F99: __vsnprintf_internal (vsnprintf.c:114)
==1813487==    by 0x1D2EF9: do_vfmt (str.c:66)
==1813487==    by 0x1D3006: tal_vfmt_ (str.c:92)
==1813487==    by 0x11A60A: paymod_log (libplugin-pay.c:167)
==1813487==    by 0x11B749: payment_chanhints_apply_route (libplugin-pay.c:534)
==1813487==    by 0x11EB36: payment_compute_onion_payloads (libplugin-pay.c:1707)
==1813487==    by 0x12000F: payment_continue (libplugin-pay.c:2135)
==1813487==    by 0x1245B9: adaptive_splitter_cb (libplugin-pay.c:3800)
==1813487==    by 0x11FFB6: payment_continue (libplugin-pay.c:2123)
==1813487==    by 0x1206BC: retry_step_cb (libplugin-pay.c:2301)
==1813487==    by 0x11FFB6: payment_continue (libplugin-pay.c:2123)
==1813487==
{
   <insert_a_suppression_name_here>
   Memcheck:Cond
   fun:__vfprintf_internal
   fun:__vsnprintf_internal
   fun:do_vfmt
   fun:tal_vfmt_
   fun:paymod_log
   fun:payment_chanhints_apply_route
   fun:payment_compute_onion_payloads
   fun:payment_continue
   fun:adaptive_splitter_cb
   fun:payment_continue
   fun:retry_step_cb
[sesh] 0:[tmux]*Z

Suggested-By: @nothingmuch
2023-02-02 18:08:27 -06:00
Rusty Russell 9e9686df20 pay: specify the channel to send the first hop.
If we only specify the node_id, we get the "first" channel.

Closes: #5803
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` uses the correct local channel for payments when there are multiple available (not just always the first!)
2023-02-03 09:17:33 +10:30
Rusty Russell ec8aab7cb2 libplugin-pay: fix (transitory) memleak which memleak detection complains about.
We assign this in the loop without freeing it first.

```
 plugin-pay: MEMLEAK: 0x55792b155e18
 plugin-pay:   label=plugins/libplugin-pay.c:3274:struct short_channel_id_dir
 plugin-pay:   backtrace:
 plugin-pay:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
 plugin-pay:     plugins/libplugin-pay.c:3274 (direct_pay_listpeerchannels)
 plugin-pay:     plugins/libplugin.c:860 (handle_rpc_reply)
 plugin-pay:     plugins/libplugin.c:1036 (rpc_read_response_one)
 plugin-pay:     plugins/libplugin.c:1060 (rpc_conn_read_response)
 plugin-pay:     ccan/ccan/io/io.c:59 (next_plan)
 plugin-pay:     ccan/ccan/io/io.c:407 (do_plan)
 plugin-pay:     ccan/ccan/io/io.c:417 (io_ready)
 plugin-pay:     ccan/ccan/io/poll.c:453 (io_loop)
 plugin-pay:     plugins/libplugin.c:1893 (plugin_main)
 plugin-pay:     plugins/pay.c:1294 (main)
 plugin-pay:     ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main)
 plugin-pay:     ../csu/libc-start.c:381 (__libc_start_main_impl)
 plugin-pay:   parents:
 plugin-pay:     plugins/libplugin-pay.c:3308:struct direct_pay_data
 plugin-pay:     plugins/libplugin.c:1775:struct plugin
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell f87c7ed439 plugins/sql: fix foreign keys.
I noticed that our subtables were not being cleaned, despite being "ON
DELETE CASCADE".  This is because foreign keys were not enabled, but
then I got foreign key errors: rowid cannot be a foreign key anyway!

So create a real "rowid" column.  We want "ON DELETE CASCADE" for
nodes and channels (and other tables in future) where we update
partially.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Rusty Russell 6176912683 plugins/pay: fix htlc_budget calc when we get temporary_channel_failure
Valgrind correctly reports it as uninitialized for this log message, and
the only way this can happen is channel_hints_update() when we receive a
temporary_channel_failure.  Put a dummy value here in this case.

```
Valgrind error file: valgrind-errors.23404
==23404== Conditional jump or move depends on uninitialised value(s)
==23404==    at 0x49E4B56: __vfprintf_internal (vfprintf-internal.c:1516)
==23404==    by 0x49F6519: __vsnprintf_internal (vsnprintf.c:114)
==23404==    by 0x1EBCEB: do_vfmt (str.c:66)
==23404==    by 0x1EBDF8: tal_vfmt_ (str.c:92)
==23404==    by 0x11A336: paymod_log (libplugin-pay.c:167)
==23404==    by 0x11B4B2: payment_chanhints_apply_route (libplugin-pay.c:534)
==23404==    by 0x11E999: payment_compute_onion_payloads (libplugin-pay.c:1707)
==23404==    by 0x11FF4C: payment_continue (libplugin-pay.c:2135)
==23404==    by 0x1245C0: adaptive_splitter_cb (libplugin-pay.c:3800)
==23404==    by 0x11FEF3: payment_continue (libplugin-pay.c:2123)
==23404==    by 0x1205FE: retry_step_cb (libplugin-pay.c:2301)
==23404==    by 0x11FEF3: payment_continue (libplugin-pay.c:2123)
==23404==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 10:11:11 +10:30
Rusty Russell 0240c24936 plugins/sql: listsqlschemas command to retrieve schemas.
Good for detection of what fields are present.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell d8320f015f plugins/sql: add bkpr-listaccountevents and bkpr-listincome support.
This *would* be a 1-line change (add it to Makefile) except that we
previously assumed a "list" prefix on commands.

These use the default refreshing, but they could be done better using
the time-range parameters.

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 9a591277f5 plugins/sql: allow some simple functions.
And document them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 14aac0769c doc: document the sql command.
In particular, we generate the schema part from the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `sql` plugin command to perform server-side complex queries.
2023-01-30 15:15:41 -06:00
Rusty Russell adb8de3e07 plugins/sql: print out part of man page referring to schemas.
We now add tables to the strmap as we allocate them, since we don't
want to call "finish_td" when we're merely invoked for the
documentation, and don't need a database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 20654ebd49 plugins/sql: pay attention to `deprecated` in schema.
For now, we ignore every deprecated field, but put in the logic so
that future deprecations will work as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 9b08c4f25a plugins/sql: refresh listnodes and listchannels by monitoring the gossip_store.
It's quite a lot of code, but these are the most expensive commands we
do so it's worth it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell aa3a1131aa plugins/sql: include the obvious indexes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 8a0ee5f56e plugins/sql: add listpeerchannels support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 51ae7118f1 plugins/sql: make tables for non-object arrays.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell c230291141 plugins/sql: rework to parse schemas.
This requires us to rename "index" fields, rename fields if we have a
sub-object, and create sub-tables if we have an array, and handle the
fact that some listX commands don't contain array X (listsendpays
contains "payments").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 260643157d plugins/sql: create `struct column` to encode column details.
Rather than two arrays "columns" (for names) and "fieldtypes" (for
types), use a struct.  This makes additions easier for successive
patches.

Also pull process_json_obj() out of the loop in list_done().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 24d86a85c3 plugins/sql: initial commit of new plugin.
This is designed to allow you to perform complex server-side
queries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 0274d88bad common/gossip_store: clean up header.
It's actually two separate u16 fields, so actually treat it as
such!

Cleans up zombie handling code a bit too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 9ab488fc41 plugins/topology: add direction field to listchannels.
It's a core concept in the spec which isn't directly exposed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listchannels` added a `direction` field (0 or 1) as per gossip specification.
2023-01-30 15:15:41 -06:00
Rusty Russell 2dec805465 decode: fix handling of blinded_payinfo.
Our pay code handles this correctly, but decode was still using an old model
where there was a payinfo per hop, not per path.

Reported-by: @t-bast
See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 6eb7a4cbf2 plugins: update to match latest offers text.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-offers" BOLTVERSION=guilt/offers
```

In this case, only trivial mods.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell e9eb5f493b common: update to latest route-blinding spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-route-blinding" BOLTVERSION=guilt/offers
```

Other than textual changes, this does:

1. Ensures we put total_amount_msat in onion final hop (reported by @t-bast).
2. Require that they put total_amount_msat in onion final hop.
3. Return `invalid_onion_blinding` exactly as defined by the spec (i.e. less
   aggressive when we're the final hop) (also reported by @t-bast, but I already
   knew).

See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `offers` breaking blinded payments change (total_amount_sat required, Eclair compat)
2023-01-30 13:24:29 +10:30