This means remembering the connection direction. We also use the address to try
to reconnect, which we shouldn't bother with if they connect to us.
For peers from the database, we currently always save the addr: we shouldn't really
do this if they connected to us, since it's not useful for reconnecting (we don't
show the addr in JSON reply to listpeers unless we're connected, so it's only an
internal issue). This is left for future work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This matters: if we connected, the address is probably usable for future connections.
But if they connected, the port is probably not (but the IP address may be).
Changelog-Added: JSON-RPC: `connect` returns "direction" ("in": they iniatated, or "out": we initiated)
Changelog-Added: plugins: `peer_connected` hook and `connect` notifications have "direction" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
As @fiatjaf points out we were notifying before we were actually set
on accepting, since the hook could also still reject. Switched them
around does and calling the notification only once it's been decided
is the correct thing to do.
Changelog-Fixed: plugin: The `invoice_payment` notification was being sent before the hook was called, which could still abort it.
Suggested-by: Fiatjaf <@fiatjaf>
Signed-off-by: Christian Decker <@cdecker>
There are perfectly valid reasons for us to not have a command on return
(something went boom while sending them our sigs and we've now gotten
their sigs during a reconnect and subsequently broadcast the tx)
This can result in us logging a warning if we've 1) dropped their sigs
response, 2) only us (the opener) added inputs, 3) and we broadcast on
their reconnect (when they retransmit their sigs)
Otherwise, we might find an address other than the one given and
the user might think that address worked.
Fixes: #4185
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `connect` returns `address` it actually connected to
And update all the in-tree callers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `fundchannel_complete` `txid` and `txout` parameters (use `psbt`)
Requiring the user to calculate the txid of the PSBT is a horrible, bad,
no-good idea.
Doesn't deprecate yet, so I can test that this path works while
multifundchannel still uses it.
Fixes: #4416 (at least for future users!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_complete` takes a psbt parameter.
Changelog-Added: lightningd: experimental-shutdown-wrong-funding to allow remote nodes to close incorrectly opened channels.
Changelog-Added: JSON-RPC: close has a new `wrong_funding` option to try to close out unused channels where we messed up the funding tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not unheard of for people to give the wrong funding tx to us,
getting their funds stuck. Interestingly, we can allow mutual close
using a different txid and output number as long as they (solely)
funded the channel, and the channel hasn't been used.
This defines a "play area" feature to do just that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Allows us to clean up an in-progress open that we won't be completing
Changelog-Added: EXPERIMENTAL JSON-RPC: Permit user-initiated aborting of in-progress opens. Only valid for not-yet-committed opens and RBF-attempts
The `rbf_channel` hook uses `our_funding_msat`, which is a nicer
and more easily understood than the `openchannel2`
`accepter_funding_msat`.
This updates the `openchannel2` hook to use the same nomenclature as
`rbf_channel`.
We were reporting the failure immediately but still continuing with
the startup. This could happen if an important plugin ends up in a
race with another plugin (important or not) for a contended
resource (CLI option or RPC method name). We would eventually notice
that we were supposed to abort, but at that point we already processed
a couple of blocks, loaded the entire state, etc.
This just aborts early with a sane error message.
Changelog-Added: plugin: If there is a misconfiguration with important plugins we now abort early with a more descriptive error message.
Reported-by: PsySc0rpi0n
Reported-by: Ján Sáreník <@jsarenik>
The receiving node: ...
- MUST fail the channel if:
- the `witness_stack` weight lowers the effective `feerate`
below the agreed upon transaction `feerate`
We consolidate to the latest/singular RFC patch for dual-funding, so
there's just a single patchfile for the change. Plus we move back to the
opener setting the desired feerate, the accepter merely declines to
participate if they disagree with the set rate.
Looks like #4394 treated a symptom but not the root cause. We were
actually sending the message framed with the WIRE_CUSTOMMSG_OUT and
the length prefix over the encrypted connection to the peer. It just
happened to be a valid custommsg...
This fixes the issue, and this time I made sure we actually send the
raw message over the wire. However for backward compatibility we
needed to imitate the faulty behavior which is 90% of this patch :-)
Changelog-Fixed: plugin: `dev-sendcustommsg` included the type and length prefix when sending a message.
We were getting a memleak error that the open_attempt isnt' being
cleaned up in test_rbf_reconnect_tx_construct. I had some trouble
reproducing it, so I removed the reliance on using `tmpctx` to clean it
up and was more surgical about cleaning it up inline.
Changelog-Added: JSON-RPC: `listpeers` now includes 'last_feerate', 'next_feerate', 'initial_feerate' and 'next_fee_step' for channels in state DUALOPEND_AWAITING_LOCKIN
fixup! listpeers: include feerate info for RBF-candidate channels
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.
Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
We're *mostly* set up for both sides doing RBF, except that it reverses
the callback flow (using the plugin vs RPC calls) and we're not
currently smart enough to flip between them gracefully
Users have no idea what they would pay for unilateral closes.
At least this gives them a clue!
Reported-by: @az0re on IRC.
Changelog-Added: JSON-RPC: `listpeers` now shows latest feerate and unilaral close fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
only needed for moving the subd->channel from an uncommitted_channel to
a channel; we removed uncommitted_channel from dual_open so it's no
longer necessary
We're going to move over to "unsaved channels" instead of using a
separate "uncommitted_channel" struct. This means a peer might have a
channel that's not saved to the database yet in its channel list
We need to make sure that there's at least one input that's represented
in every single RBF-attempt for this channel, to prevent "parallel"
subsequent RBFs from succeeding/opening (the multi-channel backdoor?!)
When the funding tx reaches depth, update the channel's data to the
"correct" funding transaction info from inflights (if necessary).
This will be necessary if:
- the transaction has been successfully RBF'd and
- the lesser fee transaction is the one successfully mined, OR
- the channel is in the process of being RBF'd
We were always prefixing the `message` field with the internal type
prefix 0x0407, followed by the length prefix. Neither is needed since
the type being constant is of no interest to the plugin and the length
being implicit due to the JSON-encoding.
Reported-by: Ilya Evdokimov
Changelog-Fixed: plugin: The `custommsg` hook no longer includes the internal type prefix and length prefix in its `payload`
Changelog-Deprecated: plugin: The `message` field on the `custommsg` hook is deprecated in favor of the `payload` field, which skips the internal prefix.
This avoids spamming the logs. We also remove the duplicate debug
logs on self-disable (plugin_kill logs it for us).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently log every kill at INFO level, even if it's during shutdown.
Change those to debug, but lift those where we got a malformed response.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use waitpid() manually for subdaemons, so we need to step
around that (otherwise we could simply ignore them).
We could destroy subdaemons only once they've exited, but
that works badly with the sd->conn, which will be freed
when error (i.e. close) is detected, so the current code
is probably the best compromise.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If a plugin died due to connection close, we'd always say
"Plugin exited before completing handshake.", which was often
wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to only set it for single-use offers (where it's required),
but it's still interesting for multi-use offers, so let's keep it
there.
We also put this field in the documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we turned many errors into warnings, we want our tests to fail
when they happen unexpectedly. We make WARNING clear in the strings
we print, too, to help out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
This is in line with the warnings draft, where all-zeroes in a
channel_id is no longer special (i.e. it will be ignored).
But gossipd would send these if it got upset with us, so it's best
practice to ignore them for now anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we treat error messages from peer which refer to "all channels" as warnings, not errors.
We fix up the test by using pay, instead of sendpay (and making pay log
the expected message).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: sendpay no longer extracts updates from errors, the caller should do it from the `raw_message`.
From #clightning:
(11:24:10) andytoshi: hiya, i'm trying to set up a new lightningd node, and when i run lightningd --network=bitcoin --log-level=debug --daemon
(11:24:17) andytoshi: i get errors of the form fetchinvoice: Malformed JSON reply '2021-01-25T00:51:16.655Z DEBUG plugin-offers: disabled itself at init: offers not enabled in config
(11:24:43) andytoshi: there are a couple variants of this, but always some form of "something: failed to parse <a log line> as json"
Indeed, we close stdout, and it ends up being reused for some plugin.
But the real problem is that we log to stdout by default, which doesn't
make sense. If they really want to discard logs, they can use
--log-file=/dev/null.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON failures when --daemon is used without --log-file.
rearranges the`peer_connected_hook_payload` definition to the location
where this is used in the file.
Fixes certain blanklines and linebreaks to make the code look nicer.
Nit: The underscore in "openchannel_hook" is wrong, bcause the name of
the hook is just "openchannel". The "_hook" implied this to be part of
the name.
Changelog-None
It doesn't make sense currently without EXPERIMENTAL_FEATURES, and
check-manpages complains about it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
payload is owned by the peer, which is freed in this case, then we
free payload (again).
==1404== Invalid read of size 8
==1404== at 0x1F39E8: to_tal_hdr (tal.c:174)
==1404== by 0x1F43A4: tal_free (tal.c:479)
==1404== by 0x14B3D1: peer_connected_hook_cb (peer_control.c:1087)
==1404== by 0x15D6E9: plugin_hook_call_ (plugin_hook.c:288)
==1404== by 0x14B40E: plugin_hook_call_peer_connected (peer_control.c:1090)
==1404== by 0x14B5B8: peer_connected (peer_control.c:1135)
==1404== by 0x122FCF: connectd_msg (connect_control.c:310)
==1404== by 0x160291: sd_msg_read (subd.c:480)
==1404== by 0x15FBE7: read_fds (subd.c:308)
==1404== by 0x1E37D1: next_plan (io.c:59)
==1404== by 0x1E434E: do_plan (io.c:407)
==1404== by 0x1E438C: io_ready (io.c:417)
==1404== Address 0x2fcd2268 is 24 bytes inside a block of size 336 free'd
==1404== at 0x4C32D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404== by 0x1F416E: del_tree (tal.c:421)
==1404== by 0x1F40F2: del_tree (tal.c:412)
==1404== by 0x1F442C: tal_free (tal.c:486)
==1404== by 0x148816: delete_peer (peer_control.c:120)
==1404== by 0x148899: maybe_delete_peer (peer_control.c:136)
==1404== by 0x13A970: destroy_uncommitted_channel (opening_common.c:29)
==1404== by 0x1F3BB1: notify (tal.c:240)
==1404== by 0x1F40A0: del_tree (tal.c:402)
==1404== by 0x1F442C: tal_free (tal.c:486)
==1404== by 0x13D3E9: peer_start_openingd (opening_control.c:911)
==1404== by 0x14B3C2: peer_connected_hook_cb (peer_control.c:1086)
==1404== Block was alloc'd at
==1404== at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404== by 0x1F3C1B: allocate (tal.c:250)
==1404== by 0x1F41B4: tal_alloc_ (tal.c:428)
==1404== by 0x14B454: peer_connected (peer_control.c:1105)
==1404== by 0x122FCF: connectd_msg (connect_control.c:310)
==1404== by 0x160291: sd_msg_read (subd.c:480)
==1404== by 0x15FBE7: read_fds (subd.c:308)
==1404== by 0x1E37D1: next_plan (io.c:59)
==1404== by 0x1E434E: do_plan (io.c:407)
==1404== by 0x1E438C: io_ready (io.c:417)
==1404== by 0x1E6552: io_loop (poll.c:445)
==1404== by 0x12E2AD: io_loop_with_timers (io_loop_with_timers.c:24)
Fixes: #4329
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
By returning 'disable: <reason>' inside getmanifest or init result.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: plugins can now disable themselves by returning `disable`, even if marked important.
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
We weren't sending a channel_open notification for dual-funded channels.
This is only sent for the 'accepter' side. We send it as soon as both
funding_tx sigs have been exchanged, even though it's possible the
funding transaction might be published without this having been the case.
Since we fail the channel if this happens, only notifying for good/valid
channels reaching the broadcast state is the right way to handle this.
Allow a user to switch on dual-funding without needing to compile
as EXPERIMENTAL_FEATURES.
Doesn't work yet, since everything is still behind
'EXPERIMENTAL_FEATURES' compile time flags... but useful for testing
Let plugins know when a channel open has failed.
We need to notify accepters now too, so we remove the check on who's
funding the channel before sending the 'failed' message from
dualopend->master.
We already know what the channel id is, we should go ahead and pass it
on to any listening plugins -- this makes clean up easier/possible
if a open fails early on and we've got reserved utxos.
When we're the opener, we get the upfront shutdown scriptpubkey (if
there is one) from the `close_to` param of `openchannel_init`.
We were passing it through dualopend, but we need to break the update
chain so that our test_option_upfront_shutdown_script test works (same
as on the openingd flow.)
Prior to this, all reconnect logic lived in channeld. If you
disconnected before we finished building a funding transaction, that was
no big deal. Now, however, we're waiting for the funding to lock in in
dualopend, instead of handing straight to channeld to wait.
So we need a way to restart dualopend.
We delegate the decision about what to do about 'out of bound' feerates
to the plugin (if one exists), however in the case that the plugin
doesnt exist or doesnt want to figure it out, we default to using the
'best' as their intended feerate, and rejecting if it's out of bounds.
We need to know if they've sent us their sigs message yet. Ideally, we'd
be able to check the 'finalness' of the PSBT, however if the peer
doesn't have any inputs to the channel this doesn't work.
Back in the days before dual-funding, the `channel` struct on subd was
only every one type per daemon (either struct channel or struct
uncommitted_channel)
The RBF requirement on dualopend means that dualopend's channel,
however, can now be two different things -- either channel or
uncommitted_channel.
To track the difference/disambiguate, we now track the channel type on a
flag on the subd. It gets updated when we swap out the channel.
This will make it possible to do RBF, since we can re-start the opening
process in dualopend while waiting for lock-in.
Note the new channel states are being used, DUALOPEND_INIT and
DUALOPEND_AWAITING_LOCKIN, to differentiate from openingd/channeld opens
v2 channel opens are going to happen over in dualopend. In order
to make sure that these don't end up in the wrong place/to keep track of
the difference between "waiting for sigs" and "have merely initiatlized
a channel", we add two new states to the channel state machine.
A channel that 'originates' in dualopend will only ever arrive at
channeld in the state CHANNELD_NORMAL.
Since this all stays in dualopend/dual_open_control, we can hold
onto the openchannel_signed command to wait for a response here locally.
Previously we were splitting across the channeld/openingd boundary.
We used this for dual funded opens, to track the receipt of signatures.
We're moving all of this over to dualopend now, however, so we no longer
need the PSBT in channeld.
This is fixed when payload is freed, but I noted a leak in the case of
an invalid payload:
```
E ValueError:
E Node errors:
E Global errors:
E - Node /tmp/ltests-ipt6ab_y/test_sendinvoice_1/lightning-2/ has memory leaks: [
E {
E "backtrace": [
E "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E "ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)",
E "gossipd/gossipd_wiregen.c:925 (fromwire_gossipd_got_onionmsg_to_us)",
E "lightningd/onion_message.c:99 (handle_onionmsg_to_us)",
E "lightningd/gossip_control.c:166 (gossip_msg)",
E "lightningd/subd.c:480 (sd_msg_read)",
E "ccan/ccan/io/io.c:59 (next_plan)",
E "ccan/ccan/io/io.c:407 (do_plan)",
E "ccan/ccan/io/io.c:417 (io_ready)",
E "ccan/ccan/io/poll.c:445 (io_loop)",
E "lightningd/io_loop_with_timers.c:24 (io_loop_with_timers)",
E "lightningd/lightningd.c:1016 (main)"
E ],
E "label": "gossipd/gossipd_wiregen.c:925:u8[]",
E "parents": [
E "lightningd/onion_message.c:96:struct onion_message_hook_payload",
E "lightningd/plugin_hook.c:87:struct hook_instance *[]"
E ],
E "value": "0x560779438db8"
E }
E ]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our new "decode" command will also handle bolt11. We make a few cleanups:
1. Avoid type_to_string() in JSON, instead use format functions directly.
2. Don't need to escape description now that JSON core does that for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The previous onion_message code required a confirmed, not-shutting-down
channel, not just a connection. That's overkill; plus before widespread
adoption we will want to connect directly as a last resort.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is required if we want to create a "bouncer" plugin (in my copious free time!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
This makes for more useful errors. It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Interestingly, we required that "sendrawtx" include "errmsg" field even
on success, otherwise we crashed in broadcast_remainder.
We only actually insist on an "errmsg" if success is false. And this
logic here is weird (the !success) was added by darosior in
947f5ddde1, which makes the msg checks redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A user reported that it is sometimes cumbersome to search an invoice
based on the payment hash or the bolt11 string in the full list, which
may be required when we don't have the label available.
This adds support for querying / filtering based on the `payment_hash`
or `bolt11` string.
Changelog-Added: JSON-RPC: `listinvoices` can now query for an invoice matching a `payment_hash` or a `bolt11` string, in addition to `label`
This makes use of the constant defined in the previous commits to more
accurately detect plaintext, encrypted, and invalid seeds. We now error
on invalid seeds.
Changelog-changed: hsmd: we now error at startup on invalid hsm_secret
Changelog-changed: hsmtool: all commands now error on invalid hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database. We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This takes an unsigned bolt11 (or bolt12 if EXPERIMENTAL_FEATURES) string
and signs it and puts it in the database.
The invoice command could now be moved out to a plugin, in fact.
Changelog-Added: JSON-RPC: `createinvoice` new low-level invoice creation API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer. We mark it used when it's paid, to avoid a race.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows us to mark an offer used when an invoice derived from it
is paid, and importantly, avoid any other invoices for the offer being
paid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
"multi" means that specifying a parameter twice will append, not override.
Multi args are always given as a JSON array, even if only one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: new "multi" field allows an option to be specified multiple times.
This was fixed in 0.8.2.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: plugins: options to init are no longer given as strings if they are bool or int types (deprecated in 0.8.2).
Still asserts that it's the standard size, but makes it a dynamic
member. For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
niftynei
Christian Decker
Jan Sarenik
Karol Hosiawa
Michael Schmoock
Antoine Poinsot
fiatjaf