libwally's API requires us to pass in NULL pointers if the array size is
zero, so we update our array from wire-er to comply with this
requirement
[ Added fix to avoid tal_resize() of NULL -- RR ]
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)
Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway). In fact, the open/accept is a TLV
because it was actually the same format.
For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381
Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're not using the change_outnum for withdraw tx's (and the way
we were calculating it was broken as of the addition of 'multiple
outputs'). This removes the change output knowhow from withdraw_tx
entirely, and pushes the responsibility up to the caller to
include the change output in the output set if desired.
Consequently, we also remove the change output knowhow from hsmd.
Update the `bitcoin_tx_add_input` interface to accept a witness script
and or scriptPubkey.
We save the amount + witness script + witness program (if known) to
the PSBT object for a transaction when creating an input.
Spec is wrong (it says it should be compulsory), and Eclair doesn't set it
at all, leading to an error when they send their announcement_signatures.
Fixes: #3703
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: large-channels: negotiate successfully with Eclair nodes.
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case. Use that more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core
Changelog-None
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those. Now we do, so
there's less excuse for this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We only use sizeof(f1->bits).
```
common/test/run-features.c:84:36: error: Uninitialized variable: f1 [uninitvar]
for (size_t i = 0; i < ARRAY_SIZE(f1->bits); i++) {
^
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This moves the notification for our coin spends from when it's
successfully submited to the mempool to when they're confirmed in a
block.
We also add an 'informational' notice tagged as `spend_track` which
can be used to track which transaction a wallet output was spent in.
Previously we were annotating every movement with the blockheight of
lightningd at notification time. Which is lossy in terms of info, and
won't be helpful for reorg reconciliation. Here we switch over to
logging chain moves iff they've been confirmed.
Next PR will fix this up for withdrawals, which are currently tagged
with a blockheight of zero, since we log on successful send.
onchaind is the only daemon that emits coin events, and those are all
onchain (ha!), so the only 'wire' facility we need for coin moves are
for the 'chain' type.
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward. This changes it to directly
access those fields.
This is not only more elegant (60 fewer lines), it would also be
more cache friendly. That's right: cache hot singles!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously we've used the term 'funder' to refer to the peer
paying the fees for a transaction; v2 of openchannel will make
this no longer true. Instead we rename this to 'opener', or the
peer sending the 'open_channel' message, since this will be universally
true in a dual-funding world.
We have several of these, and they're not always called obvious things like
"delete" or "free". `STEALS` provides a strong hint here.
I only added it to a couple I knew about off the top of my head.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that it's channeld which calculates the shared secret, too. This
minimizes the work that lightningd has to do, at cost of passing this
through.
We also don't yet save the blinding field(s) to the database.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This requires us to call ecdh() in the corner case where the blinding seed
is in the TLV itself (which is the case for the start of a blinded route).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We now track all pending RPC passthrough calls, and terminate them with an
error if the plugin dies.
Changelog-Fixed: JSON-RPC: Pending RPC method calls are now terminated if the handling plugin exits prematurely.
The spec states that invoices with an amount, but lacking a multiplier, should
be interpreted as integer Bitcoin amounts:
`amount`: optional number in that currency, followed by an optional
`multiplier` letter. The unit encoded here is the 'social' convention of a
payment unit -- in the case of Bitcoin the unit is 'bitcoin' NOT satoshis.
Suggested-by: Stefano Pellegrini <@St333p>
Signed-off-by: Christian Decker <@cdecker>
Changelog-Fixed: invoice: The invoice parser assumed that an amount without a multiplier was denominated in msatoshi instead of bitcoins.
common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV. But how it does ecdh is daemon-specific.
We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh(). This just makes it more general.
So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently abuse the added_htlc and failed_htlc messages to tell channeld
about existing htlcs when it restarts. It's clearer to have an explicit
'existing_htlc' type which contains all the information for this case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's almost always "their_features" and "our_features" respectively, so
make those names clear.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Passing 0 as minconf to withdraw allows you to use unconfirmed transaction outputs, even if explicitly passed as the `utxos` parameter
This cleans up the boutique handling of features, and importantly, it
means that if a plugin says to offer a feature in init, we will now
*accept* that feature.
Changelog-Fixed: Plugins: setting an 'init' feature bit allows us to accept it from peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is to prepare for dynamic features, including making plugins first
class citizens at setting them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We roll the `elements_add_fee_output` function and the cropping of
overallocated arrays into the `bitcoin_tx_finalize` function. This is supposed
to be the final cleanup and compaction step before a tx can be sent to bitcoin
or passed off to other daemons.
This is the cleanup promised in #3491
For messages, we use the onion but payload lengths 0 and 1 aren't special.
Create a flag to disable that logic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Expands the interface to play with onions a bit more. Potentially a bit
slower due to allocations, but that's a small price to pay. It also allows us
to avoid serializing a compressed onion to `u8*` if we process it right away.
Also implements a way to decompress an onion using the devtools/onion tool
Changelog-Added: devtools: The `onion` tool can now generate, compress and decompress onions for rendez-vous routing
Does the allocation and copying; this is useful because we can
avoid being fooled into doing giant allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ChangeLog-Added: New `getsharedsecret` command, which lets you compute a shared secret with this node knowing only a public point. This implements the BOLT standard of hashing the ECDH point, and is incompatible with ECIES.
This is a common thing to do, so create a macro.
Unfortunately, it still needs the type arg, because the paramter may
be const, and the return cannot be, and C doesn't have a general
"(-const)" cast.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
common should not include specific per-daemon files. Turns out this
caused a lot of indirect includes to be exposed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Replace `json_to_double()` (which uses `strtod(3)`) with our own
floating-point parsing function `json_to_millionths()` that
specifically expects to receive such a number that can fit in a
64 bit integer after being multiplied by 1 million.
The main piece of the code in this patch comes from
https://github.com/ElementsProject/lightning/pull/3535#discussion_r381041419
Changelog-None
Before this patch we used to send `double`s over the wire by just
copying them. This is not portable because the internal represenation
of a `double` is implementation specific.
Instead of this, multiply any floating-point numbers that come from
the outside (e.g. JSONs) by 1 million and round them to integers when
handling them.
* Introduce a new param_millionths() that expects a floating-point
number and returns it multipled by 1000000 as an integer.
* Replace param_double() and param_percent() with param_millionths()
* Previously the riskfactor would be allowed to be negative, which must
have been unintentional. This patch changes that to require a
non-negative number.
Changelog-None
Instead of making it ourselves, lightningd does it. Now we only have
two cases of failed htlcs: completely malformed (BADONION), and with
an already-wrapped onion reply to send.
This makes channeld's job much simpler.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
`getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
`getfilteredblock` in `lightningd/bitcoind`.
Didn't generally fixup inside comments and the bech32 code: reformatting that
is just anti-social.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Otherwise you can ask for a sub-millisatoshi amount, which is dumb and
violates the spec.
See-also: https://github.com/lightningnetwork/lightning-rfc/pull/736
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: We now reject invoices which ask for sub-millisatoshi amounts
This pass to json_stream helpers for commands outputs, but keeps
compatibility with existing plugins which use jout as of now, by not
starting/closing the "result"/"error" objects.
Now that we have json_stream in common/, we can move all the related
helpers from lightningd/json to common/json. This way everyone can
benefit of them (including libplugin, the plugins themselves,
potentially lightning-cli), not lightningd alone!
Note that the Makefile of the common/test/ had to be modified, because
the new helpers make use of common/wireaddr... Which turns out to
\#include <lightingd/lightningd.h> ! So we couldnt just include the .c
and add mocks if we redefined some structs (hello run-param).
This sets the nLockTime to the tip (and accordingly each input's nSequence to
0xfffffffe) for withdrawal transactions.
Even if the anti fee-sniping argument might not be valid until some time yet,
this makes our regular wallet transactions far less distinguishable from
bitcoind's ones since it now defaults to using native Segwit transactions
(like us). Moreover other wallets are likely to implement this (if they
haven't already).
Changelog-Added: wallet: withdrawal transactions now sets nlocktime to the current tip.
GCC 10 defaults to `-fno-common`. no longer automatically sharing
global variable definitions, which makes it important to define
them in only one place (otherwise there will be duplicate definition
errors). Add `extern` qualifiers where (I think) is the best place for
them.
We also update since the merged version sets feature bit 9 (as it's
supposed to now that we tied that to payment_secret).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We tag them with specific versions when they're experimental,
but do a poor job of cleaning them up (and thus ensuring they're
checked!) afterwards.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.
To resolve this:
* Introduce an error code type with a known size:
`typedef s32 errcode_t`.
* Change all error code macros to constants of type `errcode_t`.
Constants also play better with gdb - it would visualize the name of
the constant instead of the numeric value.
* Change all functions that take error codes to take the new type
`errcode_t` instead of `int`.
* Introduce towire / fromwire functions to send / receive the newly added
type `errcode_t` and use it instead of `towire_int()`.
In addition:
* Remove the now unneeded `towire_int()`.
* Replace a hardcoded error code `-2` with a new constant
`INVOICE_EXPIRED_DURING_WAIT` (903).
Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
We could use sendonion to do this, but it actually takes a different path through
pay, and I wanted to test all of it, so I made a new dev flag.
We currently get upset with the response:
lightningd/pay.c:556: payment_failed: Assertion `!hout->failcode' failed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is mainly meant as a marker so that we can later remove the code if we
decide to make the handling of custommsgs a non-developer option. It marks the
place that we would otherwise handle what in dev-mode is a custommsg.
Fixes: #3192
Changelog-Added: `waitanyinvoice` now supports a `timeout` parameter, which when set will cause the command to fail when the timeout is reached; can set this to 0 to fail immediately if no new invoice has been paid yet.
The number of outputs got updated, but the map used to calculate the
change output's location did not (still assumes only one output). This
patch fixes this to make the output map a variable size.
Changelog-Fixed: JSON API: `txprepare` no longer crashes when more than two outputs are specified
Generally I prefer structures over u8, since the size is enforced at
runtime; and in several places we were doing conversions as the code
using Sphinx does treat struct secret as type of the secret.
Note that passing an array is the same as passing the address, so
changing from 'u8 secret[32]' to 'struct secret secret' means various
'secret' parameters change to '&secret'. Technically, '&secret' also
would have worked before, since '&' is a noop on array, but that's
always seemed a bit weird.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I really want a type which means "I am a wrapped onion reply" as separate
from "I am a normal wire msg". Currently both user u8 *, and I got very
confused trying to figure out where each one was an unwrapped error msg,
or where it still needed (un)wrapping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Add "peer not connected" and "unknown peer" as error codes, so that
users can check against numeric error codes instead of textual error
messages.
Will ease https://github.com/ElementsProject/lightning/issues/3366
Changelog-None
We still close the channel if we *send* an error, but we seem to have hit
another case where LND sends an error which seems transient, so this will
make a best-effort attempt to preserve our channel in that case.
Some test have to be modified, since they don't terminate as they did
previously :(
Changelog-Changed: quirks: We'll now reconnect and retry if we get an error on an established channel. This works around lnd sending error messages that may be non-fatal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Thanks to @t-bast, who made this possible by interop testing with Eclair!
Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These used to be necessary as we could have feerate changes which
we couldn't track: now we do, we don't need these flags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is an intermediary step: we still don't save it to the database,
but we do use the fee_states struct to track it internally.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This uses the same state machine as HTLCs, but they're only
ever added, not removed. Since we can only have one in each
state, we use a simple array; mostly NULL.
We could make this more space-efficient by folding everything into the
first 5 states, but that would be more complex than just using the
identical state machine.
One subtlety: we don't send uncommitted fee_states over the wire.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).
This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
anything about "style".
The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We want to have a static Tor service created from a blob bound to
our node on cmdline
Changelog-added: persistent Tor address support
Changelog-added: allow the Tor inbound service port differ from 9735
Signed-off-by: Saibato <saibato.naga@pm.me>
Add base64 encode/decode to common
We need this to encode the blob for the tor service
Signed-off-by: Saibato <saibato.naga@pm.me>
niftynei
Rusty Russell
ZmnSCPxj jxPCSnmZ
Christian Decker
Dr. Maxim Orlovsky
Vasil Dimov
darosior
Ken Sedgwick
Wladimir J. van der Laan
Saibato