Commit Graph

183 Commits

Author SHA1 Message Date
Rusty Russell 3c8049f32c bitcoin/psbt: psbt_input_add_unknown/psbt_output_add_unknown needs a tal ctx.
Since it allocates something, it needs a context (used in the next patch!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
niftynei b44e36b99e spec-update: get rid of max-witness-len
We can use a fixed value and close the channel if they don't cover their
amount; this wasn't really helping with anything other than setting a
floor for an expected feerate
2020-09-09 19:54:20 +09:30
niftynei c50f377a85 psbt: pull out changeset logic into common, update API
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.

Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei 5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei 303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
ZmnSCPxj jxPCSnmZ 34bf0133f2 onchaind/onchaind_wire.csv: Propagate minimum relay fee to onchaind. 2020-09-09 12:38:19 +09:30
Rusty Russell ea819107eb common: remove funding_tx.
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell 39e4796ae3 json_command: command_fail_badparam helper.
It's common to want to complain a token is not what we expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 09:46:37 +09:30
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell daba3e7deb common/gossmap: helper to map the gossip store.
I went overboard on optimization.  I am so sorry:
1. Squeezed channel min/max into 16 bits.
2. Uses mmap and leaves node_ids in the file.
3. Uses offsets instead of pointers where possible.
4. Uses custom free-list to allocate inside arrays.
5. Ignores our autogenerated marshalling code in favor of direct derefs.
6. Carefully aligns everything so we use minimal ram.

The result is that the current gossip_store:
 - load time (-O3 -flto laptop): 40msec
 - load time (-g laptop i.e. DEVELOPER=0): 60msec
 - load time (-O0 laptop i.e. DEVELOPER=1): 110msec
 - Total memory: 2.6MB:
   - 1.5MB for the array of channels
   - 512k for the channel htable to map scid -> channel.
   - 320k for the node htable to map nodeid -> node.
   - 192k for the array of channels inside each node
   - 94k for the array of nodes

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell 7ad8fde060 bolt11: update ctlv expiry, always write it.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: protocol: bolt11 invoices always include CLTV fields (see lightning-rfc#785)
2020-08-24 20:55:47 +09:30
Rusty Russell 007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell 3ed03f9c8f common: make json_parse_input API retry friendly.
The jsmn parser is a beautiful piece of code.  In particular, you can parse
part of a string, then continue where you left off.

We don't take advantage of this, however, meaning for large JSON objects
we parse them multiple times before finally having enough to complete.

Expose the parser state and tokens through the API, so the caller can pass
them in repeatedly.  For the moment, every caller is allocates each time
(except the unit tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell 80b2e1e298 common: add simple json parse wrapper for the complete cases.
We're going to change the API on the more complete JSON parser, so
make and use a simple API for the easy cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell a0ede40743 bitcoin: make psbt_append_input more powerful.
It can now handle all the metadata as well as the base tx input.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-20 21:14:12 +09:30
Rusty Russell 172b9d7ae3 bitcoin: add nlocktime arg to create_psbt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-20 21:14:12 +09:30
niftynei 722fa1df6c psbt-utils: ignore the 'unknown' map for input + output comparison
There's no stable ordering on unknown serialization, so linearizing
identical but mis-ordered unknown data will lead to 'wrong' results.
Instead, we just ignore any data that's in the psbt unknown struct.

There's probably also problems here with other PSBT maps. Really, this
needs a finer grained comparison function .... fuck
2020-08-18 11:08:44 +09:30
niftynei 32e40b2cfc psbt-common: unit test for diff/sort/add serial etc 2020-08-18 11:08:44 +09:30
Rusty Russell ffbb409b47 amount: use initializers everywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell 36d43b871f amount: add amount_msat and amount_sat initializers.
Generally, importing amounts needn't be checked, and it cuts down on
the warnings we get.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
niftynei bc5a817100 elements: convenience methods for dealing with assets
We don't preserve detailed asset information at the moment, so provide a
way to convert from a sat to an amount_asset struct.

We also need a way to convert from an 'amount_asset' to a 'value' for
elements, which for explicit (i.e. non-blinded) asssets is a 0x01 prefix
plus the big-endian encoded value.
2020-07-13 11:37:24 +09:30
Rusty Russell 869fa082d4 common/json_tok: expose param_txid.
Move it out of lightningd/ so plugins can use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell 731e037b36 change_amount: routine to determine if change output is worthwhile.
This can be used by plugins which create their own txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell 31b2865791 common/json_tok: expose routines to parse addresses.
These are currently inside lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell a9427f1a8d bitcoin/feerate: new exposure for feerate parsing outside lightningd.
This exposes the numeric part of param_feerate() as param_feerate_val().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
niftynei 2e9c387f45 coin_moves: update withdrawal logic to account for 'variable owner' txs
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)

Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
2020-06-29 16:10:05 +02:00
Rusty Russell 93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
ZmnSCPxj jxPCSnmZ e8936f9d23 common/json.c: Check that JSMN result is well-formed.
xref: https://lists.ozlabs.org/pipermail/c-lightning/2020-June/000188.html

Changelog-Fixed: Reject some bad JSON at parsing.
2020-06-19 11:46:18 +09:30
niftynei dc868630a8 tx-psbt: pass in the witness script (if known) when adding an input
Update the `bitcoin_tx_add_input` interface to accept a witness script
and or scriptPubkey.

We save the amount + witness script + witness program (if known) to
the PSBT object for a transaction when creating an input.
2020-06-11 13:13:13 +02:00
niftynei 85f395f7d4 utxo: fill in scriptPubkey to NULL
We need this so we can access it when populating bitcoin inputs in the
next commit
2020-06-11 13:13:13 +02:00
niftynei 559f88faa1 psbt: add serialize to/from wire for psbts 2020-05-21 18:45:07 +09:30
Rusty Russell b920d4d21b wire: make fromwire_fail return non-const, use it more
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case.  Use that more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-20 11:36:13 +02:00
niftynei fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
Rusty Russell cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 197d1bcef2 wire: move towire/fromwire_short_channel_id out to bitcoin/short_channel_id.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 13356b75bf wire: move towire/fromwire_amount from wire/ into common/amount.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 4dbfce6057 wire: move towire/fromwire_preimage out to bitcoin/preimage.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 27220646c3 common/wire: move bitcoin-specific marshalling functions into bitcoin files.
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those.  Now we do, so
there's less excuse for this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell f77d70d546 common/json: move rest of bitcoin/lightning-specific json functions to json_helpers.
This dramatically reduces the linking requirements of lightning-cli.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 6635fe12e4 common/test/run-features.c: fix cppcheck 1.90 false positive.
We only use sizeof(f1->bits).

```
common/test/run-features.c:84:36: error: Uninitialized variable: f1 [uninitvar]
 for (size_t i = 0; i < ARRAY_SIZE(f1->bits); i++) {
                                   ^
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
lisa neigut 39d5117210 utxo: add scriptSig + scriptPubkey field
Allow the utxo object to bear the scriptSig and scriptPubKey
2020-05-07 08:43:00 +09:30
Rusty Russell 3b4a06f52b common: generalize ecdh function.
common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV.  But how it does ecdh is daemon-specific.

We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh().  This just makes it more general.

So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-04 16:08:49 +10:30
Rusty Russell cf43e44378 common/features: don't use internal global.
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
darosior 13fbce90ca common/amount: accept 0msat in parse_amount_sat()
Changelog-fixed: bcli now handles 0msat outputs in gettxout.

Co-authored-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-01 10:20:03 +10:30
Rusty Russell c95e58ad4b subdaemons: initialize feature routines with explicit feature_set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell afb76392e4 common/features: use bitmaps internally, have explicit init function.
This is to prepare for dynamic features, including making plugins first
class citizens at setting them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell 8c984fbf1d common: make sphinx.c use hmac.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-25 14:26:28 +10:30
Christian Decker 59b6159e56 sphinx: Functions to enable RV mode and serialize compressed onions
We will later use these to generate RV compressed onions and to opt into the
rendezvous style generation.
2020-03-12 10:25:01 +10:30