Commit Graph

863 Commits

Author SHA1 Message Date
Rusty Russell 5eb121ffc3 common/amount: fix OpenBSD compile warning.
```
cc common/amount.c
common/amount.c:306:15: error: implicit conversion from 'unsigned long long' to
      'double' changes value from 18446744073709551615 to 18446744073709551616
      [-Werror,-Wimplicit-int-float-conversion]
        if (scaled > UINT64_MAX)
                   ~ ^~~~~~~~~~
/usr/include/sys/stdint.h:123:21: note: expanded from macro 'UINT64_MAX'
                                ^~~~~~~~~~~~~~~~~~~~~
1 error generated.
gmake: *** [Makefile:254: common/amount.o] Error 1
bsd$
```

Fixes: #4044
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-14 03:57:17 +09:30
Vincenzo Palazzo 9fbeb9bcdc delpay: code style changes and fixed docs
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
Rusty Russell 976c6e5c83 json_add_tok: simplify, don't try to canonicalize.
I'm not sure why we were turning strings into bools, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-10 15:28:50 +09:30
niftynei b44e36b99e spec-update: get rid of max-witness-len
We can use a fixed value and close the channel if they don't cover their
amount; this wasn't really helping with anything other than setting a
floor for an expected feerate
2020-09-09 19:54:20 +09:30
niftynei aef5a5a0e5 psbt: psbt_has_serial_id -> psbt_find_serial_id
Cleans up some awkward spots in the code, makes the footprint a bit
neater

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei c50f377a85 psbt: pull out changeset logic into common, update API
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.

Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei 5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei 303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
niftynei 169b7817dc psbt: only compare a subset of psbt input/output fields
At some point, it's ok to add more extra info to a psbt and still not
have that be counted as 'diff'd.
2020-09-09 19:54:20 +09:30
niftynei 9c89184c1f dualfund: add feature flag for dual-funding
turn off until we're ready to test both sides
2020-09-09 19:54:20 +09:30
niftynei 42a38b9808 channel_id: add v2 channel_id derivtion helper
v2 of channel open uses the channel revocation basepoints to calculate
the channel_id, instead of the funding_txid + outnum

Moving away from the funding_txid opens the way for splicing + rbf
2020-09-09 19:54:20 +09:30
niftynei 72a098a4ad openingd: pull out check_configs into a common place
We'll re-use it for dualopend!
2020-09-09 19:54:20 +09:30
niftynei 864f2f3e21 channel_id: save to database, dont derive from funding_txid
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.

includes a migration for existing channels
2020-09-09 19:54:20 +09:30
niftynei 6cfac5d2a6 psbt: enforce const on max_witness_len 2020-09-09 19:54:20 +09:30
niftynei 470f5fabc9 amount: fix typo in parse_amount_msat doc 2020-09-09 19:54:20 +09:30
Rusty Russell 0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes.
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
ZmnSCPxj jxPCSnmZ 34bf0133f2 onchaind/onchaind_wire.csv: Propagate minimum relay fee to onchaind. 2020-09-09 12:38:19 +09:30
Rusty Russell e3219d3aa0 utxo: expose is_reserved, make enum constants upper case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 438953b8f0 utxo: make reserved_til a u32 not a ptr, now it's compsulory.
It's 0 for old dbs, which is the same as "available".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell ea819107eb common: remove funding_tx.
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 83298c030a wallet: switch over to withdraw in module, remove lots of unused code.
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.

This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined.  Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
2020-09-08 10:14:42 +09:30
Rusty Russell c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell d8e8426b52 Makefile: remove EXPERIMENTAL_FEATURES marker from generated files
We're going to make experimental versions of these completely separate files.

Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn.  We can always force-rebuild when we change a rule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell 82f2f43425 Generated files: more merge fallout.
We changed the generation templates!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 11:10:21 +09:30
Rusty Russell 39e4796ae3 json_command: command_fail_badparam helper.
It's common to want to complain a token is not what we expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 09:46:37 +09:30
Rusty Russell be0038e49b Makefile: include common objects in ALL_C_SOURCES.
They didn't have correct dependencies without this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 09:30:25 +09:30
Rusty Russell 3c6af3efb4 Makefile: commit and preserve all the wiregenerated files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: we no longer require extra Python modules to build.
2020-08-31 21:33:26 -05:00
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 3ae4ce736d Apply @cdecker typo suggestions from code review
Co-authored-by: Christian Decker <decker.christian@gmail.com>
2020-08-28 10:56:50 +09:30
Rusty Russell 6815a1d9a2 common/route: routing helpers built on gossmap and dijksra.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell 59cc2f7559 common/dijkstra: routine to calculate shortest/cheapest path.
The user supplies callbacks to do channel selection and comparison.
Note that this continues to map the entire network; not just to the
source, for use with random routing.

Benchmarks: (using current mainnet gossip store)
	/devtools/route gossip-store-2020-07-27 all 03c981ed4ad15837f29a212dc8cf4b31f274105b7c95274a41449bf496ebd2fe10 | grep 'Time to find path'

With nothing (i.e. DEVELOPER build)
	Averages 17ms

With -Og (i.e. standard non-DEVELOPER build)
	Averages 14ms

With -O3 -flto:
	Averages 4ms

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell daba3e7deb common/gossmap: helper to map the gossip store.
I went overboard on optimization.  I am so sorry:
1. Squeezed channel min/max into 16 bits.
2. Uses mmap and leaves node_ids in the file.
3. Uses offsets instead of pointers where possible.
4. Uses custom free-list to allocate inside arrays.
5. Ignores our autogenerated marshalling code in favor of direct derefs.
6. Carefully aligns everything so we use minimal ram.

The result is that the current gossip_store:
 - load time (-O3 -flto laptop): 40msec
 - load time (-g laptop i.e. DEVELOPER=0): 60msec
 - load time (-O0 laptop i.e. DEVELOPER=1): 110msec
 - Total memory: 2.6MB:
   - 1.5MB for the array of channels
   - 512k for the channel htable to map scid -> channel.
   - 320k for the node htable to map nodeid -> node.
   - 192k for the array of channels inside each node
   - 94k for the array of nodes

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell 496c0dd1e6 common/random_select: central place for reservoir sampling.
Turns out we can make quite a simple API out of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell 12d0d5c185 amount: cleanup usage.
We've got some recently-added primitives which help.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 18:16:28 +02:00
Rusty Russell 3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
niftynei 6f114e06a6 bugfix: save max_witness_len as big-endian 2020-08-25 12:34:56 +09:30
niftynei df7c122cb8 json: add "json-to-psbt" helper
Pull a PSBT out of a json buffer
2020-08-25 12:34:56 +09:30
Rusty Russell 57dd5be2fd gossipd: prune channels unless *both* peers have refreshed.
See https://github.com/lightningnetwork/lightning-rfc/pull/767

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: channels now pruned after two weeks unless both peers refresh it (see lightning-rfc#767)
2020-08-24 20:55:47 +09:30
Rusty Russell 7ad8fde060 bolt11: update ctlv expiry, always write it.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: protocol: bolt11 invoices always include CLTV fields (see lightning-rfc#785)
2020-08-24 20:55:47 +09:30
Rusty Russell 6e4e267f2c doc: update BOLT now option_anchor_outputs is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell 007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell 3ed03f9c8f common: make json_parse_input API retry friendly.
The jsmn parser is a beautiful piece of code.  In particular, you can parse
part of a string, then continue where you left off.

We don't take advantage of this, however, meaning for large JSON objects
we parse them multiple times before finally having enough to complete.

Expose the parser state and tokens through the API, so the caller can pass
them in repeatedly.  For the moment, every caller is allocates each time
(except the unit tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell 80b2e1e298 common: add simple json parse wrapper for the complete cases.
We're going to change the API on the more complete JSON parser, so
make and use a simple API for the easy cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell a0ede40743 bitcoin: make psbt_append_input more powerful.
It can now handle all the metadata as well as the base tx input.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-20 21:14:12 +09:30
Rusty Russell 172b9d7ae3 bitcoin: add nlocktime arg to create_psbt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-20 21:14:12 +09:30
niftynei 6371a49ee3 psbt: sort maps before serializing, to stabilize comparison
Our psbt input/output comparison functions use serialization to compare
the things, but if there's a map with things in it and the map isn't
sorted exactly the same, it's highly likely you'll mark an identical inputs
as different.

To fix this, we sort all the input/output maps before linearizing them.
2020-08-18 11:08:44 +09:30
niftynei 9b1b05617f psbt: use platform-independent serialization for psbt fields
portability for the win

Suggested-By: @jgriffiths
2020-08-18 11:08:44 +09:30