Commit Graph

34 Commits

Author SHA1 Message Date
Rusty Russell f4e7ed179f devtools/mkencoded: tool to encode short_channel_id / other for testing
In particular, it does zlib.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell 8ef996d56e devtools/mkgossip: tool to create gossip msgs for testing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell aca2e4f722 common/memleak: add dynamic hooks for assisting memleak.
Rather than reaching into data structures, let them register their own
callbacks.  This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.

Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell a134062f98 bolt11: handle `9` fields for new features.
This implements https://github.com/lightningnetwork/lightning-rfc/pull/656

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-05 23:39:05 -05:00
Rusty Russell 9dd314226d devtools/mkclose: make a mutual close transaction.
Guess who's been writing gossip protocol tests?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-26 08:44:22 +00:00
Rusty Russell f1e84b3d99 devtools: make clean should remove devtools/gen_print_onion_wire.[c,h,o]
Reported-by: @JavierRSobrino
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-13 05:12:10 +00:00
Rusty Russell 3fa375881a bigsize: make it a proper first-class type.
It doesn't belong in bitcoin, and should not be confused with varint_t.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Christian Decker 660921a9dd sphinx: Introduce a `runtest` command to the onion tool
The `runtest` command takes a JSON onion spec, creates the onion and decodes
it with the provided private keys. It is fully configurable and can be used
for the test-vectors in the spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Rusty Russell 2e68b88be8 tools: fix tlv generation
We need to hand -s to both header and body generation, or neither:

wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here

We also need it for printwire, otherwise we get static unused functions for subtypes:

devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
lisa neigut 281b4c241e bolt-gen: fixup the devtool/decodemsg printing facility
Fixup TLV handling in the bolt printing utility, `devtools/decodemsg`
2019-07-24 02:52:53 +00:00
Rusty Russell 12d8be4fdc devtools: have mkcommit support HTLCs.
This allows for complete channel simulation, including HTLC
transactions, but means we use higher-level primitives to
make the easy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell 8b09a9ad06 devtools: add mkfunding and mkcommit.
These utilities allow us to create valid test txs and information given both
sides' complete set of secrets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
lisa neigut 5c07afac7d bolt: update to BOLT spec changes (extract format + type specifications)
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.

this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
2019-07-16 06:10:58 +00:00
Rusty Russell 38d2899fbb common/per_per_state: generalize lightningd/peer_comm Part 1
Encapsulating the peer state was a win for lightningd; not surprisingly,
it's even more of a win for the other daemons, especially as we want
to add a little gossip information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Joe Netti 902bb22a92 devtools/create-gossipstore: cleanups
added sanity check to make sure scid of csv is the same as scid in gossip.
Revised style, mem allocation, and error checks

[ Minor fixups, and updated benchmark script -- RR ]

With data.tar.gz: 456609740 Apr  2 12:33

store_load_msec:35300-42354(37118.2+/-2.7e+03)
vsz_kb:582832
store_rewrite_sec:12.700000-13.430000(12.988+/-0.27)
listnodes_sec:3.000000-3.160000(3.076+/-0.057)
listchannels_sec:30.790000-31.690000(31.03+/-0.34)
routing_sec:0.00
peer_write_all_sec:63.640000-67.860000(66.294+/-1.4)
2019-04-24 13:46:39 -05:00
Rusty Russell a2fa699e0e Use node_id everywhere for nodes.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.

We didn't get much space saving in gossipd, even though we should save
24 bytes per node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Joe Netti 294394215e create-gossipstore.c can read scid -> satoshis csv file. The csv is in the format scid ,satoshis where there is a black space after scid. Made a header file that contains a struct. Modified makefile. Added cmdline arg --scidfile /path/to/csv and made the constant capacity command optional. create-gossipstore prints stats at the end. 2019-04-08 04:41:43 +00:00
Rusty Russell 3d8c1f0c02 devtools/create-gossipstore: tool to create a gossip_store file from stream of gossip.
The gossip is expected to be in format:

    16-bit-big-endian-length
    [gossip message]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 04:41:43 +00:00
Rusty Russell 7fad7bccba common/amount: new types struct amount_msat and struct amount_sat.
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.

The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 59febcb968 sphinx: explain why parse_onionpacket fails.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Rusty Russell 0925daa087 gossipwith: simple tool to snarf gossip from a node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 317a830e94 devtools: dump-gossipstore.
Not very useful by itself, but when combined with decodemsg it can tell
us quite a bit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 00:39:06 +00:00
Rusty Russell 93cf28553d devtools/decodemsg: add --onion option for decoding onion errors.
This requires a tweak to generate-wire.py too, since it always called the
top-level routine 'print_message'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
ueno 21e61a5232 fix: make devtools-clean 2018-07-25 15:44:25 +02:00
Rusty Russell 01c02fd617 devtools/decodemsg: decode encoded_short_ids.
$ devtools/decodemsg 010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f000000000000ffff0100160178da6360486760606400824c285d00a60111710144
$ devtools/decodemsg 010506226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00110000006700000100000000690000010000

Before:
    WIRE_REPLY_CHANNEL_RANGE:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    first_blocknum=0
    number_of_blocks=65535
    complete=1
    encoded_short_ids=[0178da6360486760606400824c285d00a60111710144]
    WIRE_QUERY_SHORT_CHANNEL_IDS:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    encoded_short_ids=[0000006700000100000000690000010000]

After:

    WIRE_REPLY_CHANNEL_RANGE:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    first_blocknum=0
    number_of_blocks=65535
    complete=1
    encoded_short_ids=[ (ZLIB) 103:1:0 105:1:0 112:1:0 ]
    WIRE_QUERY_SHORT_CHANNEL_IDS:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    encoded_short_ids=[ (UNCOMPRESSED) 103:1:0 105:1:0 ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell 0bd82a8138 devtools/decodemsg: decode node_announcement.addresses
$ ./devtools/decodemsg 01014bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e97fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab1300005b3315de0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c035180266e4e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000004d010102030404d202000000000000000000000000000000002607039216a8b803f3acd758aa260704e00533f3e8f2aedaa8969b3d0fa03a96e857bbb28064dca5e147e934244b9ba50230032607

Before:
    WIRE_NODE_ANNOUNCEMENT:
    signature=304402204bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e902207fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab13
    features=[]
    timestamp=1530074590
    node_id=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
    rgb_color=[0266e4]
    alias=[ナンセンス 1杯 e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000 ]
    addresses=[010102030404d202000000000000000000000000000000002607039216a8b803f3acd758aa260704e00533f3e8f2aedaa8969b3d0fa03a96e857bbb28064dca5e147e934244b9ba50230032607]

After:
    WIRE_NODE_ANNOUNCEMENT:
    signature=304402204bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e902207fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab13
    features=[]
    timestamp=1530074590
    node_id=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
    rgb_color=[0266e4]
    alias=[ナンセンス 1杯 e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000 ]
    addresses=[ 1.2.3.4:1234 [::]:9735 silkroad6ownowfk.onion:9735 4acth47i6kxnvkewtm6q7ib2s3ufpo5sqbsnzjpbi7utijcltosqemad.onion:9735 ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell c02ff11506 print_wire: hand field names to print routines.
This lets us override how we print them.

Also, add dependency on header for devtools/Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell 5aa1f37f07 devtools/Makefile: add devtools/onion.c to DEVTOOLS_TOOL_SRC
This way the object file correctly depends on external headers.  Currently
a parallel build on a clean tree can give:

```
In file included from ./common/sphinx.h:6:0,
                 from devtools/onion.c:5:
./bitcoin/pubkey.h:8:10: fatal error: secp256k1.h: No such file or directory
 #include <secp256k1.h>
          ^~~~~~~~~~~~~
compilation terminated.
<builtin>: recipe for target 'devtools/onion.o' failed
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Christian Decker c635396766 common: Moving some bech32 related utilities to bech32_util
These were so far only used for bolt11 construction, but we'll need them for the
DNS seed as well, so here we just pull them out into their own unit and prefix
them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 12:34:55 +02:00
Christian Decker ff6d5e896a sphinx: Add a dependency form the onion tool to the ccan config
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-09 17:57:55 +01:00
Christian Decker 08bfb740f5 onion: Move cli onion tool to devtools/onion
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-09 17:57:55 +01:00
Rusty Russell fff7dd0826 devtools/decodemsg: new tool.
$ ./devtools/decodemsg 00110000000000000000000000000000000000000000000000000000000000000000000e496e7465726e616c206572726f72
WIRE_ERROR:
channel_id=0000000000000000000000000000000000000000000000000000000000000000
data=[Internal error]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 00:57:10 +00:00
Rusty Russell 1648eb548a devtools/bolt11-cli: simple helper to decode bolt11.
Can be extended to encode later, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00