Commit Graph

272 Commits

Author SHA1 Message Date
Rusty Russell 11c21f97a6 devtools/decodemsg: don't crash if we have multiple tlv options.
We call tal_count(msg) after we've moved msg, and that causes an abort.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-07 12:48:13 -05:00
Vasil Dimov 3ce0552dd4 build: use locale-independent sort for mocks
Use `LC_ALL=C sort` instead of `sort` so that mocks get sorted in
the same way on all developers' environments.

Re-record the result of `make update-mocks`.

Changelog-None
2020-04-07 13:52:48 +09:30
Rusty Russell d9fc99ea39 channeld: simplify loading of pre-existing HTLCs.
We currently abuse the added_htlc and failed_htlc messages to tell channeld
about existing htlcs when it restarts.  It's clearer to have an explicit
'existing_htlc' type which contains all the information for this case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-04 16:08:49 +10:30
Rusty Russell dd690553b8 channeld: handle onion messages.
We do most of the decoding here, and just hand the results to lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-02 14:32:38 +10:30
Rusty Russell f76ab93df6 EXPERMENTAL_FEATURES: Import onion message types.
This tracks https://github.com/lightningnetwork/lightning-rfc/pull/759

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-02 14:32:38 +10:30
Rusty Russell c95e58ad4b subdaemons: initialize feature routines with explicit feature_set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Vasil Dimov 89ceb273f5 wire: remove towire_double()
Before this patch we used to send `double`s over the wire by just
copying them. This is not portable because the internal represenation
of a `double` is implementation specific.

Instead of this, multiply any floating-point numbers that come from
the outside (e.g. JSONs) by 1 million and round them to integers when
handling them.

* Introduce a new param_millionths() that expects a floating-point
  number and returns it multipled by 1000000 as an integer.

* Replace param_double() and param_percent() with param_millionths()

* Previously the riskfactor would be allowed to be negative, which must
  have been unintentional. This patch changes that to require a
  non-negative number.

Changelog-None
2020-02-27 09:07:04 +10:30
Ken Sedgwick 5c8f881a75
hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation.
Changelog-Added: hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation by signing daemon.
2020-02-04 10:40:43 +10:30
Vasil Dimov 0e3fb5e590 build: use [[:space:]] instead of \s in regex
The former is defined by IEEE Std 1003.2 ("POSIX.2"), the latter does
not work on FreeBSD.

Changelog-None
2020-02-03 15:38:11 +00:00
Vasil Dimov 55173a56b7 Use dedicated type for error codes
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.

To resolve this:

* Introduce an error code type with a known size:
  `typedef s32 errcode_t`.

* Change all error code macros to constants of type `errcode_t`.
  Constants also play better with gdb - it would visualize the name of
  the constant instead of the numeric value.

* Change all functions that take error codes to take the new type
  `errcode_t` instead of `int`.

* Introduce towire / fromwire functions to send / receive the newly added
  type `errcode_t` and use it instead of `towire_int()`.

In addition:

* Remove the now unneeded `towire_int()`.

* Replace a hardcoded error code `-2` with a new constant
  `INVOICE_EXPIRED_DURING_WAIT` (903).

Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
2020-01-31 06:02:47 +00:00
Christian Decker 5325ff6352 json-rpc: Don't let users send messages that are handled internally
We cannot let users use `sendcustommsg` to inject messages that are handled
internally since it could result in our internal state tracking being borked.
2020-01-28 23:50:52 +01:00
Rusty Russell aa6aad0131 common: add struct onionreply
I really want a type which means "I am a wrapped onion reply" as separate
from "I am a normal wire msg".  Currently both user u8 *, and I got very
confused trying to figure out where each one was an unwrapped error msg,
or where it still needed (un)wrapping.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Vasil Dimov fb7c006187 wire: add towire_int() and use it in connectd
Add towire_int() and fromwire_int() functions to "(de)serialize"
"int". This will only work as long as both the caller of towire_int()
and the caller of fromwire_int() use the same in-memory representation
of signed integers and have the same sizeof(int).

Changelog-None
2020-01-21 16:59:18 +01:00
Christian Decker 2d45b13088 hsmtool: Make the backup copy in the same directory as the original
TIL: `rename` doesn't like its source and target to be on different
partitions. This was causing the `hsmtool` tests to fail whenever we ran them
on a different partition than the lightning-dir (e.g., `/dev/shm` for faster
testing), because we made the backup copy in the current working directory.

This changes this and creates the backup next to the original file, which has
a reasonable chance to be on the same partition.

Changelog-Changed: hsmtool: The `hsmtool` now creates its backup copy in the same directory as the original `hsm_secret` file.
2020-01-13 08:55:32 +08:00
Vasil Dimov b2c4d5e952 build: fix developer mode compilation on FreeBSD
Recent FreeBSD versions use LLVM's lld as a linker, not the GNU ld.
Their behavior slightly differs, so adapt the build system to handle
either one.

* The LLVM's linker prints "undefined symbol:" instead of
  "undefined reference to". Tweak tools/mockup.sh to also look for that
  message.

* The LLVM's linker may only print the first dozen errors (omitting
  the rest to avoid flooding the screen). tools/update-mocks.sh relies
  on getting all errors as it extracts the missing symbols' names from
  the error output and creates mocks for them. Detect if errors were
  omitted and re-run, telling the linker to not omit any. The GNU linker
  does not support -error-limit=0, so unfortunately we can't just run
  with that option unconditionally from the first attempt.

* Nit: FreeBSD's sed(1) prints "t" for "\t" instead of a horizontal tab.
  Use a verbatim tab in the command, instead of "\t" which works on
  both.

Changelog-Fixed: Developer mode compilation on FreeBSD.
2020-01-02 16:56:20 +01:00
Rusty Russell eb6a768741 tools: don't use bytelength as array length for arrays in TLVs.
This matters now we have an array in tlv_init_tlvs!  We were overallocating
in fromwire by 32x!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-13 16:36:40 +01:00
Rusty Russell 72aa315b5e lightningd: save the fee_states into the database.
This is the final step: we pass the complete fee_states to and from
channeld.

Changelog-Fixed: "Bad commitment signature" closing channels when we sent back-to-back update_fee messages across multiple reconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
lisa neigut dd34cccbea hsmtool: fixup miscount on params
assuming argv is zero indexed, we should check that the argc count
is at least as big as the desired args
2019-12-11 20:41:09 +01:00
Christian Decker 626675c83c tlv: Migrate tlv serialization to typesafe function 2019-12-03 00:37:15 +00:00
Christian Decker e12b5c3824 tlv: Add a typesafe serialization function for tlv namespaces
This is the counterpart to the typesafe deserialization function implemented
in an earlier commit.
2019-12-03 00:37:15 +00:00
Christian Decker 5a78671d9f wire: Remove unused fromwire_tlvs
We are now using the typesafe variant everywhere.
2019-12-03 00:37:15 +00:00
Christian Decker 69c17d2d31 wire: Let the TLV _is_valid function actually return validity
I got this one wrong myself, since the function name implied a boolean
result. So I changed it to take the optional err_index as argument.
2019-12-03 00:37:15 +00:00
darosior 36e8d83a62 tools/hsmtool: correct bad access to argv[] 2019-11-29 21:19:05 +01:00
darosior e5e4958909 tool/hsmtool: add a 'guesstoremote' command
This, in the case of data loss on a channel with `option_static_remotekey`
negotiated, allows to likely (if the dbid is not unreasonable) recover
the funds from a remote unilateral close just with the hsm_secret.

Changelog-added: A new command, 'guesstoremote', is added to the hsmtool. It is meant to be used to recover funds after an unilateral close of a channel with `option_static_remotekey` enabled.
2019-11-29 21:19:05 +01:00
darosior 670f92002d tools/gen/impl_template: correct one-line for-loops indentation 2019-11-26 21:30:25 +01:00
darosior 5fb8e0aade tools/gen/impl_template: correct tlvs fromwire's for-loop 2019-11-26 21:30:25 +01:00
Christian Decker 2519f934aa tlv: Add validity check codegen for the tlv namespaces
Since the parser itself just parses and doesn't include validation anymore we
need to put that functionality somewhere. The validation consists of enforcing
that the types are in monotonically increasing order without duplicates and
that for the even types we know how to handle it.
2019-11-22 04:40:25 +00:00
Christian Decker 5794c83b4d tlv: Add typesafe fromwire codegen for TLV namespaces
We were weaving in and out of generic code through `fromwire_tlvs` with custom
parameters defining the types in that namespace. This hard-wires the parser
with the namespace's types. Slowly trying to deprecate `fromwire_tlvs` in
favor of this typesafe variant.
2019-11-22 04:40:25 +00:00
Christian Decker 2255024ead tlv: Add raw fields so we can store unknown fields as well 2019-11-22 04:40:25 +00:00
darosior de91eda36d hsmtool: add a tool to dump commitment points and secrets
This takes a dbid, a "depth" (how many points to dump), the hsm_secret
path, and a potential password to dump informations about all
commitments until the depth.

Co-Authored-By: Sjors Provoost <sjors@sprovoost.nl>
2019-11-12 05:29:19 +00:00
darosior dccad7784f tools/hsmtool: add a tool for encrypting hsm_secret 2019-11-12 05:29:19 +00:00
darosior 04762a1d38 tools/hsmtool: add a tool for decrypting hsm_secret
A general one, for all things hsm_secret.
2019-11-12 05:29:19 +00:00
lisa neigut 6f85088b2b tools: add handy debug error 2019-10-10 05:57:45 +00:00
lisa neigut 3f1f075421 tools: add ability to wrap wire messages with ifs
Makes it possible to hide wire messages behind EXPERIMENTAL_FEATURES
flag.
2019-10-10 05:57:45 +00:00
Christian Decker ef7a63d8f8 elements: Move from a global is_elements to a global chainparams
We now have a pointer to chainparams, that fails valgrind if we do anything
chain-specific before setting it.

Suggested-by: Rusty Russell <@rustyrussell>
2019-10-03 04:32:57 +00:00
Rusty Russell ccbc46a476 check-bolt: escape { and }.
These are special in extended regexs, and so we fail to match once we fix
the BOLT comment in common/test/run-bigsize.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
trueptolemy 090a43fd3d gossip: Add the `struct exclude_entry` and `enum exclude_entry_type` 2019-09-16 12:22:06 +08:00
Rusty Russell 6a293fd135 tools/generate-wire.py: accept multiple comma-separated options.
Somehow this change got lost, but it's needed for option_static_remotekey,
to quote gen_peer_wire_csv:

    msgtype,channel_reestablish,136
    msgdata,channel_reestablish,channel_id,channel_id,
    msgdata,channel_reestablish,next_commitment_number,u64,
    msgdata,channel_reestablish,next_revocation_number,u64,
    msgdata,channel_reestablish,your_last_per_commitment_secret,byte,32,option_data_loss_protect,option_static_remotekey
    msgdata,channel_reestablish,my_current_per_commitment_point,point,,option_data_loss_protect

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 51d7a1404f tools/check-bolt: don't get confused by 'BOLT #1' in middle of a comment.
Insist it be prefixed with '* '.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 17541e22a3 update-mocks: allow a function to specify that mock should not abort.
We're going to add memleak_add_helper_, and we want the mock to be a
noop, not an abort.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
trueptolemy cdcafdaf74 API: `txprepare` now support mutiple outputs 2019-09-05 16:05:36 -05:00
Rusty Russell 11833192a1 tools: fix Makefile warning
Makefile:424: warning: overriding recipe for target 'clean'
    tools/Makefile:12: warning: ignoring old recipe for target 'clean'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
darosior 77f34fad2b doc/Makefile: adapt 'check-manpages' to markdown 2019-08-22 01:35:01 +00:00
Rusty Russell 9dfc854d94 tools/Makefile: put all tools/ stuff here.
Generally a Makefile should control make within that directory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell 278b12982d tools/test/Makefile: disable unless we're in DEVELOPER mode.
Because it required update-mocks, which is a hack which relies on the
format of linker errors (!) I'd prefer to make this --enable-developer
only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell c74c1e56cc tools/test/Makefile: fix always-on generation of tools/test/gen_test.c
We need to tell it that the tmp file is an intermediate, so doesn't need
remaking if it doesn't exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
lisa neigut e99720344e wire-gen: ensure that tlv messages are correctly ordered
Our TLV serializer relies on TLV outputs to be ordered by type
number. Prior to this commit we relied on 1) the ordering in the
RFC to be correct and 2) users to be using a version of Python that
respects stable ordering of dicts (i.e. Python 3.7+)

Instead of relying on these implicitly, we now explicitly sort messages
by type number when the TLV sets.

Resolves #2956.

Thanks-To: @ScottTre for the sort function
Reported-By: @ZmnSCPxj
2019-08-21 03:53:54 +00:00
Rusty Russell 04a57ae4af doc/MAKING-RELEASES.md: update.
1. These days we delete the [Unreleased] tag during rcs.
2. Make sure we test the release build process during rc1, since I
   screwed that up last release.
3. Add a section on rc2, etc.
4. Do final release via a github PR, since I screwed that up on the
   prior release.
5. Update `tools/build-release.sh` and instructions to show that we now
   make a reproducible build for Ubuntu 18.04 x86-64.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-19 03:43:13 +00:00
Rusty Russell 303c221641 tools/test/Makefile: we depend on tools/test/gen_test.h
This is because update-mocks actually tries to build it to get link
errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-19 03:43:13 +00:00
Rusty Russell c0ba5879df tools/test: can't generate mocks until submodules checked out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-17 14:08:43 +09:30