Commit Graph

7986 Commits

Author SHA1 Message Date
Christian Decker ce0025c95c paymod: Implement bubbling results up the hierarchy of payments
A payment is considered finished if it is in a final state (success or
failure) or all its sub-payments are finished. If that's the case we notify
`payment_finished` and bubble up through `payment_child_finished`, eventually
bubbling up to the root, which can then report success of failure back to the
RPC command that initiated the whole process.
2020-07-01 12:19:02 +02:00
Christian Decker 5c32e33aab paymod: Add a retry modifier that retries payments on failure
This is likely a bit of overkill for this type of functionality, but it is a
nice first use-case of how functionality can be compartmentalized into
modifiers. If makes swapping retry mechanisms in and out really simple.
2020-07-01 12:19:02 +02:00
Christian Decker aeb5cc0b7c paymod: Generate type-safe accessor functions for modifier data
This should make it easy for JSON-RPC functions and modifiers to get the
associated data for a given modifier name. Useful if a modifier needs to
access its parent's modifier data, or in other functions that need to access
modifier data, e.g., when passing destination pointers into the `param()`
call.
2020-07-01 12:19:02 +02:00
Christian Decker 23b4dca3c7 paymod: Add a simple test-command to test the paymod state-machine
This commit can be reverted/skipped once we have implemented all the logic and
have feature parity with the normal `pay`. It's main purpose is to expose the
unfinished functionality to test it, without completely breaking the existing
`pay` command.
2020-07-01 12:19:02 +02:00
Christian Decker 5d1725209d paymod: Implement base state-machine transitions
The actual steps are mocked out, but we can already run through the various
phases of a payment, and the modifiers should be called for each state.
2020-07-01 12:19:02 +02:00
Christian Decker e108ebe559 paymod: Add base state-machine for payment flow 2020-07-01 12:19:02 +02:00
Christian Decker 8138ef136f plugin: Add a data backend for the new payment state-machine 2020-07-01 12:19:02 +02:00
Saibato eea6b33154 connect/tor: Set errno, if connection failed we want to reflect this.
Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato 2b0aba13a8 connectd/connectd: Display an error hint for V3 tor onions when connect fail.
@thestick613 noticed that since tor version below 0.3.2.2-alpha
will not support V3 ed25519 address formats, the error handling
is not that helpful in the error message from cli.
So now we add an hint.

Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>

connectd/connectd.h; Add helper function to update conn error list

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato e4cd2148ab Reflect in TOR.md that we need a Tor version >= 0.3.2.2-alpha to reach V3 onions
Changelog-None

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
ZmnSCPxj jxPCSnmZ 5d720536e2 Makefile: Install `tools/hsmtool` as `lightning-hsmtool`.
Changelog-Added: We now install `lightning-hsmtool` for your `hsm_secret` needs.

See: https://github.com/ElementsProject/lightning/issues/3717#issuecomment-644844594

It seems reasonable to add this to the standard install, and to document it properly as well, hopefully we can fill in the documentation better later on.
2020-07-01 09:24:03 +09:30
Christian Decker 7b899da801 db: Retrieve peer ID if it exists or create the peer if not
We were assuming `wallet_channel_insert` that there cannot be a matching peer
if our in-memory representation isn't bound to it (`dbid == 0`). If we then
attempt to create the peer, and we already had one it'd cause a unique
constraint violation. As far as I can tell this could end up happening if we
have an uncommitted channel, and then exited without cleanup (`tal_destructor`
on the uncommitted channel not running). This could then leave the peer in the
DB. This is because the constraint that every peer has at least one channel is
not enforce at DB level, but rather in destructors that may or may not run.

Changelog-Fixed: Fixed a failing assertion if we reconnect to a peer that we had a channel with before, and then attempt to insert the peer into the DB twice.
2020-07-01 09:22:29 +09:30
Rusty Russell ae0cccb293 wallet: fix memleak if we get shut down before finishing utxo cleanup.
"backtrace": [
                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
                       "wallet/wallet.c:154 (wallet_stmt2output)",
                       "wallet/wallet.c:275 (wallet_get_utxos)",
                       "wallet/wallet.c:3792 (wallet_clean_utxos)",
                       "lightningd/lightningd.c:914 (main)"
                   ],
                   "label": "wallet/wallet.c:154:struct utxo",
                   "parents": [
                       "wallet/wallet.c:273:struct utxo*[]"
                   ],
                   "value": "0x24c1be8"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-01 09:19:39 +09:30
ZmnSCPxj jxPCSnmZ d14460ef3d wallet/walletrpc.c: Show input annotations for inputs.
Changelog-None
2020-06-30 17:59:17 +02:00
Rusty Russell 6b3bcd49d2 tests: fix flake in test_penalty_htlc_tx_fulfill
If the daemon already knows about the channel before it was stopped,
it won't get this message from gossipd.  That's OK, since we explicitly
test for the channel being active two lines down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
Rusty Russell dd4a1a3510 tests: fix flake in test_txprepare_restart
Detection of reserved outputs is async, going via bitcoind.  Wait for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
Rusty Russell 74abd30da5 pyln-proto, pyln-spec: fix 'make prod-release' target.
rusty$ make prod-release
  make: *** No rule to make target 'test', needed by 'prod-release'.  Stop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-29 18:01:12 +02:00
niftynei 2e9c387f45 coin_moves: update withdrawal logic to account for 'variable owner' txs
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)

Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
2020-06-29 16:10:05 +02:00
niftynei ee549a2af9 tx: fix case where input amounts are less than total outputs
when attempting to calculate the fees for a tx where we don't own all of
the outputs, we can overshoot the feerate
2020-06-29 16:10:05 +02:00
niftynei 9830c94778 rpc: new signpsbt + sendpsbt rpcs
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
2020-06-29 16:10:05 +02:00
niftynei fd8a716695 wallet: have wallet_extract_outputs take wally_tx, not bitcoin_tx
With the incursion of PSBTs, we're moving away from bitcoin_tx
2020-06-29 16:10:05 +02:00
niftynei 0388fe6db4 tx: add helper for extracting script from a wally_tx
the bitcoin_tx version is basically a wrapper for the wally_tx script
extraction -- here we pull it apart so we can easily get a tal'd script
for a wally_tx_output
2020-06-29 16:10:05 +02:00
niftynei a68d79e390 add manpages for reserve/unreserve 2020-06-29 16:10:05 +02:00
niftynei 103dce63ef reserve/unreserve input: new RPC commands for reserving inputs/outputs
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.

Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.

Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
2020-06-29 16:10:05 +02:00
niftynei b90be4f6c8 prepare-tx: pass back the feerate, as json_tx_prepare sometimes sets it
Unused here, but we'll use it in the next commit so that we can always
pass back the effective / used feerate to the caller of `reserveinputs`

This makes opening a channel much easier if we've internally determined
the feerate
2020-06-29 16:10:05 +02:00
Rusty Russell 57f6f74ee0 external/Makefile: fix spurious rebuilds.
Quote marks are not special to make: as it can't find
external/"x86_64-linux-gnu"/libwally-core-build/src/libwallycore.la
it always insists on rebuilding it (which rebuilds the world).

If we have spaces in TARGET_DIR, we're in trouble already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-None
2020-06-29 15:48:09 +09:30
Wladimir J. van der Laan 32f9805a30 build: Make it possible to build without gcc
The external Makefile hardcodes gcc to get the machine spec.
This should use the configured C compiler instead.

This bug was introduced in 601464416b.
2020-06-29 09:08:39 +09:30
Vasil Dimov a3a37ea4ce build: fix compilation on i386
```
plugins/keysend.c:136:47: error: format specifies type 'unsigned long' but the argument has type 'time_t' (aka 'int') [-Werror,-Wformat]
        ki->label = tal_fmt(ki, "keysend-%lu.%09lu", now.ts.tv_sec, now.ts.tv_nsec);
                                         ~~~         ^~~~~~~~~~~~~
                                         %d
```

Changelog-None
2020-06-28 09:36:30 +00:00
Rusty Russell 42bf230bfc doc: regenerate and update.
My node tried to rebuild one of them because it was lagging (I noticed
because mrkd was not installed).

Turns out a few were lagging, and lightning-dev-sendcustommsg.7 was totally
empty!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-28 01:14:08 +00:00
Rusty Russell 8ee8161fb4 pyln-spec: add .gitignore file for build detritus.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 447231f52e pyln.spec.bolt*: change version numbering to include specific csv subversion.
Now they look like 1.0.1.137, so you can explicitly depend on a csv change
(without caring about a textual change).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 1c1c3349b4 pyln.spec.bolt*: unify setups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 450fb43612 pyln.spec.bolt*: test-release and prod-release targets, fix requirements.txt
We depend on packages, not modules within them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 93ae190c19 pyln.proto: bump version to 0.8.3.
Changelog-Changed: pyln.proto version now 0.8.3 to indicate pyln.proto.message
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 014ede0585 pyln.proto: fix test-release target
1. version was 0.0.2 in setup.py, which means we didn't get the dist/ files we expected.
2. We need 'bdist_wheel' to make the .whl file.
3. --no-site-packaged was apparently removed in 0.20.0, and was default long before that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 902d8f7dab pyln.proto.message: remove incorrect fundamental types now spec update.
See 9e8e29af9b

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 41e914fb37 pyln.proto.message, pyln.proto.spec*: do magic to expose mypy types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 11a0de877e pyln.proto.message: more mypy fixes.
This includes some real bugfixes, since it noticed some places we were
being loose with different types!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 3882e8bdf7 pyln.spec.bolt*: make these separate packages, so versions can change indep.
They're almost entirely autogenerated, and we use symlinks into the
top directory to reduce replication.

They can't be under pyln.spec.message, because a package can't also
be a namespace.

We also add fulltext and desc fields, and exclude our "gen" files from
flake8, since the spec quotes contain weird whitespace.

Changelog-Added: Python: pyln.spec.bolt{1,2,4,7} packages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 013eced4ce pyln.proto: fix package list in setup.py
Remove non-existant pyln.proto.bolts.  bolts will have separate setup.py, so we
can rev the versions individually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Niklas Claesson 601464416b travis: Actually build for arm
Before this we cached the whole `external` directory, but that lead to
stale files since the cache is checked out after the repository is
cloned.
2020-06-24 17:08:40 +02:00
Antoine Poinsot e0aef66ce2 plugins: make the autoclean plugin static
The autocleaning will only happen if the autocleaninvoice-cycle startup
option is passed, which cannot happen if the plugin is started
post-startup.
Thus, it's less misleading for users to restrict its usage to startup.

Changelog-Added: plugins: The `autoclean` plugin is now static (you cannot manage it with the `plugin` RPC command anymore).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-06-24 15:29:39 +02:00
Rusty Russell 93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei a66415a43e plugins-test: use pyln.client, not lightning to import Plugin 2020-06-23 14:49:32 +02:00
niftynei 0f8157a02e libwally: update to latest commit on master, which contains psbt fixes
fixes for p2sh-p2wpkh sigs in libwally etc
2020-06-23 14:49:32 +02:00
niftynei 0bd0de54fa psbt: have withdraw_tx use psbt's to create signed txs
this will allow us to add inputs that aren't ours to a tx that we sign
and finalize
2020-06-23 14:49:32 +02:00
niftynei 0a7550473b psbt: add redeemscript info to psbt for utxos that have it 2020-06-23 14:49:32 +02:00
niftynei 85dca84327 psbt-json: remove reliance on bitcoin_tx, use straight wally_psbt struct 2020-06-23 14:49:32 +02:00
niftynei 175fcf381a psbt: have wally_tx serialization methods be legible for gen'd code
our code generators expect the serialization name to match the struct
type
2020-06-23 14:49:32 +02:00
niftynei b63abef542 psbt: add method to finalize + extract a psbt
will either use a temporary psbt (and not munge the passed in psbt)
or will finalize in place -- finalization erases most of the signature
metadata from the psbt struct
2020-06-23 14:49:32 +02:00