Commit Graph

985 Commits

Author SHA1 Message Date
Rusty Russell 381b0f456c db: offer table.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 32c7c133f4 common/sphinx: make onionpacket.routinginfo a dynamic member.
Still asserts that it's the standard size, but makes it a dynamic
member.  For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell 25b0dbe7e8 wallet: remove debugging fprintf.
Left over from e81d78ec4c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-10 20:32:18 +10:30
Rusty Russell 1a3c0a0c0d invoice: fix potential race where invoice is paid/expired while we're calling hook.
There's actually a (very unlikely) race here: we would previously have
crashed with an assertion in invoices_resolve.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 16:50:12 +10:30
niftynei e81d78ec4c fundpsbt/utxopsbt: new `min_witness_weight` param
Needed for v2 of channel opens, where the minimum weight is 110; a
'simple utxo' (sig + key) weighs in at 107, so we a need a way to
establish a floor for this case.

Changelog-Added: JSON-RPC: fundpsbt/utxopsbt have new param, `min_witness_utxo`, which sets a floor for the weight calculation of an added input
2020-12-02 14:19:08 +10:30
Rusty Russell ca2bd98082 unittest: use common_setup / common_shutdown almost everywhere.
Avoids much cut & paste.  Some tests don't need any of it, but most
want at least some of this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:55:09 +10:30
Michael Schmoock 6cc96c07dc feat: adds state_changes to listpeers output
Changelog-Added: RCP: Added 'state_changes' history to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock 88c1dc56e8 db: adds a state_change history to database 2020-11-06 14:47:04 -06:00
Michael Schmoock 68ce25c92d feat: adds timestamp to state_change notification 2020-11-06 14:47:04 -06:00
Michael Schmoock b7c18517df rpc: adds opener and closer to listpeers channels
Changelog-Added: RPC: Added 'opener' and 'closer' to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock 083a856c31 db: persist channel closer and state change cause 2020-11-06 14:47:04 -06:00
Michael Schmoock 8a8dabaa58 feat: adds state change cause and message
This adds a `state_change` 'cause' to a channel.
A 'cause' is some initial 'reason' a channel was created or closed by:

  /* Anything other than the reasons below. Should not happen. */
  REASON_UNKNOWN,
  /* Unconscious internal reasons, e.g. dev fail of a channel. */
  REASON_LOCAL,
  /* The operator or a plugin opened or closed a channel by intention. */
  REASON_USER,
  /* The remote closed or funded a channel with us by intention. */
  REASON_REMOTE,
  /* E.g. We need to close a channel because of bad signatures and such. */
  REASON_PROTOCOL,
  /* A channel was closed onchain, while we were offline. */
  /* Note: This is very likely a conscious remote decision. */
  REASON_ONCHAIN

If a 'cause' is known and a subsequent state change is made with
`REASON_UNKNOWN` the preceding cause will be used as reason, since a lot
(all `REASON_UNKNOWN`) state changes are a subsequent consequences of a prior
cause: local, user, remote, protocol or onchain.

Changelog-Added: Plugins: Channel closure resaon/cause to channel_state_changed notification
2020-11-06 14:47:04 -06:00
Christian Decker d487ec5c9b wallet: Remove orphan statements.po file 2020-10-29 11:11:48 +10:30
Rusty Russell 497e71ca13 Makefile: fix bashism in NO_PYTHON
== is a bash extension; in shell it's a single =:

```
/bin/sh: 1: [: unexpected operator
sql-rewrite wallet/db_postgres_sqlgen.c
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-26 21:28:36 -05:00
Rusty Russell 1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
niftynei a7d4b8780e elementsd: disable dual-funding test; don't add non-witness utxo
Elementsd needs to be updated to v0.20.0 in order for this to work as
expected
2020-10-20 12:50:31 +10:30
niftynei 818f152618 channel: save funding_psbt to database
Only populated until we receive the peer's funding_locked
2020-10-20 12:50:31 +10:30
niftynei b2ec5a9f45 peer_channeld: pass over PSBT, remove second message
We need the PSBT to create the finalized tx from once the peer's
tx_signatures are received. Since we're passing the PSBT, we no longer
need the secondary message to be passed, as it was derived from the
PSBT.

Also removes now unused witness serialization code
2020-10-20 12:50:31 +10:30
niftynei d0702e05dd psbt: add the full transaction for the utxo in a psbt
We need this for dual funding, since the interactive tx construction
protocol requires the full tx to send places. We add it to all PSBTs (if
we have it), here
2020-10-20 12:50:31 +10:30
niftynei 3c98bec693 db: add helper for db_col_psbt
We had db_bind_psbt; this is the missing symmetric response to that
2020-10-20 12:50:31 +10:30
niftynei e2a6fd7112 common: pull up `param_psbt`
Usable other places, not just in wallet
2020-10-20 12:50:31 +10:30
Michael Schmoock db19f19eb3 chore: adds env NO_PYTHON check to Makefiles
This adds an environment variable $NO_PYTHON check to Makefiles so that:
 - It checks and runs into an defined error instead of some python hickup:
   `ModuleNotFoundError: No module named 'mako'`
 - makes it possible to manually export this environment variable NO_PYTHON=1
   to run the same testcase that travis is running on job 1 which causes
   so much pain ;)

Changelog-None
2020-10-19 13:17:03 +02:00
Christian Decker 77ca07e91d db: Fix statement expansion bugs found through dblog mode 2020-10-10 11:34:43 +10:30
Michael Schmoock 011590b20e fix: broken SQL statement in wallet db_set_utxo
I discovered this accidentally when using the `tests/plugins/dblog.py`
plugin on another testcase: tests/test_connection.py::test_fail_unconfirmed

There the plugin/hook crashes because it can't execute the statement:
```json
{
  "jsonrpc": "2.0",
  "id": 34,
  "error": {
    "code": -32600,
    "message": "Error while processing db_write: unrecognized token: \"174WHERE\"",
    "traceback": "Traceback (most recent call last):\n  File \"/home/will/projects/lightning.git/contrib/pyln-client/pyln/client/plugin.py\", line 535, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/home/will/projects/lightning.git/contrib/pyln-client/pyln/client/plugin.py\", line 520, in _exec_func\n    return func(*ba.args, **ba.kwargs)\n  File \"/home/will/projects/lightning.git/tests/plugins/dblog.py\", line 45, in db_write\n    plugin.conn.execute(c)\nsqlite3.OperationalError: unrecognized token: \"174WHERE\"\n"
  }
}
```

Changelog-Fixed: plugin: Regression with SQL statement expansion that could result in invalid statements being passed to the `db_write` hook.
2020-10-10 11:34:43 +10:30
Rusty Russell 7260d9ea3d plugins: generate list of plugins more atomically, respect V=1 and --quiet.
I got a corrupt file, which looked like multiple concurrent attempts
to build it.  So instead, build it in one command, but also use
VERBOSE so we print correctly with V=1 (and --quiet).

Also move into plugins/ where it logically belongs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 13:19:59 +02:00
Rusty Russell 488b32b003 build: run update-mocks.
Some declarations are redundant now.  Removing them does nothing, but
it makes other PRs cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 13:19:59 +02:00
Rusty Russell d5cb0d85b5 utils: use a cleaner pattern to capture wally allocations.
We force use of tal_wally_start/tal_wally_end around every wally
allocation, and with "end" make the caller choose where to reparent
everything.

This is particularly powerful where we allocate a tx or a psbt: we
want that tx or psbt to be the parent of the other allocations, so
this way we can reparent the tx or psbt, then reparent everything
else onto it.

Implementing psbt_finalize (which uses a behavior flag antipattern)
was tricky, so I ended up splitting that into 'psbt_finalize' and
'psbt_final_tx', which I think also makes the callers clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:45:53 +02:00
Christian Decker 1d9e7cf079 db: Add support for key-value pair DSNs in postgresql
These are simple space-separated key-value pair sets of options instead of the
URI style DSNs, but they are also much more flexible allowing the user to
specify client SSL certificates, server certificates, compression and
encryption levels, and much more (see [1] for more information)

[1]: https://www.postgresql.org/docs/9.1/libpq-connect.html

Changelog-Added: db: Added support for key-value DSNs for postgresql, allowing for a wider variety of configurations and environments.
2020-09-23 20:20:40 +09:30
Rusty Russell 58f6e316cc lightningd: separate fee_states out of struct channel_info.
It was always kind of weird in there anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
Rusty Russell 77b62d9e42 bitcoin/psbt: psbt_finalize needs a tal ctx.
Since it returns a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Christian Decker de7b41695f db: Fix size mismatch on postgres in a migration
Changelog-Fixed: db: Fixed a broken migration on postgres DBs that had really old channels.
2020-09-23 10:48:31 +09:30
Matt Whitlock eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
Vincenzo Palazzo f62d7bbe45 Added additional fix to code formatting and English spelling.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
Vincenzo Palazzo 9fbeb9bcdc delpay: code style changes and fixed docs
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
niftynei 273f0991b0 elements: include the value + asset tag for 'PSET's
Not strictly necessary as technically this info is included in the
witness_utxo, but nice to have
2020-09-10 12:31:36 +09:30
Michael Schmoock d86855d1f7 feat: adds channel_state_changed notification
This notification will be raised whenever a channel state changes.
The payload includes the channel and peer identifiers and the
old and the new state.

Example payload:

```
{
    "channel_state_changed": {
        "peer_id": "03bc9337c7a28bb784d67742ebedd30a93bacdf7e4ca16436ef3798000242b2251",
        "channel_id": "a2d0851832f0e30a0cf778a826d72f077ca86b69f72677e0267f23f63a0599b4",
        "short_channel_id" : "561820x1020x1",
        "old_state": "CHANNELD_NORMAL",
        "new_state": "AWAITING_UNILATERAL"
    }
}
```

Changelog-Added: Plugins: channel_state_changed notification
2020-09-10 10:24:06 +09:30
niftynei c50f377a85 psbt: pull out changeset logic into common, update API
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.

Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei 5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei 303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
niftynei 8d429ecd06 df: add needed info to any PSBT we produce
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
2020-09-09 19:54:20 +09:30
niftynei 9c89184c1f dualfund: add feature flag for dual-funding
turn off until we're ready to test both sides
2020-09-09 19:54:20 +09:30
niftynei d6558deaa4 opening: pass two messages to channeld to send to peer
v2 of channel establishment, in the accpeter case, now sends 2 messages
to our peer after saving the information to disk (our commitment
signatures and our funding transaction signatures)
2020-09-09 19:54:20 +09:30
niftynei 0643945967 peer-control: send set of messages to start, not just one
The accepter has to send 2 messages over to channeld to send at start --
their commitment_signatures and tx_signatures
2020-09-09 19:54:20 +09:30
niftynei 864f2f3e21 channel_id: save to database, dont derive from funding_txid
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.

includes a migration for existing channels
2020-09-09 19:54:20 +09:30
niftynei 437ca83fa5 elementsd: add a fee output to a fund/utxopsbt for elements transactions
fundpsbt / utxopsbt create a (typically) output-less PSBT,
however for elements we require the fees to be encapsulated in an
output.

this patch updates fundpsbt / utxopsbt to add a fee output for elements
transactions. includes test updates.

Fixes #3998
2020-09-09 16:50:14 +09:30
Christian Decker c239a7161b wallet: Remove in-memory utxoset filter
This removes the in-memory utxoset filter, meaning we now just tracks UTXOs in
the DB.
2020-09-09 09:24:29 +09:30
Christian Decker cab52f1197 topology: Notify gossipd about spends after processing the block
We defer the notification to gossipd till the end of the spends. By itself not
a huge change, but it allows us to later migrate to doing updates blindly and
using the DB as our ground truth. It also allows us to simplify the
`wallet_outpoint_spend` semantics to not return two values in the next commit.
2020-09-09 09:24:29 +09:30
Christian Decker eeb6e8091b wallet: Add function to retrieve spent UTXOs for a given height
The plan is to eventually use this to tell gossipd about any outputs that were
spent by the last block.
2020-09-09 09:24:29 +09:30
Rusty Russell 2be1f3fe1b lightningd: extract routehint selection code.
We're going to want this for bolt13 formation as well.

As a result of reworking the logic into "candidate selection" then
"route hint selection", we need to change the way round-robin works.
We use a simple incrementing index now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 19:47:11 +09:30