Commit Graph

10319 Commits

Author SHA1 Message Date
Rusty Russell 487facf1f0 pyln-client/gossmap: more fixes, make mypy happier.
Mainly fixing type annotations, but some real fixes:

1. GossmapHalfchannel.from_str() should be a classmethod.
2. update_channel had weird, unusable default values (fields can't be NULL,
   since we use it below).

[ There was one more occurence where isinstance should be used above
type() == xyz comparison. -- MS ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Michael Schmoock ba2bcac530 pyln-client/gossmap: Don't mix bytes and GossmapNodeId
Do not mix bytes and GossmapNodeId when accessing Gossmap.nodes dicts.

Therefore the definion got GossmapNodeId also needed to be pulled to the
beginning of the file.
2021-09-08 09:34:14 +09:30
Michael Schmoock ac27217114 pyln-client/gossmap: adds __repr__ functions 2021-09-08 09:34:14 +09:30
Michael Schmoock b60d3259bf pyln-client/gossmap: extend testcase 2021-09-08 09:34:14 +09:30
Michael Schmoock b6ed405d8c pyln-client/gossmap: adds channel satoshi capacity
This reads the `gossip_store_channel_amount` that always follows the
`channel_announcement` messages. Therefore it uses an internal variable
_last_scid to know what channel has been added last time.
2021-09-08 09:34:14 +09:30
Michael Schmoock c5c909acd3 pyln-client/gossmap: adds helpers to get channels and nodes 2021-09-08 09:34:14 +09:30
Michael Schmoock fd16535f68 pyln-client/gossmap: adds GossmapHalfchannel objects 2021-09-08 09:34:14 +09:30
Rusty Russell 75126a0997 pyln-client/gossmap: add NodeId class.
This is more efficient than converting them all to Pubkeys: about 3.8
seconds vs 5.4 seconds.  Usually treating them as raw bytes is what we
want anyway.

[ Fixup by Michael Schmoock <michael@schmoock.net> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell 887fb38e06 pyln-client/gossmap: use ShortChannelId class from pyln.proto, if available.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell 0ce473c656 pyln-proto: expose ShortChannelId and PublicKey.
They're generally useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell 4753ca4d35 pyln-client/gossmap: add a little documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell e495462c50 pyln-client/gossmap: save deconstructed fields instead of raw msg bytes.
We have to parse them anyway, so why not make them accessible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell 1a5a26d659 pyln.client: new functionality to access Gossmap.
It doesn't do much work, but it does parse the gossmap file and extract
nodes and channels.

[ Fixup by Michael Schmoock <michael@schmoock.net> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell 07d2639f51 pyln.client: don't try to use module inside setup.py.
This fails once it has dependencies (next patch): instead extract version
manually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell 6b8d3a05f5 pyln-spec: update to latest version of the spec.
Changelog-Changed: pyln-spec: updated to latest BOLT versions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-07 17:41:28 +09:30
Dustin Dettmer ce1c5aadf4 Mac OS X generates these files automatically.
Add them to the ignore list.
2021-09-07 13:40:19 +09:30
Vincenzo Palazzo be829bbc26 Removed submodule external/lnprototest
Update version lnprototest  with the latest one

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-06 14:37:27 +02:00
Dustin Dettmer 6f2e27c768 Shellcheck cleanup
Updates from output of latest version of shellcheck as per SC2268

“Avoid x-prefix in comparisons as it no longer serves a purpose”
https://www.shellcheck.net/wiki/SC2268
2021-09-05 18:55:53 +02:00
Rusty Russell 36baef22e2 doc: document shutdown plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell 516c38a750 lightningd: call shutdown plugin when we dynamic shutdown a single one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell 2063049559 libplugin: plugin_exit helper which flushes stdout.
We weren't actually getting the last log out; this does that.

We have to fix test_bitcoin_failure which now notices the BROKEN
log message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: libplugin: Fatal error messages from plugin_exit() now logged in lightningd.
2021-09-05 15:16:56 +02:00
Rusty Russell 1d8aecb44f lightningd: call "shutdown" notification on plugins at shutdown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `shutdown` notification for clean exits.
2021-09-05 15:16:56 +02:00
Rusty Russell 89c3541c83 lightningd: disable topology timers on shutdown.
Not necessary yet, but it will be once shutdown starts waiting for
plugins to respond: we don't want these to try to access the bcli
plugin once it's freed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell 395bf96d15 lightningd: don't keep timer ptr inside struct chain_topology.
It's a legacy from when it didn't have an ld pointer to access ld's
timer structure.  Now it's just confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell 6d1897b796 doc: fix GH/readthedocs rendering of manual pages.
They render the comment as if it's in the list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:02:17 +02:00
niftynei 0f05238e97 psbt: set the witness script to null, also
Otherwise libwally pushes the psbt-key for 'witness script' onto the
serialized version and we fail the 'is this identical' check.

Relevant line from libwally, where if bytes, we push a psbt_key.

```
static void push_typed_varbuff(unsigned char **cursor, size_t *max,
                               uint64_t type,
                               const unsigned char *bytes, size_t bytes_len)
{
    if (bytes) {
        push_psbt_key(cursor, max, type, NULL, 0);
        push_varbuff(cursor, max, bytes, bytes_len);
    }
}
```

Reported-By: @grubles
Changelog-Fixed: openchannel_signed would fail on PSBT comparison of materially identical PSBTs
2021-09-05 14:24:14 +02:00
niftynei 1782ae8590 dual-fund psbt: hoist up single use method so we can test it
Also remove all the unused code (checking for differences for a side
etc)
2021-09-05 14:24:14 +02:00
Rusty Russell d6192c2cb0 db: simple index addition to speed listpeers.
Fixes: #4718
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-04 13:41:35 +02:00
William Casarin 105a88dde4 docs/fetchinvoice: document payer_note
Signed-off-by: William Casarin <jb55@jb55.com>
2021-09-04 13:18:29 +02:00
Michael Schmoock f6709a7a90 pytest: test new commitment_revocation hook values 2021-09-04 11:54:33 +09:30
Michael Schmoock 37a7dab549 doc: new values for commitment_revocation hook 2021-09-04 11:54:33 +09:30
Michael Schmoock 1db0ec8b75 hook: adds channel_id and commitnum to commitment_revocation
If commitment_revocation hook is not being used for a whatchtower but
something else, the channel_id is missing.

Changelog-Added: addes channel_id and commitnum to commitment_revocation hook
2021-09-04 11:54:33 +09:30
Christian Decker 7e43109e33 doc: Regenerated man-pages to fix hash mismatch. 2021-09-03 20:03:02 +09:30
Rusty Russell 1c8294ef1c doc: fix listnodes schema, doc.
Shows the dangers of "additionalProperties": true.  We didn't have an else
clause, so our incorrect (and, with DF, incomplete!) schema was accepted.

I despair of getting anyone else to write a decent schema with these
semantics :(

Changelog-Fixed: doc: listnodes fields now correctly documented.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-03 20:03:02 +09:30
Rusty Russell 372fffac69 Anchor outputs are enabled by dual-funding, so this was reported by
users there (and reliably detected now).

Reported-by: @larsschenk
Reported-by: @nickfarrow
Fixes: #4728
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: EXPERIMENTAL: crash for some users while requesting dual funding leases.
2021-09-02 15:06:18 +02:00
Rusty Russell 2e01d26588 configure: update from -fstack-protector to -fstack-protector-strong.
This would have caught the bug found in the next commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-02 15:06:18 +02:00
Vincenzo Palazzo 218875a38d adding check for the deprecated api.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-02 14:19:42 +09:30
Vincenzo Palazzo 3b65a4caca Fixed order parameter in the listforwards command
Changelog-Changed: Change order parameters in the listforwards command

Changelog-Deprecated: Change order of the status parameter in the listforwards rpc command.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-02 14:19:42 +09:30
Rusty Russell 0a05bcd05e wire: fix patch file to match actual peer_wire.csv.
Running `make extract-peer-csv` changes peer_wire.csv, @niftynei
points out that the patch file is wrong (peer_wire.csv is correct).

(Any attempt to regenerate the csv files from the specs hits this issue)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-02 14:04:32 +09:30
Christian Decker df70bbcc6a docs: Remove explicit markdown configuration for rtd 2021-08-31 11:46:17 -05:00
Rusty Russell 2770bcb158 lightningd: don't reject attempts to forward 2^32 msat out of hand.
The outgoing channeld still does that, for now, but that will change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-30 15:56:20 +09:30
Rusty Russell ebb0b5791e Don't limit channel_updates to 2^32 msat.
We're in a post-Wumbo world now!  See also https://github.com/lightningnetwork/lightning-rfc/pull/902

Fixes: #4746
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-30 15:56:20 +09:30
Vincenzo Palazzo bc1eab8646 Skip following import and skip flask type to mypy code analysis.
As suggested in this issue https://github.com/python/mypy/issues/7484#issuecomment-529363083 we skip following import becuase with the recent version of mypy the __init__.py file make confusione inside the analysis (in the python issue it is unclear the main motivation of this issue. At list unclear to me).

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-08-30 12:12:32 +09:30
Vincenzo Palazzo b40812494d Move to minimum required mypy dependeces
This fix some dependencies error with lnprototest

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-08-30 12:12:32 +09:30
Vincenzo Palazzo f2d0e93ce0 Added deprecated phase to rpc framework rules changes.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

Changelog-Deprecated: RPC framwork now require the "jsonrpc" propriety inside the request.
Changelog-Fixed: RPC framwork now required the "jsonrpc" propriety to be specified inside each request.
2021-08-30 12:10:37 +09:30
Vincenzo Palazzo dca9bb8638 Required jsonrpc inside the method request.
As the json rpc specification tell, the "jsonrpc": "2.0" MUST be required.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-08-30 12:10:37 +09:30
Rusty Russell 8f782b06f7 pytest: don't copy db to query it.
Should be safe to do in-place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-26 12:44:02 +09:30
Rusty Russell 5bf2b5c0ba wallet: set a timeout if the db is busy.
This is recommended for litestream, which allows for easy async backup,
and harmless otherwise.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: db: we now set a busy timeout to safely allow others to access sqlite3 db (e.g. litestream)
2021-08-26 12:44:02 +09:30
Alex c066378576 added Litestream Replication
Litestream enables easy SQLite replication
2021-08-26 12:44:02 +09:30
Michael Schmoock da0b159323 pylightning: fix requirements to work with pyln-client
Without this, we cannot pip install pylightning and pyln-client in the
same environment anymore, as it tries to pull in an incompatible version
of pyln-client.

Changelog-None
2021-08-26 10:13:01 +09:30