Commit Graph

7961 Commits

Author SHA1 Message Date
Rusty Russell 57f6f74ee0 external/Makefile: fix spurious rebuilds.
Quote marks are not special to make: as it can't find
external/"x86_64-linux-gnu"/libwally-core-build/src/libwallycore.la
it always insists on rebuilding it (which rebuilds the world).

If we have spaces in TARGET_DIR, we're in trouble already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-None
2020-06-29 15:48:09 +09:30
Wladimir J. van der Laan 32f9805a30 build: Make it possible to build without gcc
The external Makefile hardcodes gcc to get the machine spec.
This should use the configured C compiler instead.

This bug was introduced in 601464416b.
2020-06-29 09:08:39 +09:30
Vasil Dimov a3a37ea4ce build: fix compilation on i386
```
plugins/keysend.c:136:47: error: format specifies type 'unsigned long' but the argument has type 'time_t' (aka 'int') [-Werror,-Wformat]
        ki->label = tal_fmt(ki, "keysend-%lu.%09lu", now.ts.tv_sec, now.ts.tv_nsec);
                                         ~~~         ^~~~~~~~~~~~~
                                         %d
```

Changelog-None
2020-06-28 09:36:30 +00:00
Rusty Russell 42bf230bfc doc: regenerate and update.
My node tried to rebuild one of them because it was lagging (I noticed
because mrkd was not installed).

Turns out a few were lagging, and lightning-dev-sendcustommsg.7 was totally
empty!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-28 01:14:08 +00:00
Rusty Russell 8ee8161fb4 pyln-spec: add .gitignore file for build detritus.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 447231f52e pyln.spec.bolt*: change version numbering to include specific csv subversion.
Now they look like 1.0.1.137, so you can explicitly depend on a csv change
(without caring about a textual change).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 1c1c3349b4 pyln.spec.bolt*: unify setups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 450fb43612 pyln.spec.bolt*: test-release and prod-release targets, fix requirements.txt
We depend on packages, not modules within them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 93ae190c19 pyln.proto: bump version to 0.8.3.
Changelog-Changed: pyln.proto version now 0.8.3 to indicate pyln.proto.message
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 014ede0585 pyln.proto: fix test-release target
1. version was 0.0.2 in setup.py, which means we didn't get the dist/ files we expected.
2. We need 'bdist_wheel' to make the .whl file.
3. --no-site-packaged was apparently removed in 0.20.0, and was default long before that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 902d8f7dab pyln.proto.message: remove incorrect fundamental types now spec update.
See 9e8e29af9b

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 41e914fb37 pyln.proto.message, pyln.proto.spec*: do magic to expose mypy types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 11a0de877e pyln.proto.message: more mypy fixes.
This includes some real bugfixes, since it noticed some places we were
being loose with different types!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 3882e8bdf7 pyln.spec.bolt*: make these separate packages, so versions can change indep.
They're almost entirely autogenerated, and we use symlinks into the
top directory to reduce replication.

They can't be under pyln.spec.message, because a package can't also
be a namespace.

We also add fulltext and desc fields, and exclude our "gen" files from
flake8, since the spec quotes contain weird whitespace.

Changelog-Added: Python: pyln.spec.bolt{1,2,4,7} packages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 013eced4ce pyln.proto: fix package list in setup.py
Remove non-existant pyln.proto.bolts.  bolts will have separate setup.py, so we
can rev the versions individually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Niklas Claesson 601464416b travis: Actually build for arm
Before this we cached the whole `external` directory, but that lead to
stale files since the cache is checked out after the repository is
cloned.
2020-06-24 17:08:40 +02:00
Antoine Poinsot e0aef66ce2 plugins: make the autoclean plugin static
The autocleaning will only happen if the autocleaninvoice-cycle startup
option is passed, which cannot happen if the plugin is started
post-startup.
Thus, it's less misleading for users to restrict its usage to startup.

Changelog-Added: plugins: The `autoclean` plugin is now static (you cannot manage it with the `plugin` RPC command anymore).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-06-24 15:29:39 +02:00
Rusty Russell 93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei a66415a43e plugins-test: use pyln.client, not lightning to import Plugin 2020-06-23 14:49:32 +02:00
niftynei 0f8157a02e libwally: update to latest commit on master, which contains psbt fixes
fixes for p2sh-p2wpkh sigs in libwally etc
2020-06-23 14:49:32 +02:00
niftynei 0bd0de54fa psbt: have withdraw_tx use psbt's to create signed txs
this will allow us to add inputs that aren't ours to a tx that we sign
and finalize
2020-06-23 14:49:32 +02:00
niftynei 0a7550473b psbt: add redeemscript info to psbt for utxos that have it 2020-06-23 14:49:32 +02:00
niftynei 85dca84327 psbt-json: remove reliance on bitcoin_tx, use straight wally_psbt struct 2020-06-23 14:49:32 +02:00
niftynei 175fcf381a psbt: have wally_tx serialization methods be legible for gen'd code
our code generators expect the serialization name to match the struct
type
2020-06-23 14:49:32 +02:00
niftynei b63abef542 psbt: add method to finalize + extract a psbt
will either use a temporary psbt (and not munge the passed in psbt)
or will finalize in place -- finalization erases most of the signature
metadata from the psbt struct
2020-06-23 14:49:32 +02:00
niftynei 0d1f1bc66e psbt: add helper method for setting a psbt input's redeemscript 2020-06-23 14:49:32 +02:00
niftynei c3ae44e296 psbt: don't crash if we can't add a partial sig
instead return a boolean indicating the success/failure of a sig set
2020-06-23 14:49:32 +02:00
niftynei 80072b389e psbt: remove script sig info from inputs before adding them to global
PSBT's dont' serialize / unserialize if there's any sig info set on the
global transaction
2020-06-23 14:49:32 +02:00
lisa neigut 16656a85cf withdraw: refactor change output handling
We're not using the change_outnum for withdraw tx's (and the way
we were calculating it was broken as of the addition of 'multiple
outputs'). This removes the change output knowhow from withdraw_tx
entirely, and pushes the responsibility up to the caller to
include the change output in the output set if desired.

Consequently, we also remove the change output knowhow from hsmd.
2020-06-23 14:49:32 +02:00
niftynei 09815c7e7f psbt: return NULL instead of aborting on wally-lib problems
This lets us parse invalid/bad psbt data from user input without
crashing
2020-06-23 14:49:32 +02:00
niftynei 5ecacf3dd0 psbt: add type-to-string that prints b64 string
Re-uses code from what was the bitcoin_tx_to_psbt_b64
2020-06-23 14:49:32 +02:00
ZmnSCPxj jxPCSnmZ c100de6d93 common/utils.c: Correctly handle NULL `take`n pointer in `tal_dup_talarr`.
Fixes: #3757

Reported-by: @sumBTC

Changelog-None
2020-06-23 12:48:20 +09:30
ZmnSCPxj jxPCSnmZ a25af262e9 tests/test_pay.py: Add test to replicate #3757. 2020-06-23 12:48:20 +09:30
ZmnSCPxj jxPCSnmZ 5db69f1b41 lightningd/opening_control.c: Remove 'Try fundchannel_cancel again' error.
Changelog-Changed: `fundchannel_cancel` will now succeed even when executed while a `fundchannel_complete` is ongoing; in that case, it will be considered as cancelling the funding *after* the `fundchannel_complete` succeeds.

Let me introduce the concept of "Sequential Consistency":
All operations on parallel processes form a single total order agreed upon by all processes.

So for example, suppose we have parallel invocations of `fundchannel_complete` and `fundchannel_cancel`:

                          +--[fundchannel_complete]-->
                          |
    --[fundchannel_start]-+
                          |
                          +--[fundchannel_cancel]---->

What "Sequential Consistency" means is that the above parallel operations can be serialized as a single total order as:

    --[fundchannel_start]--[fundchannel_complete]--[fundchannel_cancel]-->

Or:

    --[fundchannel_start]--[fundchannel_cancel]--[fundchannel_complete]-->

In the first case, `fundchannel_complete` succeeds, and the `fundchannel_cancel` invocation also succeeds, sending an `error` to the peer to make them forget the chanel.

In the second case, `fundchannel_cancel` succeeds, and the succeeding `fundchannel_complete` invocation fails, since the funding is already cancelled and there is nothing to complete.

Note that in both cases, `fundchannel_cancel` **always** succeeds.

Unfortunately, prior to this commit, `fundchannel_cancel` could fail with a `Try fundchannel_cancel again` error if the `fundchannel_complete` is ongoing when the `fundchannel_cancel` is initiated.
This violates Sequential Consistency, as there is no single total order that would have caused `fundchannel_cancel` to fail.

This commit is a minimal patch which just reschedules `fundchannel_cancel` to occur after any `fundchannel_complete` that is ongoing.
2020-06-22 03:15:55 +00:00
ZmnSCPxj jxPCSnmZ e8936f9d23 common/json.c: Check that JSMN result is well-formed.
xref: https://lists.ozlabs.org/pipermail/c-lightning/2020-June/000188.html

Changelog-Fixed: Reject some bad JSON at parsing.
2020-06-19 11:46:18 +09:30
Antoine Poinsot 4302afd9a5 rpc: don't go below feerate_floor when converting vbytes
We passed below the floor when the user specified `1000perkb`.
Matt Whitlock says :

    I was withdrawing with feerate=1000perkb, which should be the minimum-allowed fee rate. Indeed, bitcoin-cli getmempoolinfo reports:

    {
      "loaded": true,
      "size": 15097,
      "bytes": 9207924,
      "usage": 32831760,
      "maxmempool": 64000000,
      "mempoolminfee": 0.00001000,
      "minrelaytxfee": 0.00001000
    }

Changelog-fixed: rpc: The `feerate` parameters now correctly handle the standardness minimum when passed as `perkb`.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Reported-by: Matt Whitlock
2020-06-18 14:07:00 +02:00
Antoine Poinsot 0b99e2cc42 pytest: reproduce issue #3591
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-06-18 14:07:00 +02:00
Antoine Poinsot 3e9fcc43f9 lightningd/jsonrpc: don't assume the jcon to be alive at command execution
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-06-18 14:07:00 +02:00
niftynei 2900da6112 migrations: ignore channels that don't have a peer_id
We erase peer data after the last channel close transaction for that
peer is 100 blocks deep. We were failing to finish the migration because
the peer_id lookup on these was failing.

Now we ignore any channel with a null peer_id.

Fixes #3768
2020-06-17 14:30:41 +02:00
Rusty Russell 02338a6b25 pyln.proto.message: add to_py() operation.
This delivers the message contents in a much friendlier form for
manipulation: in particular, it makes it easy to compare two
messages without having to know all the message type internals.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-16 16:17:52 +02:00
Rusty Russell ee76504e53 pyln.proto.message: fix handling of ... with subtypes.
This time, with a test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-16 16:17:52 +02:00
Rusty Russell 2309a7a561 pyln.proto.message: don't leave 'None' in dict for missing fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-16 16:17:52 +02:00
niftynei 431463b57a listfunds: also list reserved outputs
Currently 'listfunds' lies, a teensy eeinsy bit, in that it doesn't list
all of the funds in a wallet (it omits reserved wallet UTXOs). This
change makes the reserved outputs visible by listing them in the
'outputs' section along with a new field, 'reserved', which denotes the
UTXO's state

Changelog-Changed: JSON-RPC: `listfunds` 'outputs' now includes reserved outputs, designated as 'reserved' = true
2020-06-16 15:58:58 +02:00
Christian Decker 5bb1fd4205 pyln-proto: Add Makefile targets for releases 2020-06-15 16:14:53 +02:00
Christian Decker 48cee3c3d2 pyln: Bump version to 0.8.2 in order to match with binary distro
Keeping them out of sync seems like a great way to create confusion. Let's
sync them up whenever possible.
2020-06-15 16:14:53 +02:00
Christian Decker 1933cc2523 docs: Fix typo in the `pay` command docs 2020-06-15 16:14:53 +02:00
Christian Decker cbc959ef30 pytest: Mark test_funding_push as network-specific
It checks for both exact values, that may differ due to fees, as well as the
unit which varies from network to network.
2020-06-15 16:14:53 +02:00
Rusty Russell aaefbe2e9e pyln.proto.message: fix handling of missing optional fields.
If they don't exist, that's OK. These will eventually be going away
from the spec, but there are still some in gossip messages for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell fd3ea91b44 pyln.proto.message: expose array types, add set_field for Message class.
Exposing the array types is required for our dummyrunner in the lnprototest suite, since
it wants to be able to generate fake fields.

The set_field is similarly useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell f52065201b pyln.proto.message.*: add type annotations.
Other changes along the way:

1. In a couple of places we passed None as a dummy for for
   `otherfields` where {} is just as good.
2. Turned bytes into hex for errors.
3. Remove nonsensical (unused) get_tlv_by_number() function from MessageNamespace
4. Renamed unrelated-but-overlapping `field_from_csv` and
   `type_from_csv` static methods, since mypy thought they should have
   the same type.
5. Unknown tlv fields are placed in dict as strings, not ints, for
   type simplicity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00