Remove deprecated serialization format and fields #685
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
t-bast
force-pushed
the
remove-old-serialization
branch
from
1a8b468
to
ab7fdd4
Compare
t-bast
force-pushed
the
remove-old-serialization
branch
2 times, most recently
from
d53ef40
to
0f7796a
Compare
It is usually the wallet that decides that it needs a channel, but we want the LSP to pay the commit fees to allow the wallet user to empty its wallet over lightning. We previously used a `please_open_channel` message that was sent by the wallet to the LSP, but it doesn't work well with liquidity ads. We remove that message and instead send `open_channel` from the wallet but with a custom channel flag that tells the LSP that they should be paying the commit fees. This only works if the LSP adds funds on their side of the channel, so we couple that with liquidity ads to request funds from the LSP. We also add a `recommended_feerates` message from the LSP which lets the wallet know the on-chain feerates that the LSP will accept for on-chain funding operations, since those feerates are set in the `open_channel` message that is now sent by the wallet.
We previously only used liquidity ads with splicing: we now support it during the initial channel opening flow as well. This lets us add more unit tests, including tests for the case where the node receiving the `open_channel` message is responsible for paying the commitment fees. We also update liquidity ads to use the latest version of the spec from lightning/bolts#1153. This introduces more ways of paying the liquidity fees, to support on-the-fly funding without existing channel balance (not implemented in this commit). Note that we need some backwards-compatibility with the previous liquidity ads types in our state serialization code: when we're in the middle of signing a splice transaction, we may have a legacy liquidity lease in our splice status. We ignore it when finalizing the splice: the only consequence is that we won't store an entry in our DB for that lease, but the channel will otherwise work correctly.
We replace the previous pay-to-open protocol with a new protocol that only relies on liquidity ads for paying fees. We simply transmit HTLCs that cannot be relayed on existing channels with a new message called `will_add_htlc` that contains all the HTLC data. The recipient can verify that the HTLC that would match this promise is valid, and if it wishes to accept that payment, it can trigger a channel open or a splice to purchase the required inbound liquidity. Once that transaction completes, the sender will relay HTLCs matching the proposed `will_add_htlc`, which completes the payment. If the fees for the inbound liquidity purchase couldn't be paid from the previous channel balance, they can be taken from the HTLCs relayed after the funding transaction. When that happens, one side needs to trust that the other will comply. Each side can independently configure the options they're comfortable with, depending on whether they trust their peer or not.
Creating a new channel has an additional cost compared to adding liquidity to an existing channel: the channel will be closed in the future, which will require paying on-chain fees. Node operators can include a `channel-creation-fee-satoshis` in their liquidity ads to cover some of that future cost.
We clarify some of our event types that previously had an `amount` field to detail whether this amount includes fees or not. This impacts: - SwapInEvents.Accepted - StoreIncomingPayment.ViaNewChannel - StoreIncomingPayment.ViaSpliceIn - Origin.OnChainWallet - Origin.OffChainPayment There was an inconsistency in the `ViaSpliceIn` event, where in some cases we used the received amount, and in others the amount with fees.
We forgot to match the `payment_hash` for this payment type, and also didn't check that the `funding_fee` was `0 msat`.
We previously forced wallets to purchase additional inbound liquidity every time an on-chain transaction was created. We now allow wallets to disable automatic liquidity purchases: the LSP will need to add enough funds on their side to cover the commitment fees, which the wallet won't be paying for. But we still make a dummy purchase of 1 sat to ensure that the liquidity ads flow is used and the wallet refunds the mining fees paid by the LSP.
Instead of using a hard-coded value from `WalletParams`, we read the liquidity funding rates from our peer's `init` message.
We add an optional feature that lets on-the-fly funding clients accept payments that are too small to pay the fees for an on-the-fly funding. When that happens, the payment amount is added as "fee credit" without performing an on-chain operation. Once enough fee credit has been obtained, we can initiate an on-chain operation to create a channel or a splice by paying part of the fees from our fee credit. This feature makes more efficient use of on-chain transactions by trusting that our peer will honor our fee credit in the future. The fee credit takes precedence over other ways of paying the fees (from our channel balance or future HTLCs), which guarantees that the fee credit eventually converges to 0.
Add a parameter to our liquidity policy to limit the amount that can be allocated to fee credit. We don't want a temporary high feerate to cause wallet users to allocate too much funds towards their fee credit, which they may not use later.
If our balance, combined with our fee credit, doesn't allow us to pay the liquidity fees with the payment type we chose, we immediately abort the splice.
We previously supported having multiple channels with our peer, because we didn't yet support splicing. Now that we support splicing, we always have at most one active channel with our peer. This lets us simplify greatly the outgoing payment state machine: payments are always made with a single outgoing HTLC instead of potentially multiple HTLCs (MPP). We don't need any kind of path-finding: we simply need to check the balance of our active channel, if any. We may introduce support for connecting to multiple peers in the future. When that happens, we will still have a single active channel per peer, but we may allow splitting outgoing payments across our peers. We will need to re-work the outgoing payment state machine when this happens, but it is too early to support this now anyway. This refactoring makes it easier to create payment onion, by creating the trampoline onion *and* the outer onion in the same function call. This will make it simpler to migrate to the version of trampoline that is currently specified in lightning/bolts#836 where some fields will be included in the payment onion instead of the trampoline onion.
The v4 serialization format was released in July 2023 with better support for backwards-compatibility: the v2 and v3 formats used kotlin's serialization, which prevents us from removing legacy classes and requires additional code whenever we change something in the channel state. Users who haven't migrated their data to the v4 format will not be able to deserialize their channel data: they will need to use a previous version of the app to read their old channel data, write it to the v4 format and then upgrade to the latest version. We also remove a bunch of fields/options that were deprecated a while ago: - legacy channel funding states - unused legacy features - legacy private key generation - legacy node events
t-bast
force-pushed
the
remove-old-serialization
branch
from
0f7796a
to
44d5d86
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We do some clean-up of deprecated fields, classes and serialization formats. This is a breaking change for users who have never launched the app since the v1.5.0 release (from july 2023). It is not possible to upgrade directly from a version < 1.5.0, an intermediate update will be required.
NB: builds on top of #692