On-the-fly channel funding based on splicing and liquidity ads #649

t-bast · 2024-05-24T13:22:42Z

We revamp our protocols for on-the-fly funding to use a unified approach based on liquidity ads. We simplify messages exchanged and reduce the number of custom extensions required on top of the BOLT specifications.

The main changes to the protocol are:

the wallet initiates all channel operations (it is responsible for sending open_channel2 and splice_init)
- it thus automatically pays the on-chain fees for the common transaction fields, as specified by the interactive-tx protocol
- it uses a custom channel flag to ask the service provider to pay the commit tx fees and closing fees (won't be necessary anymore once we use 0-fee commit txs with package relay), which enables 0-reserve and allows emptying a wallet using an off-chain payment
we replace the pay_to_open messages with a will_add_htlc message, which is a clone of update_add_htlc without a channel_id or htlc_id:
- the wallet processes them in the IncomingPaymentHandler
- if it wishes to fulfill them, it first sends open_channel2 or splice_init to purchase enough liquidity:
  - we introduce new payment options that can be used with liquidity ads to pay the fees from the HTLCs themselves
the only fees that are applied are the liquidity ads fees:
- on-chain fees for the inputs/outputs contributed by the service provider to add inbound liquidity (as specified by the liquidity ads protocol)
- service fees for the inbound liquidity purchased (as specified by the liquidity ads protocol)

The specification can be found in lightning/blips#36. An implementation of the seller side is available in ACINQ/eclair#2861.

This PR is best reviewed commit-by-commit: each commit is self-contained and contains a piece of the puzzle.

Replaces #618.

src/commonMain/kotlin/fr/acinq/lightning/payment/IncomingPaymentHandler.kt

pm47

Review for commit e307660.

src/commonMain/kotlin/fr/acinq/lightning/channel/ChannelData.kt

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

src/commonMain/kotlin/fr/acinq/lightning/payment/LiquidityPolicy.kt

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

src/commonMain/kotlin/fr/acinq/lightning/wire/ChannelTlv.kt

pm47

Review for commit b5c187d.

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

src/commonMain/kotlin/fr/acinq/lightning/wire/LiquidityAds.kt

src/commonMain/kotlin/fr/acinq/lightning/channel/states/WaitForAcceptChannel.kt

pm47

Review for commit 22316a5.

src/commonMain/kotlin/fr/acinq/lightning/wire/LightningMessages.kt

pm47 · 2024-06-05T09:37:58Z

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

+                                // Once the channel funding is complete, we may have enough inbound liquidity to receive the payment without
+                                // an on-chain operation, which is more efficient. We thus reject that payment and wait for the sender to retry.
+                                logger.warning { "cannot accept on-the-fly funding: another funding attempt is already in-progress" }
+                                val failure = OutgoingPaymentPacket.buildWillAddHtlcFailure(nodeParams.nodePrivateKey, msg, TemporaryNodeFailure)
+                                input.send(SendOnTheFlyFundingMessage(failure))
+                                nodeParams._nodeEvents.emit(LiquidityEvents.Rejected(msg.amount, 0.msat, LiquidityEvents.Source.OffChainPayment, LiquidityEvents.Rejected.Reason.ChannelFundingInProgress))
+                            }


This constitutes a UX regression compared to the previous protocol, which was fully parallel, but seems acceptable for simplicity. A potential, not too far fetched scenario would be a popular social media account posting a BOLT 12 donation link: there would be some seemingly random failures everytime the channel gets resized.

Perhaps we can devise a more lenient mechanism in a follow up PR.

I think in that case we could keep the will_add_htlc messages around, and replay them once the current splice is finished. This would create more risks around timeouts though, so we'll need to carefully think about this.

Also, this would be sub-optimal: if we have a splice in progress, since we always buy more inbound liquidity than what we need for the associated payment, the second payment would likely go through without needing an additional splice. By failing it immediately, we give the sender an opportunity to retry (potentially automatically) and get it relayed after signing the splice.

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

pm47 · 2024-06-05T11:53:08Z

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

+                                    spliceOut = null,
+                                    requestRemoteFunding = LiquidityAds.RequestFunds(cmd.requestedAmount, cmd.fundingLease, paymentDetails),
+                                    feerate = targetFeerate,
+                                    origins = listOf(Origin.OffChainPayment(cmd.preimage, cmd.paymentAmount, totalFees))


I was expecting this channel origin to be forwarded to our peer and associated to the splice, but it's not the case. How do we expect the peer to synchronize between channel opens/splices and htlc forwarding? Something like "after an open/splice, send all htlcs that we have sent a WillAddHtlc for, and didn't receive a WillFailHtlc"?

Edit: okay the reference is in LiquidityAds.RequestFunds.paymentDetails. Isn't the channel origin kind of redundant? I'm still not clear on how exactly we deal with multiple parallel payments on the sender's side. I also assume there will be a timeout for replies to WillAddHtlc (even in the single-payment case).

Isn't the channel origin kind of redundant?

I think the paymentAmount provided in Origin.OffChainPayment is useful, and cannot be included in the payment_details.

I'm still not clear on how exactlt we deal with multiple parallel payments on the sender's side.

Can you detail what you mean by "multiple parallel payments on the sender's side"? Who is the sender here?

I also assume there will be a timeout for replies to WillAddHtlc (even in the single-payment case).

Yes, that is specified in the gist, the LSP times out will_add_htlc if it doesn't receive an answer or a matching open/splice.

src/commonMain/kotlin/fr/acinq/lightning/payment/IncomingPaymentHandler.kt

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

src/commonMain/kotlin/fr/acinq/lightning/channel/InteractiveTx.kt

t-bast · 2024-08-28T07:58:51Z

Rebased without any conflict.

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

src/commonMain/kotlin/fr/acinq/lightning/payment/LiquidityPolicy.kt

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt

It is usually the wallet that decides that it needs a channel, but we want the LSP to pay the commit fees to allow the wallet user to empty its wallet over lightning. We previously used a `please_open_channel` message that was sent by the wallet to the LSP, but it doesn't work well with liquidity ads. We remove that message and instead send `open_channel` from the wallet but with a custom channel flag that tells the LSP that they should be paying the commit fees. This only works if the LSP adds funds on their side of the channel, so we couple that with liquidity ads to request funds from the LSP. We also add a `recommended_feerates` message from the LSP which lets the wallet know the on-chain feerates that the LSP will accept for on-chain funding operations, since those feerates are set in the `open_channel` message that is now sent by the wallet.

We previously only used liquidity ads with splicing: we now support it during the initial channel opening flow as well. This lets us add more unit tests, including tests for the case where the node receiving the `open_channel` message is responsible for paying the commitment fees. We also update liquidity ads to use the latest version of the spec from lightning/bolts#1153. This introduces more ways of paying the liquidity fees, to support on-the-fly funding without existing channel balance (not implemented in this commit). Note that we need some backwards-compatibility with the previous liquidity ads types in our state serialization code: when we're in the middle of signing a splice transaction, we may have a legacy liquidity lease in our splice status. We ignore it when finalizing the splice: the only consequence is that we won't store an entry in our DB for that lease, but the channel will otherwise work correctly.

We replace the previous pay-to-open protocol with a new protocol that only relies on liquidity ads for paying fees. We simply transmit HTLCs that cannot be relayed on existing channels with a new message called `will_add_htlc` that contains all the HTLC data. The recipient can verify that the HTLC that would match this promise is valid, and if it wishes to accept that payment, it can trigger a channel open or a splice to purchase the required inbound liquidity. Once that transaction completes, the sender will relay HTLCs matching the proposed `will_add_htlc`, which completes the payment. If the fees for the inbound liquidity purchase couldn't be paid from the previous channel balance, they can be taken from the HTLCs relayed after the funding transaction. When that happens, one side needs to trust that the other will comply. Each side can independently configure the options they're comfortable with, depending on whether they trust their peer or not.

Creating a new channel has an additional cost compared to adding liquidity to an existing channel: the channel will be closed in the future, which will require paying on-chain fees. Node operators can include a `channel-creation-fee-satoshis` in their liquidity ads to cover some of that future cost.

We clarify some of our event types that previously had an `amount` field to detail whether this amount includes fees or not. This impacts: - SwapInEvents.Accepted - StoreIncomingPayment.ViaNewChannel - StoreIncomingPayment.ViaSpliceIn - Origin.OnChainWallet - Origin.OffChainPayment There was an inconsistency in the `ViaSpliceIn` event, where in some cases we used the received amount, and in others the amount with fees.

We forgot to match the `payment_hash` for this payment type, and also didn't check that the `funding_fee` was `0 msat`.

We previously forced wallets to purchase additional inbound liquidity every time an on-chain transaction was created. We now allow wallets to disable automatic liquidity purchases: the LSP will need to add enough funds on their side to cover the commitment fees, which the wallet won't be paying for. But we still make a dummy purchase of 1 sat to ensure that the liquidity ads flow is used and the wallet refunds the mining fees paid by the LSP.

Instead of using a hard-coded value from `WalletParams`, we read the liquidity funding rates from our peer's `init` message.

* enable feature FundingFeeCredit * emit a liquidity event for each liquidity purchase Independently of whether the purchase was triggered by an on-chain or off-chain payment. Also renamed `LiquidityEvents.Accepted` to `LiquidityEvents.Purchased`. * expose serviceFee in InboundLiquidityOutgoingPayment * add events PaymentEvents.PaymentSent

Adds support for liquidity-ads based protocol for on-the-fly liquidity as specified in lightning/blips#36 and lightning/blips#41, implemented respectively in ACINQ/lightning-kmp#649 and ACINQ/lightning-kmp#660. ### Lightning-kmp update Phoenixd now uses the main branch of `lightning-kmp` (v1.8.0). ### Database update - `LiquidityAds.Lease` is replaced by `LiquidityAds.Purchase`, so we need to update the liquidity table. - the `receivedWith` data have been updated in lightning-kmp, and we need a new `Part.Htlc.V1` object that may contain a `LiquidityAds.FundingFee`. With the `Lease->Purchase` change, we've updated our pattern for versioning database objects. We now have `asDb()` & `asCanonical()` mapping methods and store the type of the db object inside the json (which means we don't need the `type` column anymore, except for convenience). --------- Co-authored-by: pm47 <[email protected]>

t-bast force-pushed the extensible-continuous-liquidity-ads branch 2 times, most recently from 01ff74c to a0702b2 Compare May 30, 2024 14:02

t-bast mentioned this pull request May 30, 2024

Update protocol for continuous inbound liquidity #618

Closed

t-bast force-pushed the extensible-continuous-liquidity-ads branch from a0702b2 to 11d94a6 Compare June 4, 2024 09:57

t-bast commented Jun 4, 2024

View reviewed changes

src/commonMain/kotlin/fr/acinq/lightning/payment/IncomingPaymentHandler.kt Outdated Show resolved Hide resolved

pm47 reviewed Jun 4, 2024

View reviewed changes

pm47 reviewed Jun 5, 2024

View reviewed changes

t-bast commented Jun 6, 2024

View reviewed changes

src/commonMain/kotlin/fr/acinq/lightning/channel/InteractiveTx.kt Outdated Show resolved Hide resolved

pm47 mentioned this pull request Jun 11, 2024

Add payCommitTxFees flag to LocalParams ACINQ/eclair#2845

Merged

t-bast mentioned this pull request Jun 13, 2024

Add fee credit feature for on-the-fly funding #660

Merged

t-bast force-pushed the extensible-continuous-liquidity-ads branch 3 times, most recently from 0d0a30d to 1242282 Compare June 17, 2024 15:12

t-bast force-pushed the extensible-continuous-liquidity-ads branch 2 times, most recently from c684dcc to 172c032 Compare July 2, 2024 09:56

t-bast mentioned this pull request Jul 2, 2024

Extensible Liquidity Ads lightning/bolts#1153

Open

t-bast force-pushed the extensible-continuous-liquidity-ads branch 3 times, most recently from 55f0c2f to b895794 Compare July 8, 2024 09:51

This was referenced Jul 8, 2024

Purge PleaseOpenChannel upon disconnection #570

Closed

Ignore expired utxos migrated from the legacy app #681

Merged

t-bast marked this pull request as ready for review July 8, 2024 14:56

t-bast force-pushed the extensible-continuous-liquidity-ads branch from b895794 to aa4fc3b Compare July 9, 2024 15:41

t-bast mentioned this pull request Jul 15, 2024

Include paymentHash in LiquidityEvent.Source #538

Closed

t-bast force-pushed the extensible-continuous-liquidity-ads branch 2 times, most recently from ab9ef6e to 371ac59 Compare July 19, 2024 08:50

t-bast force-pushed the extensible-continuous-liquidity-ads branch from 371ac59 to 7f95eb4 Compare August 28, 2024 07:58

t-bast force-pushed the extensible-continuous-liquidity-ads branch from 87a1d54 to 46ef5c0 Compare September 3, 2024 14:21

pm47 reviewed Sep 5, 2024

View reviewed changes

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt Show resolved Hide resolved

t-bast force-pushed the extensible-continuous-liquidity-ads branch 2 times, most recently from baf9259 to 4211e66 Compare September 9, 2024 15:03

pm47 reviewed Sep 10, 2024

View reviewed changes

src/commonMain/kotlin/fr/acinq/lightning/payment/LiquidityPolicy.kt Outdated Show resolved Hide resolved

pm47 reviewed Sep 11, 2024

View reviewed changes

src/commonMain/kotlin/fr/acinq/lightning/io/Peer.kt Outdated Show resolved Hide resolved

t-bast force-pushed the extensible-continuous-liquidity-ads branch 2 times, most recently from 2d91f0a to 85f2de9 Compare September 16, 2024 08:20

This was referenced Sep 18, 2024

Timeout fee estimation request #265

Closed

Retrieve onchain feerate when we connect to an electrum server #490

Open

Retrieve feerates everytime we reconnect to electrum #489

Open

t-bast added 8 commits September 18, 2024 18:01

Fix payments using FromChannelBalanceForFutureHtlc

0877f36

We forgot to match the `payment_hash` for this payment type, and also didn't check that the `funding_fee` was `0 msat`.

Read remote funding rates from their init message

1b7e95b

Instead of using a hard-coded value from `WalletParams`, we read the liquidity funding rates from our peer's `init` message.

t-bast force-pushed the extensible-continuous-liquidity-ads branch from 85f2de9 to 1b7e95b Compare September 18, 2024 16:15

dpad85 mentioned this pull request Sep 20, 2024

Support for new on-the-fly funding ACINQ/phoenixd#113

Merged

pm47 approved these changes Sep 24, 2024

View reviewed changes

t-bast merged commit 149b8c3 into master Sep 25, 2024
2 checks passed

t-bast deleted the extensible-continuous-liquidity-ads branch September 25, 2024 01:13

dpad85 mentioned this pull request Sep 25, 2024

Support new on-the-fly funding ACINQ/phoenix#630

Closed

dpad85 mentioned this pull request Oct 1, 2024

Support new on-the-fly funding ACINQ/phoenix#632

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

On-the-fly channel funding based on splicing and liquidity ads #649

On-the-fly channel funding based on splicing and liquidity ads #649

t-bast commented May 24, 2024 •

edited

Loading

pm47 left a comment •

edited

Loading

pm47 left a comment

pm47 left a comment

pm47 Jun 5, 2024

t-bast Jun 6, 2024 •

edited

Loading

pm47 Jun 5, 2024 •

edited

Loading

t-bast Jun 6, 2024

t-bast commented Aug 28, 2024

On-the-fly channel funding based on splicing and liquidity ads #649

On-the-fly channel funding based on splicing and liquidity ads #649

Conversation

t-bast commented May 24, 2024 • edited Loading

pm47 left a comment • edited Loading

Choose a reason for hiding this comment

pm47 left a comment

Choose a reason for hiding this comment

pm47 left a comment

Choose a reason for hiding this comment

pm47 Jun 5, 2024

Choose a reason for hiding this comment

t-bast Jun 6, 2024 • edited Loading

Choose a reason for hiding this comment

pm47 Jun 5, 2024 • edited Loading

Choose a reason for hiding this comment

t-bast Jun 6, 2024

Choose a reason for hiding this comment

t-bast commented Aug 28, 2024

t-bast commented May 24, 2024 •

edited

Loading

pm47 left a comment •

edited

Loading

t-bast Jun 6, 2024 •

edited

Loading

pm47 Jun 5, 2024 •

edited

Loading