Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow HTLC receiver to dip into channel reserve #1083

Conversation

t-bast
Copy link
Collaborator

@t-bast t-bast commented May 22, 2023

When an HTLC is added, the size of the commitment transaction increases while that HTLC is pending, which also increases the on-chain fee that the channel initiator must deduce from its main output.

Before sending an HTLC, when we're not the channel initiator, we thus must ensure that our peer has enough balance to pay for the increased commitment transaction fee. We previously went further than that, and also required that our peer stayed above their channel reserve.

That additional requirement was unnecessary, and there was no matching receiver-side requirement, so it should be safe to delete.

It makes a lot of sense to allow the HTLC receiver to dip into its channel reserve to pay the fee for an HTLC they're receiving, because this HTLC will either:

  • be failed, in which case the balance goes back above channel reserve
  • be fulfilled, in which case the balance increases which also helps meet the channel reserve

This also prevents channels from being stuck if the reserve becomes dynamic (which will be the case with splicing). Without that change, we can end up in a situation where most of the channel funds are on the non-initiator side, but they cannot send HTLCs because that would make the initiator dip into their reserve to pay the increased fee.

Since this effectively shrinks the channel initiator's reserve, the sender must ensure that the resulting reserve is still large enough
to incentivize the initiator to behave honestly (for example by allowing only X% of the channel reserve to be used for fees).

I'd like to know whether implementations do check this condition on the receiver side (when receiving update_add_htlc) and verify that their channel balance is met: if it is the case, then this would be a backwards-incompatible change that could lead to force-closing channels, so we'd need to deploy this in two steps (relaxing the receiver checks first, then the sender checks).

@t-bast
Copy link
Collaborator Author

t-bast commented May 23, 2023

@rustyrussell @TheBlueMatt @Roasbeef can you please check your implementation's behavior? When you are the initiator and receive an HTLC that will make you go below your channel reserve because of the increased commit tx fee, do you accept it or do you error out?

If you error out, would you consider removing that check (on the receiver side) in your next release? This would pave the way for removing the check on the sender side as well once we're confident enough nodes have updated, which will be important to avoid stuck channels when using splicing.

@morehouse
Copy link
Contributor

Couldn't this get us into a situation where all of the receiver's channel reserve is allocated to fees while HTLCs are pending?

If so, I'm afraid we may incentivize cheating attempts.

Suppose the downstream HTLC(s) are failed while the upstream node (sender) goes offline. The receiver is then faced with the following choices:

  • Publish the latest commitment, where the entire channel reserve is burned as fees.
  • Publish an old commitment and attempt to steal funds. In the worst case the entire channel reserve is lost.

It seems cheating is incentivized in this scenario.

@TheBlueMatt
Copy link
Collaborator

TheBlueMatt commented May 23, 2023

This seems like the opposite of the way we should go here - instead of allowing us to violate a key security limit set in the protocol any concern about this should be fixed by being more conservative about when to dip into that limit. Indeed, its the case that its the one sending the message that is screwing themselves, but the protocol is strict here for a reason, and I think this is fundamentally the wrong fix.

And, yes, LDK currently does check and will error the channel if its reserve limit is violated.

@t-bast
Copy link
Collaborator Author

t-bast commented May 24, 2023

Thanks for your feedback!

Couldn't this get us into a situation where all of the receiver's channel reserve is allocated to fees while HTLCs are pending?

Yes, that can indeed happen, but I don't think this is an issue. Let's look at your example scenario: A -> B -> C:

  • B is the initiator of the A - B channel
  • B's balance is just above its channel reserve
  • A sends multiple HTLCs, which consume all of B's channel reserve in commit tx fees

If the HTLCs are fulfilled, B has no incentive to try to cheat (he's earning money), so let's only consider the case where the HTLCs are failed:

  • Remember that force-closing is costly for B:
    • B is the initiator, so he's paying the fees for the force-close transaction and he paid fees to open the channel
    • A is relaying HTLCs (since we got in this situation) that generate routing fees for B, so he would be missing out on those fees by force-closing
  • If A is online:
    • B cannot risk publishing a revoked commitment
    • B doesn't want to force-close, which would cost him on-chain fees
    • B's best choice is thus to just fail back those HTLCs, which brings its balance back again above the channel reserve
  • If A is offline:
    • If the only pending HTLCs are HTLCs from A to B that should be failed:
      • B has no reason to force-close, as he's not losing funds on those HTLCs (since they have been failed)
      • Also, B doesn't have any funds in the channel in the latest commitment (since all his funds are spent on fees)
      • But if the HTLCs are removed from the commitment, it will shift funds back from the commit tx fee into his main output
      • If Bob force-closes, he loses the opportunity of getting those funds back, as even a revoked commitment will most likely end up with A claiming everything
      • He should rather wait for A to come back online and fail those HTLCs
    • If there are other HTLCs where some funds are at risk for Bob, then he must force-close:
      • If he broadcasts the latest commitment, he is able to get funds back for those HTLCs that are at risk
      • If he broadcasts a revoked commitment, A will most likely claim everything, so B loses those HTLCs
      • Broadcasting a revoked commitment thus only makes sense if B is sure that A won't react, which he cannot know for sure

So it seems to me that the incentives are properly aligned, B always risks losing funds if he broadcasts a revoked commitment (on top of the fees he's paying for force-closing since he's the initiator).
Please make sure I haven't missed something in this analysis!

This seems like the opposite of the way we should go here - instead of allowing us to violate a key security limit set in the protocol any concern about this should be fixed by being more conservative about when to dip into that limit. Indeed, its the case that its the one sending the message that is screwing themselves, but the protocol is strict here for a reason, and I think this is fundamentally the wrong fix.

I disagree, I don't believe the protocol is strict in this case for a reason, but instead just to make the behavior uniform with other cases. I believe this specific case has no security reason for forcing the channel reserve to be met. If you think it does, can you detail why?

Making the behavior uniform would be a good reason in itself if it didn't create issues.
We had not realized that this behavior could lead to channels being stuck until I reported #728.
We mitigated this issue with the very hacky funder fee buffer, but it didn't fully fix the issue, it just made it less frequent.
Unfortunately, splicing will make this issue more frequent: we've already run into it during our splicing tests, so users will run into it, and it's extremely annoying to get out of that stuck state.
I believe that this proposal is a good fix for this issue, as long as it doesn't create the wrong incentives, but as explained above, I don't think it does.
Please correct me if you see a scenario where this leads to the wrong incentives or a security issue.

@ellemouton
Copy link
Contributor

ellemouton commented May 24, 2023

This part is a little bit fuzzy to me:

- If there are other HTLCs where some funds are at risk for Bob, then he must force-close:
    - If he broadcasts the latest commitment, he is able to get funds back for those HTLCs that are at risk
    - If he broadcasts a revoked commitment, A will most likely claim everything, so B loses those HTLCs
    - Broadcasting a revoked commitment thus only makes sense if B is sure that A won't react, which he cannot know for sure

I think we want to keep the axiom of "Bob must always stands to lose amount r (the reserve amount) if he cheats. and we never want that amount to be less than r". Anything less than r and Bob might have the attitude of "I might as well try to cheat".

If he broadcasts the latest commitment, he is able to get funds back for those HTLCs that are at risk

I think there is a chance that the outputs on the HTLCs are small enough that after paying for all their fees, and then sweeping them, then final output that Bob gets is less than r. In other words, in this scenario, Bob now stands to lose less than r and so our axiom breaks.

I totally agree that there are cases where we should allow a dip below the reserve but I think it should be dependent on the value of the HTLC (plus a healthy fee range) and hence if the effective r value is still maintained.

Also, to answer your initial question: an LND initiator receiver will currently fail if the incoming HTLC dips into the reserve: https://github.com/lightningnetwork/lnd/blob/b9b20acd4126fef40d1ff7e06afdd4654012e1cf/lnwallet/channel.go#L3630

@t-bast
Copy link
Collaborator Author

t-bast commented May 24, 2023

That's a very good point, in the scenario where B has to force-close because he also has HTLCs at risk, if all of his reserve has been moved to miner fees, the amount he would lose by cheating is the sum of those HTLCs that belong to him (minus the corresponding on-chain fees), which can be smaller than his channel reserve.

I think we want to keep the axiom of "Bob must always stands to lose amount r (the reserve amount) if he cheats. and we never want that amount to be less than r". Anything less than r and Bob might have the attitude of "I might as well try to cheat".

I'm not sure we do to be honest, at least not at the spec level. I'd leave that decision to the sender (because he's the one taking that risk to unblock its channel) on a per-implementation basis. If senders want to preserve that property, they can use one of the following strategies:

  • only send an outgoing HTLC that will make the receiver dip into its channel reserve if there is no pending HTLC from the receiver
  • only send an outgoing HTLC that will make the receiver dip into its channel reserve if the fee delta is lower than the sum of HTLCs from the receiver (by any safety margin the sender chooses)
  • only dip into part of the channel reserve (e.g. X%)

But implementations can also choose to be fine without that, because B would still lose something (even if it's smaller than the reserve) and some non-quantifiable expected revenue (from future routing and from already having a channel instead of needing to open a new one). Also, we need to consider that A earns something by catching B's cheating attempt: we should probably consider the sum of what B loses and what A earns, not just what B loses? Even though this can be tricky to analyze because it depends on all the channel history and which revoked commit B broadcasts.

I can detail in the rationale that implementations need to decide what behavior they want to implement if we are ok with that approach.

In any case, do we at least agree that the receiver (when it is the channel initiator) shouldn't check that reserve requirement? If the sender lets them dip in their channel reserve, they should happily do so, the sender is taking a risk, not them? And thus the choice is only on what behavior the sender wants to have?

@morehouse
Copy link
Contributor

An alternative solution is to not let the initiator spend down to the channel reserve in the first place. Allow some buffer, somewhere between 1 and max_accepted_htlcs times the HTLC relay fee, so that the initiator can keep receiving HTLCs to stay above the channel reserve.

For splices that increase the initiator's required reserve, the initiator can be allowed a grace period similar to what we already do for the non-initiator. During the grace period, the initiator cannot send HTLCs until they are above the reserve requirement plus buffer. They can dip into the new channel reserve to receive HTLCs provided they don't violate the previous channel reserve.

This would ensure the initiator always has at least the original channel reserve at risk. And after the initiator has a balance above the new channel reserve plus buffer, they will thereafter always have at least the new channel reserve at risk.

@morehouse
Copy link
Contributor

Another corner case to consider is an initiator reserve requirement of 0. In this case we definitely need to maintain a buffer to pay HTLC relay fees, since there's no reserve to dip into.

@t-bast
Copy link
Collaborator Author

t-bast commented May 25, 2023

An alternative solution is to not let the initiator spend down to the channel reserve in the first place.

This is already what we do with the "funder fee buffer" introduced after #728. It mitigated that kind of issue, but it isn't a perfect solution at all, so we can still reach cases where we're stuck (after an update_fee for example) and would need to dip into the initiator's channel reserve to unblock us.

For splices that increase the initiator's required reserve, the initiator can be allowed a grace period similar to what we already do for the non-initiator.

That could work in most cases, but it still doesn't solve the issue of an update_fee that puts the initiator just above the (new) channel reserve: when that happens, your channel gets stuck...we can arguably delay those update_fee when using anchor outputs, because we'd use the anchor to pay on-chain fees, but until we get package relay there are still cases where update_fee is absolutely required for security to make sure the commit tx meets the mempool mininum relay feerate.

Another corner case to consider is an initiator reserve requirement of 0. In this case we definitely need to maintain a buffer to pay HTLC relay fees, since there's no reserve to dip into.

This is already done with the "funder fee buffer".

Also, in the long run, we'll be able to get rid of this kind of issue entirely once we have package relay + v3 transactions and can make commitment transactions pay 0 fee. I really can't wait for that to happen because it's so much simpler and cleaner!

Back to my current proposal, I'd like to focus on why we shouldn't do it before considering more complex alternatives. My current understanding is that:

  1. The receiver of an HTLC, if it is the channel initiator, should always allow dipping into its channel reserve to pay the fees, as he has nothing to lose by doing so and a lot to gain (more balance on his side + routing fees if the HTLC is fulfilled)
  2. The receiver of an HTLC, if it is the channel initiator, always loses some funds (but potentially less than its channel reserve) if it tries to cheat after dipping into its channel reserve
  3. The sender of an HTLC should be able to send outgoing HTLCs when most of the channel's balance is on its side
  4. The sender of an HTLC, if it is not the channel initiator, can control how much additional risk it is taking by choosing when to dip into the receiver's reserve (other HTLCs in the commitment or not) and how much to dip into it (one HTLC vs multiple), and the spec should let them decide that for themselves

Do you disagree on any of those points? Why?

@morehouse
Copy link
Contributor

Back to my current proposal, I'd like to focus on why we shouldn't do it before considering more complex alternatives. My current understanding is that:

1. The receiver of an HTLC, if it is the channel initiator, should always allow dipping into its channel reserve to pay the fees, as he has nothing to lose by doing so and a lot to gain (more balance on his side + routing fees if the HTLC is fulfilled)

Yep, makes sense. Though this is not restricted by the current spec.

2. The receiver of an HTLC, if it is the channel initiator, always loses _some_ funds (but potentially less than its channel reserve) if it tries to cheat after dipping into its channel reserve

Loses funds compared to what? There is the corner case I mentioned previously where 100% of channel reserve is allocated to fees. In this case, if the initiator needs to go onchain (e.g., shutting down the node permanently), they may as well try to cheat, since they have nothing to lose.

And there doesn't need to be 100% of channel reserve allocated to fees before cheating attempts become higher EV than being honest. Maybe it's only 90%. Maybe it's 50%. Maybe less.

3. The sender of an HTLC should be able to send outgoing HTLCs when most of the channel's balance is on its side

Yes, that's what we're trying to solve here, though there are alternative solutions (buffer).

4. The sender of an HTLC, if it is not the channel initiator, can control how much additional risk it is taking by choosing when to dip into the receiver's reserve (other HTLCs in the commitment or not) and how much to dip into it (one HTLC vs multiple), and the spec should let them decide that for themselves

Yes, they can control their own risk. There seems to be room for this in the current spec already -- it says the sender SHOULD NOT allow dipping into the receiver's reserve but doesn't say MUST NOT. And implementations can always decide what to implement themselves, regardless of what the spec says. What I'm not sure about is changing the spec to recommend violating the channel reserve in this case.


Some thoughts:

If we decide to let the receiver dip into their reserve, we should really set a limit to how far they can dip into their reserve. Let's call this limit X such that we consider it unsafe to dip more than X sats into their reserve because then the receiver might be tempted to cheat. Then the "real" reserve requirement we cannot violate is channel_reserve - X.

This is equivalent to simply setting the original channel reserve requirement to channel_reserve - X and requiring a buffer of X. With either approach, we're SOL if an update_fee increases HTLC relay costs by more than X. The only difference is that with the second approach the channel_reserve requirement actually means what it says.

@t-bast
Copy link
Collaborator Author

t-bast commented May 25, 2023

Loses funds compared to what? There is the corner case I mentioned previously where 100% of channel reserve is allocated to fees. In this case, if the initiator needs to go onchain (e.g., shutting down the node permanently), they may as well try to cheat, since they have nothing to lose.

Loses funds by publishing a revoked commitment instead of:

  1. If they have HTLCs at risk, publishing the latest commitment
  2. If they have no HTLCs at risk, waiting for their peer to come back online
  3. If they have no HTLCs at risk but their peer sees to be gone, publishing the latest commitment

Assuming that the revoked commitment is caught and punished (otherwise, there's an incentive to cheat regardless of the channel reserve):

  • In case (1), they would lose the HTLCs they have at risk
  • In case (2), they would pay on-chain fees that could be avoided, and lose their remaining balance (which may be below their channel reserve)
  • In case (3), they would lose their remaining balance (which may be below their channel reserve)

I agree with you that if all of their remaining balance is consumed in on-chain fees, then there are cases where they wouldn't lose any funds. The sender should thus make sure that the receiver still has an output in the commitment transaction, with an amount that is not too far below the channel reserve (where each sender chooses their safety value).

I will re-work the PR to present if this way: "the sender may dip into the channel reserve, but shouldn't exceed some threshold they're comfortable with".


Regarding the buffer on top of the reserve: as I previously mentioned, we already have that mechanism in place. But it is not sufficient, as the following scenario highlights:

  • Alice and Bob have a 1 000 000 sat channel where Alice is the initiator, reserve is set to 1% (10 000 sat), and the balance is:
    • Alice = 15 000 sat
    • Bob = 985 000 sat
  • Bob splices-in an additional 500 000 sat, which makes the reserve grow to 15 000 sat, and the balance is now:
    • Alice = 15 000 sat (exactly at the reserve)
    • Bob = 1 485 000 sat

Alice has now "lost" her buffer, so it's not a good enough solution to ensure that the channel doesn't get stuck.

Your suggestion was to use the previous reserve after that splice, but what do you do if other splices are performed? You'll need to track a list of previous reserves, which can probably open up games where your peer makes a splice-out first (to shrink the reserve) and a splice-in afterwards (to grow it back, while still being allowed to use the smaller previous reserve)...

I think that allowing the sender to temporarily dip into the receiver's reserve for HTLCs that will make the receiver increase their balance if fulfilled is simpler to reason about and implement (while taking into account the risk exposure that you and @ellemouton rightfully reported).

When an HTLC is added, the size of the commitment transaction increases
while that HTLC is pending, which also increases the on-chain fee that
the channel initiator must deduce from its main output.

Before sending an HTLC, when we're not the channel initiator, we thus
must ensure that our peer has enough balance to pay for the increased
commitment transaction fee. We previously went further than that, and
also required that our peer stayed above their channel reserve.

That additional requirement was unnecessary, and there was no matching
receiver-side requirement, so it should be safe to delete.

It makes a lot of sense to allow the HTLC receiver to dip into its
channel reserve to pay the fee for an HTLC they're receiving, because
this HTLC will either:

- be failed, in which case the balance goes back above channel reserve
- be fulfilled, in which case the balance increases which also helps
  meet the channel reserve

This also prevents channels from being stuck if the reserve becomes
dynamic (which will be the case with splicing). Without that change,
we can end up in a situation where most of the channel funds are on
the non-initiator side, but they cannot send HTLCs because that would
make the initiator dip into their reserve to pay the increased fee.

Since this effectively shrinks the channel initiator's reserve, the
sender must ensure that the resulting reserve is still large enough
to incentivize the initiator to behave honestly (for example by
allowing only X% of the channel reserve to be used for fees).
@t-bast t-bast force-pushed the relax-htlc-receiver-reserve-requirement branch from 2ddc9f3 to cc232a8 Compare May 25, 2023 15:50
@morehouse
Copy link
Contributor

morehouse commented May 25, 2023

Regarding the buffer on top of the reserve: as I previously mentioned, we already have that mechanism in place. But it is not sufficient, as the following scenario highlights:

* Alice and Bob have a 1 000 000 sat channel where Alice is the initiator, reserve is set to 1% (10 000 sat), and the balance is:
  
  * Alice = 15 000 sat
  * Bob = 85 000 sat

* Bob splices-in an additional 500 000 sat, which makes the reserve grow to 15 000 sat, and the balance is now:
  
  * Alice = 15 000 sat (exactly at the reserve)
  * Bob = 135 000 sat

Alice has now "lost" her buffer, so it's not a good enough solution to ensure that the channel doesn't get stuck.

This scenario is solved by the grace period. Specifically, Alice is not allowed to send any HTLCs, but she can receive HTLCs as long as the fees don't put her below 10,000 sats. Once she's above 15,000 sats plus the buffer, the grace period is over.

Your suggestion was to use the previous reserve after that splice, but what do you do if other splices are performed? You'll need to track a list of previous reserves, which can probably open up games where your peer makes a splice-out first (to shrink the reserve) and a splice-in afterwards (to grow it back, while still being allowed to use the smaller previous reserve)...

I'm not understanding the problem with multiple splices, whether they happen concurrently or not.

Non-concurrent splices

Each time a splice confirms to sufficient depth, all commitments for the old funding transaction are invalidated and can be thrown away. We only need to track the previously confirmed funding transaction and any splices that are still in flight. So splicing out to shrink the reserve and then splicing back in to grow the reserve causes no problems:

  • Once the splice out confirms, the smaller reserve takes effect and is sufficient protection for the smaller capacity.
  • Once the splice back in confirms, the smaller reserve is still sufficient protection if the peer's balance is below the new reserve. The peer cannot publish any commitments from before the splice out that might have a higher balance.
  • We allow the peer to receive HTLCs but not send HTLCs (grace period).
  • Once the peer's balance goes above the new reserve, the smaller reserve may no longer be sufficient protection, so we start enforcing the new reserve instead.
  • We can now forget the previous reserve requirement.

Concurrent splices

If we want to send/receive HTLCs while there are multiple unconfirmed splices in flight, we already need to track all potential channel capacities and reserves and ensure we satisfy all of them while splices are in flight. In the example above, even though Bob's new channel balance will be 135,000 sats he cannot spend more than 75,000 sats until the splice is confirmed to a sufficient depth. Otherwise Bob's balance on the pre-splice commitment balance would be below his channel reserve.

Again, we already need to remember all potential channel reserves to do HTLCs safely while splicing. Adding a grace period doesn't make this more difficult. If one or more pending splices increase the reserve, we simply apply the grace period rule when deciding whether to add an HTLC to all commitments. This should be a small branch in the HTLC-handling code.

And once a potential splice confirms to sufficient depth, we can forget everything about the other commitments and channel reserves, as we normally would.


I think that allowing the sender to temporarily dip into the receiver's reserve for HTLCs that will make the receiver increase their balance if fulfilled is simpler to reason about and implement (while taking into account the risk exposure that you and @ellemouton rightfully reported).

I'm not sure it really is simpler. If all implementations have already implemented a buffer, it seems to me it would be simpler to tune the size of the buffer than to add new code.

And the grace period doesn't seem very tricky to me either.

@t-bast
Copy link
Collaborator Author

t-bast commented May 26, 2023

Let me add to the previous scenario to explain why keeping track of previous reserves needs additional code that I think isn't entirely trivial. The reserve is set to 1% at every step.

  • State 1: Alice and Bob have a 1 000 000 sat channel where Alice is the initiator, and the balance is:
    • Alice = 15 000 sat
    • Bob = 985 000 sat
    • reserve = 10 000 sat
  • State 2: Bob splices-in an additional 500 000 sat:
    • Alice = 15 000 sat
    • Bob = 1 485 000 sat
    • reserve = 15 000 sat
    • this transaction confirms -> Alice and Bob can forget the previous commitment
  • State 3: Bob splices-in an additional 500 000 sat:
    • Alice = 15 000 sat
    • Bob = 1 985 000 sat
    • reserve = 20 000 sat
    • this transaction confirms -> Alice and Bob can forget the previous commitment

With that proposal, we now need to remember previous channel reserves even though we've forgotten the previous commitments.
In state 2, there is a subtlety: Alice is meeting the new reserve requirement, but we should still remember the previous reserve to be able to add HTLCs.
In state 3, we still need to remember the reserve from state 1.
Once Alice's balance grows to 15 000 sat + some buffer while staying below 20 000 sat + some buffer, we can finally forget the reserve from state 1, but still need to remember the reserve from state 2.

But what buffer should we use?
Should it be indexed on the current commit_feerate (otherwise an update_fee could screw up our accounting)?
What happens when that buffer isn't met anymore (we may have forgotten the previous reserve and the corresponding commitment by then)?

Again, we already need to remember all potential channel reserves to do HTLCs safely while splicing.

That is only true while the splice transaction is unconfirmed. As soon as it is confirmed, we can forget the previous channel reserve (and we currently do), but we can't anymore if we need to keep applying them to the new commitments.

I'm not sure it really is simpler. If all implementations have already implemented a buffer, it seems to me it would be simpler to tune the size of the buffer than to add new code.

But there will never be a buffer size that can be guaranteed to be enough, that's why I believe this solution is unsufficient.
The channel initiator is allowed to dip into this buffer with an update_fee, because that update_fee may be critical for funds safety (otherwise the commit transaction might not meet the mempool min-relay-fee).
So there will always be cases (even without taking splicing into account) where the channel initiator may meet its channel reserve, but not the additional buffer.
This means that in the worst case, the channel initiator will be exactly at its channel reserve.
When that happens, the only solutions I see are:

  1. Unblock the channel by sending dust HTLCs from the non-initiator to the initiator until the buffer is met again
  2. Make a splice-out to lower the reserve
  3. Dip into the initiator's channel reserve

Solutions (1) and (2) are clearly impractical, that's why I think we must do solution (3). I'm not very satisfied by this, but until we have package relay and 0-fee commitment transactions, I don't think we can have a satisfying solution to that kind of issues (update_fee is a necessary evil that none of us like, but we can't really live without it yet).

The simplest form of solution (3) that has the lowest security impact is that the sender restricts itself to a single HTLC that dips into the receiver's channel reserve. This lets them unblock the channel, while only lowering the security by weight_of_htlc_output * commit_feerate. This is also completely trivial to implement (much simpler than keeping track of previous reserves), which is a good thing!

@morehouse
Copy link
Contributor

Let me add to the previous scenario to explain why keeping track of previous reserves needs additional code that I think isn't entirely trivial. The reserve is set to 1% at every step.

* State 1: Alice and Bob have a 1 000 000 sat channel where Alice is the initiator, and the balance is:
  
  * Alice = 15 000 sat
  * Bob = 985 000 sat
  * reserve = 10 000 sat

* State 2: Bob splices-in an additional 500 000 sat:
  
  * Alice = 15 000 sat
  * Bob = 1 485 000 sat
  * reserve = 15 000 sat
  * this transaction confirms -> Alice and Bob can forget the previous commitment

* State 3: Bob splices-in an additional 500 000 sat:
  
  * Alice = 15 000 sat
  * Bob = 1 985 000 sat
  * reserve = 20 000 sat
  * this transaction confirms -> Alice and Bob can forget the previous commitment

With that proposal, we now need to remember previous channel reserves even though we've forgotten the previous commitments. In state 2, there is a subtlety: Alice is meeting the new reserve requirement, but we should still remember the previous reserve to be able to add HTLCs. In state 3, we still need to remember the reserve from state 1. Once Alice's balance grows to 15 000 sat + some buffer while staying below 20 000 sat + some buffer, we can finally forget the reserve from state 1, but still need to remember the reserve from state 2.

At any time, we only need to store a single grace_period_reserve and a flag grace_period for the current channel.

  • State 1: grace_period = false.
  • State 2: grace_period = true, grace_period_reserve = 10,000. Can forget previous commitment.
  • State 3: grace_period = true, grace_period_reserve = 10,000. Can forget previous commitment.

Since we never got out of the initial grace period, the reserve carries over to the next splice.

The reason this works is that Alice can't send any HTLCs until she gets above the new reserve (plus buffer), and therefore any revoked commitments she publishes would actually benefit Bob. So if she never exits the grace period, we can safely roll over the grace_period_reserve indefinitely.

Only once Alice gets out of the grace period and can start sending HTLCs again do we need to update grace_period_reserve for future splices. At this point, we can immediately forget the previous grace_period_reserve.

But what buffer should we use? Should it be indexed on the current commit_feerate (otherwise an update_fee could screw up our accounting)?

The buffer must always be current, regardless of the grace_period_reserve. Implementations should be keeping the buffer current already, or they would be screwed during relay fee spikes.

What happens when that buffer isn't met anymore (we may have forgotten the previous reserve and the corresponding commitment by then)?

The buffer exists to avoid going below the channel reserve when receiving HTLCs. If we don't meet the buffer, not a big deal, as long as we can keep meeting the channel reserve when receiving HTLCs. And obviously we can't send HTLCs while the buffer is violated.

As described in the existing spec, the buffer should be chosen to handle fee spikes. If it can't do that, the buffer is too small.

Again, we already need to remember all potential channel reserves to do HTLCs safely while splicing.

That is only true while the splice transaction is unconfirmed. As soon as it is confirmed, we can forget the previous channel reserve (and we currently do), but we can't anymore if we need to keep applying them to the new commitments.

As described above, we can forget everything except a single grace_period_reserve that we store with the channel data.

I'm not sure it really is simpler. If all implementations have already implemented a buffer, it seems to me it would be simpler to tune the size of the buffer than to add new code.

But there will never be a buffer size that can be guaranteed to be enough, that's why I believe this solution is unsufficient. The channel initiator is allowed to dip into this buffer with an update_fee, because that update_fee may be critical for funds safety (otherwise the commit transaction might not meet the mempool min-relay-fee). So there will always be cases (even without taking splicing into account) where the channel initiator may meet its channel reserve, but not the additional buffer. This means that in the worst case, the channel initiator will be exactly at its channel reserve. When that happens, the only solutions I see are:

1. Unblock the channel by sending dust HTLCs from the non-initiator to the initiator until the buffer is met again

2. Make a splice-out to lower the reserve

3. Dip into the initiator's channel reserve

Solutions (1) and (2) are clearly impractical, that's why I think we must do solution (3). I'm not very satisfied by this, but until we have package relay and 0-fee commitment transactions, I don't think we can have a satisfying solution to that kind of issues (update_fee is a necessary evil that none of us like, but we can't really live without it yet).

The simplest form of solution (3) that has the lowest security impact is that the sender restricts itself to a single HTLC that dips into the receiver's channel reserve. This lets them unblock the channel, while only lowering the security by weight_of_htlc_output * commit_feerate. This is also completely trivial to implement (much simpler than keeping track of previous reserves), which is a good thing!

Dipping into the channel reserve does not solve this problem better than the existing buffer solution. There MUST be some limit on how far we are comfortable dipping into the channel reserve, let's call it X. Surely X is less than 100% of the channel reserve, otherwise we may as well eliminate the reserve.

If update_fee causes weight_of_htlc_output * commit_feerate to exceed X, we won't be able to dip into the reserve to receive HTLCs anymore.

The result is identical to what would happen using a channel reserve X sats lower with a buffer of X. This PR is basically proposing to widen the buffer by going below channel reserve, rather than by keeping the channel reserve firm and widening the buffer on the other end.

So if today we're getting stuck channels in practice, an equivalent solution is to reconsider and adjust the channel reserve and buffer values we're using. If we are actually comfortable with a lower reserve, let's adjust it. If we also need more buffer, let's adjust that. This equivalent solution has the benefit of keeping the definition of "channel reserve" the same, in that it is the lower bound on channel balance either peer can have.

@TheBlueMatt
Copy link
Collaborator

What about the case where Alice has opened a channel to Bob. Alice has some balance and sends 400 dust HTLCs to Bob. Alice now has some balance which is exactly their required reserve. Bob now wishes to send a single non-dust HTLC to Alice, and in doing so pushes Alice's reserve balance into dust. This gives Alice a state where a large portion of the commitment transaction goes to mining fees with unenforceable HTLCs. Finally, the 400 dust HTLCs clear with a preimage, "giving" Bob that value.

Alice now broadcasts the stale state, burning HTLCs which should have gone to Bob to fees. If Alice is a miner and mines a block within a few days, Alice gets these HTLCs back.

In the 0-HTLC-fee anchor case this is a bit better because the dust threshold is much lower or really such HTLCs are burned as unclaimable rather than given back to Alice the miner, but I'm still not very comfortable with that.

Dust HTLCs are ultimately enforced by the reserve value, as imperfect as that is I'm not super comfortable with breaking it more.

Another approach to addressing this is Rusty's single-direction-updates-at-a-time stuff where Bob world identify that the HTLC he wants to add will violate reserve before he actually commits to it, allowing him to reject it instead. Alice must still ensure any fee updates she wants to push allow for at least one HTLC from Bob but everyone should be doing that already.

@t-bast
Copy link
Collaborator Author

t-bast commented May 29, 2023

If update_fee causes weight_of_htlc_output * commit_feerate to exceed X, we won't be able to dip into the reserve to receive HTLCs anymore.

Indeed, but you are already supposed to force-close if your peer proposes an absurdly large update_fee. With anchor outputs, the commit_feerate only needs to get you into the lowest bucket of the mempool, and is currently capped by all implementations.

The fee for an additional HTLC at 10 sat/byte is 430 sat (~ 10 cents). Even at 100 sat/byte (which is much larger than the highest historical mempool min-relay-fee), this isn't a very attractive target for attackers?

The result is identical to what would happen using a channel reserve X sats lower with a buffer of X. This PR is basically proposing to widen the buffer by going below channel reserve, rather than by keeping the channel reserve firm and widening the buffer on the other end.

Yes, that's a good way of looking at it. I've given this more thought, and I agree that the end result is quite similar, if we have a solution for the splice scenario where one peer drops below the new reserve.

The only issue I see is that while the reserve is strict, the buffer is not, so nothing would prevent the initiator from consuming all of its buffer and have a balance of exactly the channel reserve. This shouldn't happen with honest participants, but since this is an area of the code that is quite complex (because it's hard to figure out the possible future states of your commitment when there are proposed unsigned update_add_htlc / update_fail_htlc / update_fulfill_htlc on both sides and an update_fee is required), we will likely have some bugs in edge cases.

At any time, we only need to store a single grace_period_reserve and a flag grace_period for the current channel.

I think that would work, at the cost of two new fields in the channel data. We would check whether we're out of the grace period whenever we send/receive revoke_and_ack.

In summary, we currently have two proposed solutions:

  1. Allow senders to dip into the receiver channel reserve (up to an amount they choose)
  2. Keep channel reserve strict, but use old reserve after splices until the new reserve is met and rely on non-strict buffers to accept HTLCs / fee changes

I would be ok with either of these solutions. I'm still convinced that the first solution is simpler, both conceptually and in terms of implementation, but I agree that it forces senders to take an extra risk. However, this is IMHO a very controlled risk as the fee for one HTLC is pretty low, even at high feerates.

Bob now wishes to send a single non-dust HTLC to Alice, and in doing so pushes Alice's reserve balance into dust.

To push Alice's balance into dust with a single HTLC, the channel reserve would either need to be very low or the feerate very high (see HTLC cost at the beginning of this message). In this case, Alice may probably still want to cheat even if her balance was still in the commitment, since it would be a very small balance anyway?

Note that I'm dismissing a far-away future where every sat is economically relevant, because I hope that we'll be able to get 0-fee commitment transactions before that happens, so any solution we're trying to find for this issue is only there to protect us for the next couple of years, but should be obsolete after that.

@TheBlueMatt
Copy link
Collaborator

To push Alice's balance into dust with a single HTLC, the channel reserve would either need to be very low or the feerate very high (see HTLC cost at the beginning of this message). In this case, Alice may probably still want to cheat even if her balance was still in the commitment, since it would be a very small balance anyway?

Fair point, but replace one with 400, the example still works.

I'm not convinced this is the simplest method to address the stuck channel issue. Two other options would be to implement rustys one-direction-updates-at-a-time thing or to limit the number of new HTLCs in each direction's updates to, eg, 4, and then set the buffer to 4.

Maybe more importantly, you have more stats on how often this happens in practice? (And not just if it happened but if the peer maybe could have avoided it/doesn't implement a reasonable buffer).

@t-bast
Copy link
Collaborator Author

t-bast commented May 31, 2023

Fair point, but replace one with 400, the example still works.

But we don't need to replace it with 400, the sender will only dip into the receiver's reserve once, for a single HTLC: that's sufficient to unblock the channel. This way the sender's additional risk exposure remains pretty low, while making progress towards shifting liquidity back to the receiver side.

It really is just one if statement to modify before sending an outgoing HTLC. All implementations currently have an if that does something like:

if (remote_reserve_not_met_in_next_commitment) {
    // don't send HTLC
}

We would simply change it to something like:

val can_dip_into_reserve = remote_reserve_met_in_current_commitment && proposed_outgoing_htlcs.isEmpty()
if (remote_reserve_not_met_in_next_commitment && !can_dip_into_reserve) {
    // don't send HTLC
}

Two other options would be to implement rustys one-direction-updates-at-a-time thing

In the longer term, sure, that will give us an opportunity to find a better solution without caring about backwards-compatibility. But changing the whole commitment update mechanism isn't a simple change and we can't expect to have that soon.

to limit the number of new HTLCs in each direction's updates to, eg, 4, and then set the buffer to 4.

I think this will be a change that would trigger a lot of debate, and I'm not sure we'd get consensus on that...You'd also still have the issue that whenever there's an update_fee, that may dip into the buffer.

Maybe more importantly, you have more stats on how often this happens in practice? (And not just if it happened but if the peer maybe could have avoided it/doesn't implement a reasonable buffer).

It almost never happens since we added the funder fee buffer. Without splicing, I think we can dismiss it as just a theoretical issue that doesn't happen in practice. But with splicing, this happens all the time when one side does a splice-in, so we need a solution before we can ship splicing (either dip into reserve, or as @morehouse suggests, use the older reserve until the new one is met).

@t-bast
Copy link
Collaborator Author

t-bast commented Jun 7, 2023

As discussed during the spec meeting, I'm closing this PR in favor of @morehouse's proposal to specifically handle this for splicing by tracking the old reserve until the new one is met (thanks for the thorough discussion!).

I still think that implementations should remove their existing receiver check (if you receive an HTLC that makes you dip into your reserve, you should be happy to let it go through as the spec doesn't tell you to reject it), so that senders may still choose to take that risk and dip into their peer's reserve if they want to (even though the spec says you should not do it).

@t-bast t-bast closed this Jun 7, 2023
@t-bast t-bast deleted the relax-htlc-receiver-reserve-requirement branch June 7, 2023 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants