BOLT12 sending doesn't allow to override {Route,Payment}Parameters

[tnull]

For BOLT11 we allow to set RouteParameters as an argument to send_payment. However, pay_for_offer only allows to set max_total_routing_fee_msat, but not other routing-related fields such as max_total_cltv_expiry_delta, max_path_count, max_channel_saturation_power_of_half.

I'm a bit dubious if pay_for_offer and paying for refunds should be parametrized by RouteParameters, PaymentParameters, or the relevant fields independently, but we should provide a way to set them for BOLT12 payments, too.

[TheBlueMatt]

Right, so the {Route,Payment}Parameters abstractions don't make any sense in a BOLT 12 world. They also don't really make sense in a BOLT 11 world if we pass a Bolt11Invoice directly to ChannelManager (which we should do now that lightning depends on lightning-invoice).

So I think this tees us up for a nice patchset - define some new struct which captures the options you describe, pass it to pay_for_offer and use that + the BOLT 12 invoice to build an initial RouteParameters, then do the same on the BOLT 11 side - drop lightning::ln::bolt11_payment and replace it with a Channelmanager method that does the same thing as the BOLT 12 path, just with BOLT 11 and the new parameters struct.

[shaavan]

Giving this a shot! 🚀

[vincenzopalazzo]

It this issue include two separate PRs @TheBlueMatt ?

The first is adding some kind of structure (Probably with @shaavan did in #3342 (review)) for the BOLT 12 world that includes the parameters needed by @tnull

Then, add a method inside the ChannelManager like pay_for_offer that do the same thing like with the bolt11.

Not sure what you mean by dropping lightning::ln::bolt11_payment and replacing it with a Channelmanager methods but probably I am missing something here

[shaavan]

Hey @vincenzopalazzo!

From what I understand, the {bolt11, bolt12} invoice contains the PaymentParams, and RouteParams includes the PaymentParams plus two additional fields

Since send_payment_for_bolt12_invoice takes the whole BOLT12Invoice as input, it can build the RouteParams on the fly based on BOLT12Invoice::PaymentParams.

Now, if we tweak the bolt11's send_payment to also take the bolt11 invoice directly as input, it could construct the RouteParams similarly, eliminating the need for that extra step of manually extracting params from the invoice.

Given that lightning::ln::bolt11_payment contains the functions for extraction, this change will make the file redundant, allowing us to clean things up a bit by removing it.

@TheBlueMatt, does this sound right to you? Thanks so much!

[TheBlueMatt]

It this issue include two separate PRs @TheBlueMatt ?

That's how I was thinking of it, yea.

Then, add a method inside the ChannelManager like pay_for_offer that do the same thing like with the bolt11.

Right.

Not sure what you mean by dropping lightning::ln::bolt11_payment and replacing it with a Channelmanager methods but probably I am missing something here

Once we have ChannelManager::pay_for_bolt11_invoice, lightning::ln::bolt11_payment should be basically redundant and can just be dropped.

From what I understand, the {bolt11, bolt12} invoice contains the PaymentParams, and RouteParams includes the PaymentParams plus two additional fields

Almost, but its still kinda a mix. The old PaymentParameters/RouteParameters distinction was/is "RouteParameters are the params for picking an individual route, PaymentParameters are the ones that apply to all route-finding session for a payment across retries". Thus, you can convert a BOLT 11/12 invoice plus some additional options into a PaymentParameters and then outbound_payment can build per-path-finding RouteParameters.

With pay_for_offer/pay_for_bolt12_invoice/a future ChannelManager::pay_for_bolt11_invoice we need to change the distinction to "ManualRoutingParameters (or maybe a different name) contains all the parameters that the sender picks, with the rest coming from the BOLT 11/12 invoice".

Fields like max_total_cltv_expiry_delta, max_path_count, max_path_length (though the invoice could further reduce the max length the user specifies), and max_channel_saturation_power_of_half are all things the user should be able to specify, whereas payee/expiry_time/previously_failed_* are things that the invoice/payment session should take control over.

[shaavan]

@TheBlueMatt

Thank you so much for the detailed explanation! That really clears things up.

In the current approach I implemented in #3342, I've kept the PaymentParameters / RouteParameters distinction but introduced a separate RouteParametersOverride to let users adjust specific parameters.

However, now I think shifting to a UserRouteParams / InvoiceRouteParams distinction feels cleaner and more intuitive. To keep a record of the current method for comparison, I’ll go ahead and open a new PR with this new distinction.

I really appreciate the guidance—thanks again!

[jkczyz]

Almost, but its still kinda a mix. The old PaymentParameters/RouteParameters distinction was/is "RouteParameters are the params for picking an individual route, PaymentParameters are the ones that apply to all route-finding session for a payment across retries". Thus, you can convert a BOLT 11/12 invoice plus some additional options into a PaymentParameters and then outbound_payment can build per-path-finding RouteParameters.

With pay_for_offer/pay_for_bolt12_invoice/a future ChannelManager::pay_for_bolt11_invoice we need to change the distinction to "ManualRoutingParameters (or maybe a different name) contains all the parameters that the sender picks, with the rest coming from the BOLT 11/12 invoice".

Fields like max_total_cltv_expiry_delta, max_path_count, max_path_length (though the invoice could further reduce the max length the user specifies), and max_channel_saturation_power_of_half are all things the user should be able to specify, whereas payee/expiry_time/previously_failed_* are things that the invoice/payment session should take control over.

@TheBlueMatt Currently, we serialize RouteParameters with Route. Not sure if we serialize it elsewhere. Given that, what are you thinking about for updating the serialization format and backwards compatibility?

[TheBlueMatt]

However, now I think shifting to a UserRouteParams / InvoiceRouteParams distinction feels cleaner and more intuitive.

Hmm, I'm not sure I understand the motivation for that. The way I was thinking about it is that we'd continue to keep the PaymentParameters/RouteParameters's around as a general encapsulation of the parameters we use when finding a route/paying. Those would continue to be built from BOLT 11/12 invoices as well as user parameters.

I thought #3342 made sense - the high-level interface is "parameters + BOLT 11 invoice/BOLT 12 offer" which then (basically, even if not actually) calls the low-level interface which just takes a PaymentParameters (which gets turned into RouteParameters on a per-path basis).

[shaavan]

Hey @TheBlueMatt,

First off, thank you for the pointers and for helping me get a clearer understanding of the approach.

My initial thought behind suggesting a shift to a UserRouteParams / InvoiceRouteParams distinction was to streamline things by merging the two-level distinction (UserParams / Params from Invoice at a higher level and RouteParams / PaymentParams at a lower level) into a single, cohesive abstraction. The idea was to make the way we pass UserParams or Params from Invoice align directly with how they're used.

However, after diving deeper and working through it hands-on, I began to see some conceptual gaps with this single abstraction approach. As you pointed out:

“PaymentParameters are the ones that apply to all route-finding sessions for a payment across retries.”

I now see how this distinction really does make sense, given our current code structure. So, even if the two-level approach isn’t the most minimalistic, it does seem to be the more accurate one conceptually.

Thanks again for your valuable feedback and for guiding me to view this from the right angle. I’ll go ahead and continue building on #3342!