Update ldk-node dependency & expose bolt12 proofs

[benthecarman]

Update ldk-node and adapt payment events to its current API. Expose payment IDs in events and retain manual handling for unknown BOLT 11 payments.

Add BOLT 12 payer-proof creation to the gRPC, CLI, and MCP interfaces. Include the preimage and invoice in successful-payment events for stateless proof creation.

Use ldk-node pagination for payment history. Remove duplicate payment records from the ldk-server SQLite store, which now contains only forwarded-payment history.

[ldk-reviews-bot]

👋 Thanks for assigning @tankyleo as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

[benthecarman]

Rebased and updated ldk-node to new commit with the pagination changes. Now using the paginated payments instead of the ldk-server version

[tnull]

Looks good, but we should probably also include the new channel type in list_channels now.

[tankyleo]

Codex wants to classify this to an InvalidRequestError rather than an InternalServerError. It's not perfect, but seems InvalidRequestError is more likely if we hit PayerProofCreationFailed

[benthecarman]

Yeah not perfect mapping, but fixed

[tankyleo]

I asked Codex to cast a broader net, here's what it found, feel free to dismiss aggressively, but all seemed worth taking a look to me.

[tankyleo]

Also confirmed this patch now keeps my fans quiet on mainnet

[tankyleo]

Codex found this can race with update_all_pollable_metrics and suggests to remove fn update_payments_count entirely and have update_all_pollable_metrics be the sole writer.

I then asked about the other metrics here's what it had to say:

_ Yes, but not all in the same way.

  - Payment counters had the strongest race: update_all_pollable_metrics() performed an absolute store(), while update_payments_count() performed fetch_add() on the same value. That could permanently double-count or lose an
    update until reconciliation.

  - Balance gauges can still race: both the polling task and payment event handlers call update_all_balances(). Because both write absolute snapshots, an older, slower call could overwrite a newer snapshot. This causes
    temporary staleness, not double-counting, and the next refresh corrects it.

  - Channel metrics avoid this particular race: total_channels_count is event-driven, while public/private channel counts are poll-driven. They do not concurrently update the same atomic value.
  - Peer count is polling-only.

  For complete consistency, balance refreshes should also be serialized or assigned to a single updater. Prometheus gauges usually tolerate brief staleness, whereas payment counters decreasing or double-counting is more
  operationally problematic.

[benthecarman]

thanks fixed

[tankyleo]

Codex suggests we initialize other metrics in addition to the total channels count:

    Initialize poll-only metrics at startup

    Seed peer and channel visibility gauges before the first delayed metrics poll.\n\nAI assistance: OpenAI Codex was used for this change.

diff --git a/ldk-server/src/util/metrics.rs b/ldk-server/src/util/metrics.rs
index 1c92b87..893b1b9 100644
--- a/ldk-server/src/util/metrics.rs
+++ b/ldk-server/src/util/metrics.rs
@@ -133,9 +133,20 @@ impl Metrics {
                        Err(e) => error!("Failed to initialize payment metrics: {e}"),
                }

-               let channels_count = node.list_channels().len() as i64;
-               self.total_channels_count.store(channels_count, Ordering::Relaxed);
+               let all_channels = node.list_channels();
+               self.total_channels_count.store(all_channels.len() as i64, Ordering::Relaxed);
+
+               let public_channels_count =
+                       all_channels.iter().filter(|channel_details| channel_details.is_announced).count()
+                               as i64;
+               self.total_public_channels_count.store(public_channels_count, Ordering::Relaxed);

+               let private_channels_count =
+                       all_channels.iter().filter(|channel_details| !channel_details.is_announced).count()
+                               as i64;
+               self.total_private_channels_count.store(private_channels_count, Ordering::Relaxed);
+
+               self.update_peer_count(node);
                self.update_all_balances(node);
        }

[benthecarman]

fixed, collect all before a delayed poll

[tankyleo]

Grok 4.6 found this: should we call event_node.event_handled() in these branches too ?

[bug] ldk-server/src/main.rs:728 _ send_payment_event skips event_handled() when payment-store lookup fails, which can stall the event queue

Then codex:

[P1] Resolve payment events when payment lookup fails _
With an active subscriber, Ok(None) and Err(_) return without calling event_handled().
Since LDK Node repeatedly returns the queue-head event until acknowledged,
a missing or unreadable payment record blocks every later event and creates a tight retry loop.
Emit a partial event with payment: None, or otherwise explicitly acknowledge/drop the event;
use controlled backoff if retrieval errors should be retried.

[tankyleo]

this is the suggested patch:

diff --git a/ldk-server/src/main.rs b/ldk-server/src/main.rs
index c5fd800..e1fab26 100644
--- a/ldk-server/src/main.rs
+++ b/ldk-server/src/main.rs
@@ -706,27 +706,23 @@ fn send_payment_event(
 ) {
        if event_sender.receiver_count() == 0 {
                debug!("No event subscribers connected, skipping payment event");
-               if let Err(e) = event_node.event_handled() {
-                       error!("Failed to mark event as handled: {e}");
+       } else {
+               match event_node.payment(payment_id) {
+                       Ok(Some(payment_details)) => {
+                               let payment = payment_to_proto(payment_details);
+
+                               let event = payment_to_event(payment);
+                               if let Err(e) = event_sender.send(EventEnvelope { event: Some(event) }) {
+                                       debug!("No event subscribers connected, skipping event: {e}");
+                               }
+                       },
+                       Ok(None) => error!("Unable to find payment with payment ID: {payment_id}"),
+                       Err(e) => error!("Failed to retrieve payment with payment ID {payment_id}: {e}"),
                }
-               return;
        }

-       match event_node.payment(payment_id) {
-               Ok(Some(payment_details)) => {
-                       let payment = payment_to_proto(payment_details);
-
-                       let event = payment_to_event(payment);
-                       if let Err(e) = event_sender.send(EventEnvelope { event: Some(event) }) {
-                               debug!("No event subscribers connected, skipping event: {e}");
-                       }
-
-                       if let Err(e) = event_node.event_handled() {
-                               error!("Failed to mark event as handled: {e}");
-                       }
-               },
-               Ok(None) => error!("Unable to find payment with payment ID: {payment_id}"),
-               Err(e) => error!("Failed to retrieve payment with payment ID {payment_id}: {e}"),
+       if let Err(e) = event_node.event_handled() {
+               error!("Failed to mark event as handled: {e}");
        }
 }

[benthecarman]

done

[benthecarman]

Also fixed the proto definition to be a single string for the PageToken and it's handling in the cli.

[tankyleo]

One more comment, found with a quick "look for ways to simplify the code" with codex.

[tankyleo]

Good to squash thanks, also filed this issue in ldk-node to get clarification on this claimable_amount_msat parameter

[tankyleo]

Keep telling my AIs that we can aggressively break compat here since we haven't released yet no ?

[benthecarman]

oh woops, missed that, will remove, maybe should put in the claude.md for now, but we are so close to release so..

[benthecarman]

squashed and removed the backwards compat thing

https://github.com/benthecarman/ldk-server/compare/9e77526f9ab3a34d1c134f95f819a40359882612..738caff998a4c5e472760d6aeb8687c839256691

[tankyleo]

Worth breaking this PR into a few more commits I think ? No need for any changes to the contents. The comment below is from codex:

  > Could you split this PR into more focused commits? 5ceee90 does considerably more than update the ldk-node dependency: it also changes the hold-invoice RPCs from hash to ID, changes payment-event schemas, renames a public
  > payment field, exposes new channel/HTLC fields, and rewrites payment metrics. Likewise, 738caff combines the payment-storage migration with pagination API changes, event-acknowledgement behavior, new PaymentClaimable
  > data, and unrelated CLI/metrics refactors.
  >
  > These are independently reviewable changes_several affecting public API or wire compatibility_and each should have a commit message explaining its intent. At minimum, please separate the dependency/API adaptation, newly
  > exposed fields, payment storage and pagination migration, event-delivery behavior, and cleanup refactors. The protobuf tag compatibility fix should be kept with the isolated payment-event schema commit.

[benthecarman]

Split up. No changes made https://github.com/benthecarman/ldk-server/compare/738caff998a4c5e472760d6aeb8687c839256691..54a6192ac24f910ba2be8a7d445d620c3eb1a9a0

[tankyleo]

thank you!