Skip to content

fix(net): judge a pending tail by recorded arrivals, and error a short one - #5802

Merged
kixelated merged 3 commits into
mainfrom
quest/m1/tail-arrivals
Oct 17, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m1/tail-arrivals

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Quest: quest/m1/tail-arrivals.md (deleted here, references repointed).

Summary

A received track now judges which groups its pending tail still owes from a per-track record of live arrivals instead of scanning the cache.

  • TrackState::arrivals is a Ledger (new rs/moq-net/src/model/ledger.rs) of the sequences the live feed delivered, as a group (at its header) or a datagram. Fetched copies never count, and eviction never touches it.
  • The run/fold logic moved out of tail::Tail into Ledger unchanged, so the session's tail and the track's record give up on a gap by one rule. Tail keeps its API and delegates.
  • A gap between two arrivals folds once it is a subscription's grace old, using tail::grace of the track's aggregate max delay. Only a write folds (an arrival or an abort), so the end check needs no clock.
  • The end check (is_complete) no longer scans the cache: awaits_tail asks the ledger, and withholds reads a count of withheld (pending) slots kept in step at every insert, reveal, and removal, with a debug_assert against the scan.
  • An abort while the pending tail still owes a group is no longer settled: readers get the abort error instead of a clean end. A reader that got every group still ends cleanly, and an evicted group or a datagram now counts as delivered.

Decisions (settled by the maintainer)

The maintainer accepted all four on 2026-10-16.

  1. A gap older than the grace counts as lost, not owed. The open question was whether an arrival that ages past grace before the end is declared still counts. Arrivals never age out: runs merge, so the record is bounded by gaps rather than arrivals. A gap between arrivals that ages past the grace folds into accounted, so it neither holds readers nor fails them on abort: the group was lost before the abort, not cut off by it, and the session's Tail already gives up on it by the same rule. Alternative: keep such holes and error on abort, which fails a reader for a loss it already skipped and needs unbounded hole storage on a lossy track.
  2. The gap below the first arrival never folds, matching Tail (only gaps between runs fold). With a declared start below the first arrival, an abort before the session settles errors readers. Alternative: fold it from the first arrival's time, which reopens questions about a lowered start (Tail::demand).
  3. Grace comes from the model's own aggregate subscription (tail::grace(max_delay)), not plumbed from the session. That keeps the model self-contained toward the public received-end API (quest/m1/public-model/ingest.md). IETF's session tail uses the fixed GRACE, so with a nonzero max delay the model may fold sooner than the IETF session settles. The subscriptions lock is taken under the track lock only when a gap exists; it is a leaf lock.
  4. The received-end shape goes back to quest/m1/public-model/ingest.md. The note added there records that this PR landed the arrival record without a grace timer (a gap folds only on a write, so the session still settles with set_tail_pending(false)), and leaves the public shape to that quest.

Public API and wire

Public API: behavior only. A received track whose pending tail is cut off by an abort now ends readers with the abort error instead of a clean end; poll_complete reports it as cut off. Wire: none.

Benchmark

New track_pending_tail (cargo bench -p moq-net --features fuzz --bench tail), re-polling a reader held at a hole, swept over tracks x groups cached above it (short runs on a loaded machine):

tracks x groups before after
1x1 51 ns 47 ns
1x32 98 ns 46 ns
1x1024 4.77 us 46 ns
32x1024 3.53 us 54 ns
1024x1024 2.80 us 57 ns

track_fanout_group/1 (local appends, which now record arrivals) stayed within noise (1.136 us vs 1.145 us).

Tests

  • Regression tests that fail on main: an abort short of a group errors; an abort past the grace ends cleanly (within it errors); an evicted group and a datagram fill a pending tail. Plus a full tail aborted before the reader polls past it still ends cleanly.
  • cargo nextest run -p moq-net --tests --features fuzz with clippy as the wrapper and warnings denied: 2266 passed.
  • cargo nextest run -p moq-relay -p moq-native --tests: 333 passed.
  • cargo check -p moq-net --no-default-features: no warnings. cargo clippy -p moq-net --benches --features fuzz -D warnings: clean. quest check: ok.
  • Not run: just test interop (no wire change), JS (untouched).

Follow-ups

  • quest/m1/js-pending-tail.md is unblocked and now describes the record to mirror, including the short-tail error.
  • quest/m1/resume-reorder.md is unblocked: poll_group's "passed" verdict can read TrackState::arrivals.
  • The model could own the grace timer so a plain finish_at settles a received tail without set_tail_pending(false) (left to quest/m1/public-model/ingest.md).

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 16, 2026 16:55
…t one

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome (background agent, no user prompt available):

  • Implemented the full quest: arrival record (TrackState::arrivals), scan-free end check, and the short-tail abort error. Regression tests fail on main and pass here.
  • Opened as a draft because three choices were made without the maintainer: aged gaps count as lost (not owed), the gap below the first arrival never folds (matches tail::Tail), and the grace comes from the track's aggregate max delay. Recommendation: accept all three as described in the PR body.
  • Follow-ups: js-pending-tail and resume-reorder are now unblocked; a model-owned grace timer is left to public-model/ingest.

(Written by Claude Opus 5.5)

@kixelated
kixelated marked this pull request as ready for review October 17, 2026 00:05
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T00:08:34.655675Z 0ebb959 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2f8bcf3b-8ecf-4cf1-9be5-f7dc0355e07e
📥 Commits

Reviewing files that changed from the base of the PR and between 6f95a54 and 0ebb959.

📒 Files selected for processing: 16
  • quest/m1/README.md
  • quest/m1/js-pending-tail.md
  • quest/m1/public-model/README.md
  • quest/m1/public-model/ingest.md
  • quest/m1/resume-reorder.md
  • quest/m1/subscribe-drop-early.md
  • quest/m1/subscribe-drop.md
  • quest/m1/tail-arrivals.md
  • quest/m2/ordered-owed-wait.md
  • rs/moq-net/Cargo.toml
  • rs/moq-net/benches/tail.rs
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/model/ledger.rs
  • rs/moq-net/src/model/mod.rs
  • rs/moq-net/src/model/track.rs
  • rs/moq-net/src/tail.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging.

  • CI green on 0ebb9596c807d8c397beab6ed0d9cacc0dbe3b09; Codex reviewed that head with no findings (thumbs-up). CodeRabbit was rate limited.
  • The four decisions in the description were settled by the maintainer; nothing left open.
  • feat(net): a timed track refuses a group that rewinds #5803 (refuse-rewinds) also edits rs/moq-net/src/model/track.rs and is still a draft. Whichever lands second rebases; if this lands first, feat(net): a timed track refuses a group that rewinds #5803 should route its lookup removals through remove_slot so the withheld count stays in step.
  • Follow-ups: quest/m1/js-pending-tail.md and quest/m1/resume-reorder.md are unblocked; the model-owned grace timer stays with quest/m1/public-model/ingest.md.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 17, 2026
Merged via the queue into main with commit c98deb7 Oct 17, 2026
15 checks passed
@kixelated
kixelated deleted the quest/m1/tail-arrivals branch October 17, 2026 01:06
kixelated added a commit that referenced this pull request Oct 17, 2026
Keeps main's fetch age budgets (#5680) and tail ledger (#5802) under the
narrowed model: `exemptFetch` is gone, `fetchBudget` stays model-internal,
`groupExpired` and `latency_expired` are marked pending the serve quest,
and the ledger the session tail reaches is marked pending the ingest
quest. `Tail arrivals` landed, so ingest has no blockers left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant