Skip to content

fix(net): a fired deadline costs its driver one poll, not two - #5393

Merged
kixelated merged 2 commits into
mainfrom
fix/driver-double-poll
Oct 13, 2026
Merged

kixelated merged 2 commits into
mainfrom
fix/driver-double-poll

Conversation

@kixelated

@kixelated kixelated commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Every time::Deadline that fired cost its session driver two polls instead of one.

Driver::poll calls Clock::advance(now), which wakes the due deadlines. When a deadline belongs to a kio::Tasks child (the lite publisher's control streams, for example), the child's wake sets its bit and wakes the set's owner. That owner is the driver task, which is already being polled. So the executor polls the driver again once it returns, and that poll finds nothing, because the Tasks::poll later in the first pass already ran the child.

A second wake hid behind the first. The child usually re-arms its deadline, and Deadline::set wakes the driver's clock registration when the new instant is the earliest. That registration was still live from the previous poll. Removing either wake alone changes nothing: the other still costs the second poll.

Fix

  • kio::Waiter::batch(|| ..) runs one poll of the waiter's task. Inside it, a Tasks set whose owner is that task holds its wake of the owner. When the batch returns, the wake goes out only if the set is still pending, meaning nothing polled it after the wake. The child's own ready bit is set immediately, as before. Wakes from other threads, and wakes of any other task, are never held. No latency is added: the executor cannot re-poll a task before its current poll returns. The scope is thread-local (loom::thread_local! under loom) and is read with try_with, so a wake from another thread-local's destructor at thread exit is delivered rather than aborting the process.
  • Clock::advance drops the driver's registration. It uses take() so the list keeps its serial; a fresh WaiterList would invalidate the driver waiter's records and make it re-register on every list. Clock::park re-registers it and reads the earliest deadline under one lock at the end of the poll. A deadline armed during the poll is reported by park; one armed elsewhere after park wakes the driver.
  • Both time::Driver impls (the session Driver and the origin Driver) wrap their poll in waiter.batch.

Why opt-in, not the default for every kio::wait poll

Measured with the same workloads as below:

batch placement idle instructions active instructions lite polls moq-transport polls
none (main) 119.9M 2489.3M 33659 33609
driver polls (this PR) 106.8M 2487.5M 29403 33609
every kio::wait poll 108.0M 2539.0M (+2.0%) 29403 29757
driver + every Tasks child poll 107.5M 2533.5M 32827 32605

Most kio::wait polls own no Tasks set, so a default scope is pure overhead there. A batch nested inside another also shadows the outer one, because only the innermost batch holds wakes. A child's wake of a sibling in the driver's set then reaches the executor, which is why scoping children added lite polls back. The batch pays at a task's outermost poll that wakes its own children before polling them, which is what a driver advancing its clock does. The batch doc says to use it once, there.

Measurements

Deterministic: simulated time (moq_net_sim), lite-06, --profile bench, pinned to one P-core, perf stat -e instructions:u. Every variant ran at least twice with identical counts. Instructions cover the whole process, including setup, the client sessions, the mock transport and the executor.

workload server driver polls instructions
idle: 64 viewers with PROBE open, 600 s 7680 → 3840 (−50%) 119.9M → 106.8M (−10.9%)
active: 16 viewers, 1800 frames at 30 fps 33659 → 29403 (−12.6%) 2489.3M → 2487.5M (−0.1%)
active: 256 viewers, 300 frames at 30 fps 90086 → 78823 (−12.5%) 3529.7M → 3482.3M (−1.3%)

The session_delivery_paced room from benches/session.rs (lite-06, 16 publishers), run for a fixed 256 rounds:

viewers instructions
1 27.37M → 26.47M (−3.3%)
16 352.1M → 337.4M (−4.2%)
256 3058.7M → 2936.2M (−4.0%)

Criterion (cargo bench -p moq-net --bench session -- 'session_idle|session_delivery_paced/moq-lite-06'), run as before / after / before / after on a heavily loaded 24-core host (load average 25 to 40). Medians:

bench before after
session_idle/sessions=1 700 ns, 686 ns 641 ns, 648 ns
session_idle/sessions=64 22.2 µs, 27.0 µs 19.4 µs, 19.2 µs
session_idle/sessions=1024 1.28 ms, 1.11 ms 1.07 ms, 1.38 ms
paced viewers=1 11.89 µs, 11.74 µs 11.07 µs, 11.11 µs
paced viewers=16 206 µs, 196 µs 207 µs, 190 µs
paced viewers=256 2.88 ms, 2.41 ms 5.12 ms, 4.18 ms

The paced viewers=256 criterion row disagrees with the fixed-round count for the same room (−4.0% instructions, 524 ms → 513 ms wall). Treat the criterion numbers as noise on this host; the instruction counts are the measurement.

The batch's own cost, from the new kio bench (cargo bench -p kio --bench tasks -- wake_16_of_1000): kio_tasks 752.7 ns, kio_tasks_batch 771.6 ns per pass, so about 19 ns per driver poll against a whole saved poll.

The moq-transport sessions' extra polls come from a different path than Clock::advance. Their driver is a boxed future, so its children's own sets are woken inside nested polls. That is left for another PR.

Tests and benchmarks

  • kio task::tests::a_wake_from_the_owners_own_poll_is_not_redelivered: the owner's batch wakes a task in its own set, then polls the set. The owner must not be woken. Fails without the hold.
  • kio task::tests::a_held_wake_lands_when_the_set_is_not_polled: the same wake, with the batch returning before polling the set, must still reach the owner.
  • kio task::tests::a_wake_during_thread_teardown_reaches_the_owner: a task woken from a thread-local's destructor after the batch scope's thread-local is destroyed. Aborts the process (SIGABRT, TLS access after destruction) with with in place of try_with.
  • kio loom a_held_wake_never_hides_a_remote_wake: a held wake races a wake from another thread, with the batch either consuming its wake in the same pass or returning first. Making release a no-op deadlocks the model.
  • moq-net time::tests::a_deadline_costs_the_driver_one_poll: a Tasks child re-arming a periodic deadline under the driver's advance, poll and park sequence. Three ticks wake the driver zero times, and an earlier deadline armed outside the poll still wakes it once. Fails with either half of the fix removed.
  • moq-net tests/idle_session.rs: with fix(net): stop idle viewer sessions from waking every 100ms #5382 merged, the bound tightens from 2 * refreshes to refreshes driver polls per 60 s idle. Main: 12 polls, fails. This branch: 6. With batch made a passthrough (the Clock half alone): still 12.
  • Bench kio wake_16_of_1000/kio_tasks_batch: the existing sparse-wake bench with the gates fired inside the owner's batch, so the hold and release cost is tracked. The session-level win stays tracked by moq-net's session_idle bench and the deterministic idle_session bound.

just rs loom for kio passes, including the new model.

Decisions

Settled during /quest-iterate by an agent that could not prompt; each took the recommended option. The maintainer confirmed decision 1.

  1. Does the fix need a public kio API? (Maintainer confirmed: public, named batch.)
    • ✅ Keep Waiter::batch public, under the name batch (recommended). The kio half is required (batch as a passthrough leaves idle_session at 12 polls), only the owner knows it is mid-poll, and the owner (moq-net's drivers) is in another crate. Driver::poll is itself public and called with external waiters, so the scope belongs inside it.
    • Open the scope automatically in every kio::wait poll: no new API, but +2.0% active instructions (table above), and callers driving Driver::poll from their own executor would not get it.
    • Drop the kio half and keep only the Clock change: no measurable gain.
  2. How is the improvement captured in a benchmark?
    • ✅ Add wake_16_of_1000/kio_tasks_batch for the batch's own cost, and rely on session_idle plus the tightened idle_session poll bound for the session win (recommended).
    • Add a poll-counting session bench: duplicates the idle_session bound, which is deterministic where criterion is not.
  3. The previous CI failure (finite_relay::relay_waits_for_reordered_upstream_headers) was a pre-existing IETF relay bug this branch exposed by changing task order.

API and wire impact

  • New public kio::Waiter::batch<R>(&self, poll: impl FnOnce() -> R) -> R.
  • Tasks now uses crate::sync atomics, so loom permutes them. No change outside loom builds.
  • No wire change. No change to moq-net's public API.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

`Driver::poll` advances its clock, which wakes the due deadlines. A deadline
owned by a `kio::Tasks` child wakes the child's set, and the set wakes its
owner: the driver task, already mid-poll. The executor then polled the driver
again for a wake the same pass had taken. The deadline's re-arm did the same
through the driver's own clock registration, still live from the last poll.

`Waiter::batch` runs one driver poll; a `Tasks` set woken from inside it holds
its wake of the owner until the batch returns, and drops it if the set was
polled since. Wakes from other threads are never held. The clock drops the
driver's registration on `advance` and re-registers it with the timeout read,
under one lock, when the poll ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated force-pushed the fix/driver-double-poll branch from 55e5f60 to 5927253 Compare October 13, 2026 16:38
A `Tasks` wake from another thread-local's destructor at thread exit can run
after the batch scope's own thread-local is destroyed; `with` then panicked
inside a destructor and aborted. `try_with` delivers the wake unheld.

Tighten `idle_session` to one driver poll per PROBE refresh now that #5382 is
in, and bench the batch's own cost next to the plain sparse-wake pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 13, 2026 16:39
@coderabbitai

coderabbitai Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9e44e026-218c-485f-b457-b3800e7910f3
📥 Commits

Reviewing files that changed from the base of the PR and between 1998d34 and eec3bff.

📒 Files selected for processing (9)
  • rs/kio/benches/tasks.rs
  • rs/kio/src/loom.rs
  • rs/kio/src/sync.rs
  • rs/kio/src/task.rs
  • rs/kio/src/waiter.rs
  • rs/moq-net/src/driver.rs
  • rs/moq-net/src/model/origin.rs
  • rs/moq-net/src/time.rs
  • rs/moq-net/tests/idle_session.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T16:41:52.886790Z eec3bff Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Queuing for merge at eec3bff.

  • CI failure was the pre-existing IETF relay bug in relay_waits_for_reordered_upstream_headers, fixed by fix(net): an IETF relay waits for groups below the first to arrive #5390; rebased onto main.
  • Added: try_with for the batch scope so a wake from a thread-local destructor at thread exit is delivered instead of aborting (regression test included), the idle_session bound tightened to one driver poll per PROBE refresh (12 on main, 6 here), and a kio_tasks_batch bench for the batch's own cost (~19 ns per pass).
  • Decision: the maintainer confirmed kio::Waiter::batch stays public under that name.
  • Review: Codex reviewed eec3bff with no findings.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 13, 2026
kixelated added a commit that referenced this pull request Oct 13, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged via the queue into main with commit b5d1de6 Oct 13, 2026
7 checks passed
@kixelated
kixelated deleted the fix/driver-double-poll branch October 13, 2026 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant