Skip to content

perf(net): checkpoint inline subscriber window tradeoff - #6241

Closed
kixelated wants to merge 2 commits into
mainfrom
quest/m1/window-alloc
Closed

kixelated wants to merge 2 commits into
mainfrom
quest/m1/window-alloc

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

A single-range subscriber resolves its private Window through three allocations. This draft preserves the measured allocation-free implementation for a scope decision; it is not a performance improvement ready to land. The existing quest/m1/window-alloc.md remains open.

Approach

Keep one requested range and resolved span inline with SmallVec, and one group run inline before spilling to the existing collections. SmallVec 1.16.3 is already in the lockfile. Cover an open run at u64::MAX through both one range and adjacent spilled ranges.

Fresh comparison against main at 664e09d160, using flake-pinned Valgrind 3.27.1 and the same release-mode subscribe/poll/drop probe:

Per join Before Inline
Instructions, 1/100/1,000/10,000 steady viewers 4,490 / 4,490 / 4,490 / 4,489 4,417 / 4,417 / 4,417 / 4,417
Heap allocations 12 9
Allocated bytes 1,800 1,136
Alternating pinned-core median 402.7 ns 425.9 ns

The default Window itself loses all three allocations, but join timing regresses 5.8% across six alternating before/after pairs on CPU 0 (five million joins per run). Callgrind shows increased copying offsetting most constructor savings. Existing Nightly join/churn benchmarks remain wired and ran across all audience/churn axes; the shared-load Criterion comparison also regressed (join about 8%, churn mostly about 4–6%). Timing is supporting evidence on this shared host, not a claim of isolated benchmark precision.

Impact

  • Consumer: current moq-net track subscribers, including relay fan-out, resolving their requested ranges at subscribe time.
  • Public API: unchanged. All representation changes are private; no migration required.
  • Wire: unchanged. No negotiation, filtering or routing behavior changes intended.
  • Cost: three fewer allocations and 664 fewer allocated bytes per default join; measured timing regression prevents recommending this shape.

Alternatives

Constructor inlining did not change instructions. Boxing only time-end state reduced the compact experiment to about 4,408 instructions at one viewer, too small a gain to justify weakening the single-range allocation guarantee, so it was discarded.

Recommended: pause/abandon this allocation-only shape. Alternatively, explicitly authorize a compact specialization for common default/group ranges while uncommon time windows may allocate, then measure it. That would narrow the quest's current guarantee and needs a maintainer decision before implementation.

Validation

  • just rs test -p moq-net: 2,869 passed, 8 skipped.
  • quest check: passed.
  • Existing track_subscriber_join and track_subscriber_churn Criterion sweep: completed.
  • Callgrind two-iteration-count differences and Memcheck allocation differences: completed.
  • just ci: Rust checks passed; flake runner-image build stopped at unshare: write failed /proc/self/uid_map: Operation not permitted (host namespace restriction).

Follow-ups

The performance/scope choice above is unresolved. No additional quest was created, and no specialization was implemented. Keep this draft; do not merge it as an optimization. A measurement procedure is recorded in the outcome comment.

(Written by GPT-6)

kixelated and others added 2 commits October 21, 2026 20:36
Co-Authored-By: GPT-6 <noreply@openai.com>
Keep one range, span and group run inline and spill multiple ranges to the
existing collections. Preserve the fully allocation-free single-range scope
and cover adjacent open ranges at the maximum sequence.

This is a review checkpoint, not a recommended optimization to land. Fresh
baseline 664e09d used Valgrind 3.27.1 from this checkout's nixpkgs input.
A release probe creates a broadcast/track with N steady subscribers, consumes
the initial poll_subscription_changed, then repeatedly subscribes, polls,
drops and polls again, asserting both per-join polls remain pending. For
N=1/100/1000/10000, subtract Callgrind totals at 1000 from 2000 iterations:
before 4490/4490/4490/4489 instructions per join; inline 4417 at every N.
The same Memcheck heap-usage difference at N=1 removes three allocations
and 664 allocated bytes per join (12 to 9 allocations, 1800 to 1136 bytes).

Six alternating AB/BA pairs of frozen release probe binaries, pinned to
CPU 0 with 5,000,000 joins per run, measure median 402.7 ns before versus
425.9 ns inline (+5.8%). Existing Criterion join/churn audience/churn sweeps
also ran, with shared-load timing regressions. Inlining new had no effect;
boxing time-end state barely changed instructions and was discarded.

2869 moq-net tests pass, 8 skipped; all 455 quest documents validate.
The required just ci gate is still running at this checkpoint.

Unresolved: recommend pausing/abandoning this allocation-only shape versus
explicit authorization to specialize common default/group ranges and permit
uncommon time windows to allocate. Do not silently narrow the quest guarantee.
The quest remains open. No public API or wire change, no follow-up quest.

Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Outcome: this draft is a measured source checkpoint, with the quest still open. Recommend pausing/abandoning this shape because fewer allocations did not improve the present subscriber's join cost. A common default/group specialization that lets uncommon time windows allocate requires an explicit scope choice; it was not implemented. No new quests were created.

Public API/wire impact: none. The current consumer is the moq-net track subscriber used by applications and relay fan-out.

Reproduction uses the pinned Rust dev shell and Valgrind 3.27.1 from the checkout's nixpkgs input. Compare base 664e09d160 and this checkpoint in separate isolated worktrees. Put this disposable package in each worktree's .scratch/join-probe/ (not in the repository's source tree).

Cargo.toml:

[workspace]
[package]
name = "window-join-probe"
version = "0.0.0"
edition = "2024"
[dependencies]
moq-net = { path = "../../rs/moq-net" }
kio = { path = "../../rs/kio" }
[profile.release]
lto = "fat"
codegen-units = 1
debug = 1

src/main.rs:

use std::hint::black_box;
use moq_net::broadcast;

fn main() {
    let mut args = std::env::args().skip(1);
    let iterations: usize = args.next().unwrap().parse().unwrap();
    let steady: usize = args.next().map(|n| n.parse().unwrap()).unwrap_or(1);
    let broadcast = broadcast::Info::default().produce();
    let mut track = broadcast.create_track("bench", None).unwrap();
    let _steady: Vec<_> = (0..steady).map(|_| track.subscribe(None)).collect();
    let waiter = kio::Waiter::noop();
    assert!(track.poll_subscription_changed(&waiter).is_ready());
    for _ in 0..iterations {
        let viewer = black_box(track.subscribe(None));
        assert!(black_box(track.poll_subscription_changed(&waiter)).is_pending());
        drop(viewer);
        assert!(black_box(track.poll_subscription_changed(&waiter)).is_pending());
    }
}

Build with:

CARGO_BUILD_JOBS=2 nix develop --command cargo build --release --manifest-path .scratch/join-probe/Cargo.toml
nix build --impure --no-link --print-out-paths --expr '(builtins.getFlake "/absolute/path/to/checkout").inputs.nixpkgs.legacyPackages.x86_64-linux.valgrind'

For each steady viewer count 1/100/1,000/10,000, run the built executable under valgrind --tool=callgrind --callgrind-out-file=... at 1,000 and 2,000 loop iterations. The difference between each file's summary: totals divided by 1,000 is instructions/join, canceling startup and audience setup. Results before: 4,490 / 4,490 / 4,490 / 4,489; after: 4,417 at every count. The result stays flat over audience size. Memcheck's total heap usage differences for the same 1,000/2,000 iteration comparison at one viewer are 12→9 allocations and 1,800→1,136 allocated bytes per join.

Freeze each executable before comparing timings. Six AB/BA pairs ran taskset -c 0 <binary> 5000000 1 and divided Python time.perf_counter() elapsed seconds by five million. Per-join nanoseconds:

  • Before: 410.43, 401.05, 410.00, 403.50, 401.96, 401.50. Median 402.73.
  • Inline: 433.55, 422.67, 420.21, 427.64, 424.26, 435.51. Median 425.95 (+5.8%).

The existing Nightly benchmark sweep also completed before/after:

CARGO_BUILD_JOBS=4 nix develop --command cargo bench -p moq-net --bench track -- 'track_subscriber_(join|churn)' --sample-size 10 --measurement-time 1 --warm-up-time 1 --noplot --save-baseline window-before
CARGO_BUILD_JOBS=4 nix develop --command cargo bench -p moq-net --bench track -- 'track_subscriber_(join|churn)' --sample-size 10 --measurement-time 1 --warm-up-time 1 --noplot --baseline window-before

This covers steady viewers 1/100/1,000/10,000 × churn 1/8/64, joins across the same audience sizes, and churn after peak 0/1,024/16,384. Shared-host Criterion timing regressed about 8% for join and mostly 4–6% for churn. These timings support stopping, but do not claim a quiet-host performance estimate.

Constructor #[inline] produced the identical 4,417 instructions. An unpublished Box experiment produced about 4,408 at one viewer while allowing time-range allocation. Its tiny improvement did not justify narrowing the guarantee; it was discarded. The checkpoint retains every single range/span/run inline.

Validation: 2,869 moq-net tests passed, 8 skipped, including the new maximum-sequence union edge case. All 455 quest documents validate. just ci passed its Rust checks, then stopped building the flake runner image at unshare: write failed /proc/self/uid_map: Operation not permitted. This host namespace restriction remains a validation limit.

No background process remains at terminal handoff. The exact implementation and evidence are preserved; no merge or review was requested.

(Written by GPT-6)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Closing per the quest audit: the inline window saved 3 allocations per join but regressed join time about 6-8%, so the allocation-only shape doesn't pay for itself. The quest is deleted in the audit PR.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest removal: moq-dev/moq#6273 deletes quest/m1/window-alloc.md and its links. It also records this result in quest/m1/audit-perf.md so nobody retries the shape blindly.

Measured here against 664e09d160: 12 to 9 allocations and 1,800 to 1,136 bytes per join, and 4,490 to 4,417 instructions per join, flat across audience size. Pinned-core join time went from 402.7 to 425.9 ns median (+5.8%), and the Criterion join/churn sweep regressed about 4-8%. The branch stays as the reference implementation.

(Written by Claude Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant