test(ts): audit random-loss decisions with bufferbloat

.github/workflows/ts.yml

@@ -218,11 +218,11 @@ jobs:
           MOQ_TEST_RUNS: ${{ runner.temp }}/moq-test-runs
 
       # The subscriber's leg congested the way real paths are, through moq-shaper: 7 Mb/s under
-      # the 9 Mb/s video, behind a 2 s queue, 25 ms each way. The export must stay T-STD compliant
-      # and continuous, shed only whole groups, and keep its latency, so the last group arrives.
+      # the 9 Mb/s video and four audio PIDs, behind a 2 s queue, 25 ms each way. The export must
+      # stay T-STD compliant and continuous, shed whole groups, and keep its latency through EOF.
       - name: TS congested path
         if: ${{ !cancelled() && github.event_name != 'pull_request' }}
-        run: nix develop --command just test ts --hrd --rate 7000000 --duration 120
+        run: nix develop --command just test ts --headroom --rate 7000000 --duration 120
         env:
           MOQ_TEST_RUNS: ${{ runner.temp }}/moq-test-runs
 
@@ -231,18 +231,19 @@ jobs:
       # inside the export's 1 s delay, and the run fails unless the queue overflowed.
       - name: TS overflowing path
         if: ${{ !cancelled() && github.event_name != 'pull_request' }}
-        run: nix develop --command just test ts --hrd --rate 7000000 --queue 250ms --duration 120
+        run: nix develop --command just test ts --headroom --rate 7000000 --queue 250ms --duration 120
         env:
           MOQ_TEST_RUNS: ${{ runner.temp }}/moq-test-runs
           MOQ_QUIC_CONGESTION_CONTROL: loss
 
-      # The passthrough over the congested path, under its 10 Mb/s multiplex: what arrives must be the
-      # source's packets in place.
-      - name: TS passthrough congested path
+      # Passthrough over the overflowing path, under its 10 Mb/s multiplex: what arrives must be
+      # the source's packets in place, through the source's last packet.
+      - name: TS passthrough overflowing path
         if: ${{ !cancelled() && github.event_name != 'pull_request' }}
-        run: nix develop --command just test ts --passthrough --rate 7000000 --duration 120
+        run: nix develop --command just test ts --passthrough --rate 7000000 --queue 250ms --duration 120
         env:
           MOQ_TEST_RUNS: ${{ runner.temp }}/moq-test-runs
+          MOQ_QUIC_CONGESTION_CONTROL: loss
 
       - name: Upload failure artifacts
         if: ${{ failure() }}

quest/m1/README.md

@@ -106,7 +106,6 @@ lands before the cut.
 - [Transcode finishes its group](/quest/m1/transcode-open-group.md) - `moq-transcode` never aborts a group a fetch's live loop opened
 - [Rust request credit](/quest/m1/rs-request-credit.md) - a moq-net request fails at once without stream credit instead of waiting
 - [qmux no-wait opens](/quest/m1/qmux-no-wait.md) - @moq/qmux rejects an over-limit create when waitUntilAvailable is false, like Chrome
-- [Random-loss audit](/quest/m1/random-loss-audit.md) - every decision random-loss gates drove is kept, reworked, or reverted on shaped paths
 - [Timed-out requests clean up](/quest/m1/legacy-request-timeout-cleanup.md) - on drafts 14-16, Rust and JS send PUBLISH_NAMESPACE_DONE or UNSUBSCRIBE_NAMESPACE for a timed-out request before its local reset
 - [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide
 - [Resume reorder](/quest/m1/resume-reorder.md) - a group whose stream header arrives after its successor's still resumes on failover; only a dropped or long-gone group fails

quest/m1/ts-reliability/recovery.md

@@ -21,7 +21,11 @@ nothing, in its stream row's `units` and `quiet`.
 
 Test content loss separately from IP packet loss. QUIC may retransmit IP
 loss; a netem percentage does not establish that any MoQ object was omitted.
-Keep the real network impairment arm for transport and pacing evidence, and
+Keep the real network impairment arm for transport and pacing evidence,
+using a rate below the media bitrate behind a deep but finite queue with
+verified overflow drops. Random loss only proves retransmission correctness,
+never output quality or latency. Reuse the overflowing lane in
+`test/ts/README.md`, and
 add deterministic fault schedules at the MoQ content boundary.
 
 Cover a cold join, a skipped group, a partially delivered group, late objects

quest/m2/probe/native.md

@@ -40,8 +40,12 @@ Acceptance, in moq-quic's seeded simulation (`rs/moq-quic/src/tests/`,
 extending `Bottleneck` to step its rate) and wired into CI at least nightly:
 an illustrative 2 Mbit/s source seeking 4 Mbit/s while the path steps 3 to 12
 to 3 Mbit/s, on identical media traces and wire budgets. Cover keyframes, idle
-restart, stale high estimates, receiver credit limits, random and burst loss,
-shallow queues, compressed ACKs, and competing CUBIC and BBR flows. Set the
+restart, stale high estimates, receiver credit limits, compressed ACKs, and
+competing CUBIC and BBR flows. Add a congested arm below the media bitrate
+behind a deep but finite queue for impairment acceptance and performance.
+Verify actual overflow drops, adding competing loss-based traffic if BBR
+alone will not fill it. Sweep queue depth separately. Random or scripted burst loss may
+prove recovery correctness, but never gates performance. Set the
 deadline, overhead, and fairness budgets before tuning. Report validated
 upward adaptation, downward reaction, estimate age, frame deadlines, queue
 delay, unique goodput, and redundant bytes, against a no-probing baseline. A

quest/m2/quic-bbr-natural-drain.md

@@ -8,8 +8,10 @@ ProbeRTT, recognizing bandwidth growth within a round, and precautionary
 bandwidth probing. Reduce demonstrated frame deadline interference without
 stale minimum RTT, persistent queues, or unfairness. Application limitation
 alone is not evidence that the path drained, and Google parity is not
-assumed to be an improvement. Grade them, and BBR's response to random
-loss, on a 10 % random-loss media profile as well.
+assumed to be an improvement. Acceptance and performance use a rate below
+the media bitrate behind a deep but finite queue with verified overflow
+drops. Random loss may prove recovery correctness, but never grades media
+quality or performance.
 
 ## Plan
 
@@ -42,8 +44,10 @@ path change. Blanket skipping is only a diagnostic control. Retain the
 simpler baseline if no material interference is demonstrated.
 
 Use identical media traces and seeds. Start with short and long RTT paths,
-then test capacity changes, base-RTT changes, competing bulk traffic, shallow
-queues, loss bursts, and compressed or delayed ACKs. Record deadlines and
+then test capacity changes, base-RTT changes, competing bulk traffic, queue
+depths, and compressed or delayed ACKs. Use loss-based cross traffic when
+BBR alone does not fill the deep queue; verify actual overflow drops. Keep
+scripted loss bursts as recovery correctness checks only. Record deadlines and
 frame completion, withheld useful bytes, sender backlog separately from
 network queueing, RTT-baseline error, sample labels, and mode residence.
 Set latency and fairness acceptance limits before tuning. Persist the
@@ -78,38 +82,30 @@ BBR3 state machine.
   and [Google QUICHE](https://github.com/google/quiche/blob/535a2730e77d47e0dc03746555cc9c34b17bc9e9/quiche/quic/core/congestion_control/bbr3_sender.cc#L1079)
   stop precautionarily when probing reaches a previously lossy inflight
   bound, then accelerate a later probe if feedback is clean. Noq lacks this
-  state and transition. Compare shallow queues, capacity increases, random
-  loss, and competing flows.
+  state and transition. Compare queue depths, capacity increases, and
+  competing flows on shaped paths with verified overflow drops.
 
 Report throughput, queue delay, loss, convergence time, and fairness with
 pinned code and configurations. Keep transport differences and QUICHE flags
 explicit and compare each algorithm change separately. A simulation result
 is not an end-to-end network measurement.
 
-### Random loss
-
-Moved from the finished T-STD export epic (2026-10-16), whose nightly
-stays at 1 %.
-`just test ts --hrd --loss <percent>` drops that share of the relay's
-datagrams to the subscriber at random through `moq-shaper`, 25 ms each way,
-and grades the export with T-STD, PCR accuracy and `holes.py`. Measured on a
-loaded host (rerun before deciding):
-
-| Clip | Loss | Result |
-|---|---:|---|
-| default (low media rate) | 10 % | passes |
-| `--hrd` (9 Mb/s video, 10 Mb/s mux) | 1 %, 3 % | passes, no holes |
-| `--hrd` | 5 %, 7 %, 10 % | groups evicted `Old`, export fails |
-| `--hrd`, `LOSS_THRESH` 20 % (experiment) | 10 % | T-STD and PCR pass; 1.56 s of video misses the 1 s delay |
-| `--hrd`, CUBIC | 10 % | no output |
-
-BBRv3 takes a round with more than 2 % loss (`LOSS_THRESH`) as congestion
-and cuts its inflight bound, so random loss above it holds the leg below the
-media rate. Decide whether and how BBR should tell random loss from
-congestion without giving up fairness against loss-based flows, and how that
-interacts with the precautionary probe stop under Google differences.
+### Congestion evidence
+
+Use the overflowing and deep-queue lanes in `test/ts/README.md` as the
+end-to-end baseline. A BBR-only run that sheds groups at the relay but never
+overflows the network queue cannot judge the loss response. A CUBIC run
+that overflows establishes that the harness exercises recovery; it cannot
+establish whether changing BBR improves it.
+
+Keep the current 2% `LOSS_THRESH` and baseline controller policy unless a
+candidate improves the shaped-path comparison, including queues and
+fairness. Surviving an independent random-loss percentage is not a reason
+to weaken the congestion response or precautionary probe stop. This study
+still decides natural-drain credit, growth recognition, and probe behavior
+separately; it does not adopt any of them in advance.
 
 ## Related
 
-- [TS validation](/test/ts/README.md) - the `--loss` rig, `holes.py` and the full measurements
+- [TS validation](/test/ts/README.md) - shaped paths, whole-group shedding, and measured baselines
 - [Native probing](/quest/m2/probe/native.md) - preserving an estimate and discovering spare capacity are separate problems

quest/m2/quic-deadline.md

@@ -44,7 +44,11 @@ Implement in `moq-quic`.
   existing group-expired code on the MoQ wire, so a viewer sees the same
   signal it sees for a relay-side expiry today.
 
-Measure on the impaired path profile: delivery latency p95 and p99, bytes
+Measure on bufferbloat: a rate below the media bitrate behind a deep but
+finite queue whose overflow demonstrably drops packets, with competing
+loss-based traffic if needed to fill it. Random loss may prove recovery
+correctness, but never grades latency or throughput. Report delivery latency
+p95 and p99, bytes
 retransmitted past their deadline (must be zero), spurious resets under
 reordering, and probe overhead versus the default PTO. A proactive probe that
 raises loss or latency under any profile stays off by default.

quest/m2/quic-outage-probe.md

@@ -17,7 +17,7 @@ Decided 2026-10-19 by the maintainer (#5953):
 - Declined from #5953: a congestion controller that holds a fixed rate
   through random loss, exposing the reorder thresholds, and a named
   provisioned-path profile. Random loss is not a quality target
-  ([random-loss audit](/quest/m1/random-loss-audit.md)), and the reorder
+  (see [shaped-path grading](/test/ts/README.md)), and the reorder
   evidence came from random netem reorder only. Revisit the thresholds with
   a bufferbloat or real-path measurement.
 - Test on mocked time: a link that drops everything for 5 s, then returns.

quest/m2/quic-sim.md

@@ -16,6 +16,11 @@ driver. Start from moq-shaper's in-memory `UdpSocket` (`rs/moq-shaper/src/mem.rs
 BBR's controller-level `Sim` in `bbr3/mod.rs` stays separate unless this
 replaces it cleanly.
 
+Keep random loss for recovery correctness. Any acceptance or performance
+consumer uses a rate below its media bitrate behind a deep but finite queue,
+and checks that the queue overflowed. A stalled sender that never filled the
+queue is not evidence about congestion loss.
+
 ## Related
 
 - [Per-stream deadlines](/quest/m2/quic-deadline.md) - a consumer

quest/m3/cut-through/bench.md

@@ -13,11 +13,17 @@ Decided 2026-09-30: real stacks, not a mock. The loss recovery being measured
 belongs to the QUIC stack, so a mock transport with a made-up retransmit model would
 partly measure its own assumptions.
 
-- Publisher, then a seeded `moq-shaper` (loss and delay), then `moq-relay` over
-  QUIC, then a clean or rate-limited hop to the viewer. Reuse the shaper wiring
-  from `rs/moq-relay/tests/drills.rs`.
-- Sweep loss rate, frame size (audio-sized through a large I-frame), and egress
-  headroom (egress rate relative to the media rate).
+- Publisher, then `moq-shaper` with an ingress rate below the media bitrate
+  and a deep but finite queue whose overflow drops packets, then `moq-relay`
+  over QUIC, then a clean or rate-limited hop to the viewer. Reuse the shaper
+  wiring from `rs/moq-relay/tests/drills.rs` and the overflowing bufferbloat
+  profiles described in `test/ts/README.md`. Verify that the measured path
+  actually overflows its queue.
+- Sweep ingress rate, queue depth, frame size (audio-sized through a large
+  I-frame), and egress headroom (egress rate relative to the media rate).
+  The go or no-go verdict uses these bufferbloat runs. Seeded random loss
+  may check correctness, such as the payload-loss and header-loss cases
+  below, but never gates acceptance or performance.
 - Report per-frame completion at the viewer, from publish to last byte, p50 and
   p99. The opportunity is the drain time on this same two-hop path: per frame,
   the time from the relay's ingress hole filling to the frame completing at the

quest/m3/quic-careful-resume.md

@@ -28,8 +28,12 @@ start on the first loss or a mismatched RTT.
   `failover.rs`) seeds a redial from the session it replaces, and
   the relay's cluster peers seed from the previous session to the same peer.
 
-Measure time to the encoder's target rate after a reconnect on the impaired
-path profile, plus loss and latency during the jump. Ship it on by default
+Measure time to the encoder's target rate after a reconnect on bufferbloat:
+a rate below the media bitrate behind a deep but finite queue, with actual
+overflow drops, plus loss and latency during the jump. Include competing
+loss-based traffic when BBR alone does not fill the queue. Random loss may
+prove fallback correctness, but never grades the jump's performance.
+Ship it on by default
 only when the jump never makes the first second worse than slow start.
 
 ## Related

rs/moq-mux/src/container/ts/export.rs

@@ -1148,10 +1148,11 @@ where
 	/// ride is laid, and each slot goes out the delay after its own time on the source's
 	/// clock rather than a delay later still.
 	///
-	/// The clock is acquired before anything goes out: frames are held until a track starts
-	/// a new group, or for the delay at most, and the freshest of them against its release
-	/// time sets the clock, so a joiner runs at the delay from its first output and every
-	/// frame has the whole delay to arrive in. A source that sends its video its send-ahead
+	/// The clock is acquired before anything goes out: frames are held until every expected
+	/// track has delivered (two delays at most) and a track starts a new group, or for a delay
+	/// at most once all have. The track sent latest against its release time sets the clock,
+	/// so a joiner runs at the delay from its first output and every frame has the whole delay
+	/// to arrive in. A source that sends its video its send-ahead
 	/// early, as a TS mux does, is trailed by the delay; one that sends every track at its
 	/// decode time, by the delay and the video's send-ahead. The output then keeps the
 	/// source's pace and muxes every track in `(release time, PID)` order whatever the