Skip to content

Re-measure the AAC browser budgets after the TS importer groups each PES - #8309

Merged
kixelated merged 10 commits into
mainfrom
quest/m1/audio-jitter-target/capped-budgets
Nov 11, 2026
Merged

kixelated merged 10 commits into
mainfrom
quest/m1/audio-jitter-target/capped-budgets

Conversation

@kixelated

@kixelated kixelated commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m1/audio-jitter-target/capped-budgets.md: the six AAC browser rows (all but congestion) in test/audio-quality/budgets.json are re-measured, by the note's rule, from the seven scheduled nightlies after moq-dev/moq#7898 began publishing each PES as one group. The note drops its sentence that the AAC rows predate the grouping and names the seven runs instead. The quest file is deleted, with its one link from calibrate.md.

Changes

budgets.json, rule: the worst of seven nights times 1.25, plus one runner hiccup, rounded up to two significant figures.

row key was now worst night
bursty underruns/min 260 250 0
bursty gap ms/min 710 700 0
bursty stalled 0.022 0.017 0.005 (11-07)
bursty silence 0.03 0.023 0.006 (11-07)
step underruns/min 260 250 0
step gap ms/min 710 700 0
wide silence 0.034 0.042 0.021 (11-06)
wide target p95 ms 430 400 303.2 (11-10)

Every other key of the six rows lands where it already was (bursty target 450 from 343.2 on 11-11; step stalled 0.022, silence 0.023, target 300; wide underruns 260, gap 710, stalled 0.033 from 0.018 on 11-10; fixed-250, mild, near-zero unchanged). The four keys moq-dev/moq#8459 added as provisional (player_skips_per_min, trimmed_ms_per_min, av_lag_ms p50 and p95) are left as they are: no row has seven nights of them, and calibrate re-measures them. The rule on the three nights that carry the first two gives the same 3.3 and 280.

Public API: none. Wire: none.

Nights graded

night run qualifies
2026-11-05 42050683248 yes
2026-11-06 42236107954 yes
2026-11-07 42377867523 yes
2026-11-08 42487375545 yes
2026-11-09 42659882487 yes
2026-11-10 42842827718 yes
2026-11-11 43041909115 yes

A night qualifies when it is a schedule run of Nightly on main after #7898 merged (2026-11-05 05:55 UTC) and its browser matrix ran in full with no voids. All seven do; every artifact was re-downloaded and re-graded for this round.

The Audio quality job concluded failure on 11-10 and 11-11, on neither the matrix nor an AAC row: its Audio quality step passed both nights ("within budget"), and the later Push-to-talk step failed with native: an utterance did not resume on its head. 11-11's matrix also has the new congestion rows from moq-dev/moq#8459, outside this quest. The failures on 11-05 to 11-09 (Opus wide and bursty stalled, replay rows, native-* rows, the qmux-bench handshake timeout) were all outside the AAC browser rows too.

AAC browser rows, per night

- is a key that night's harness did not report: the player's own skip and trim counters arrived with moq-dev/moq#8308 (first on 11-09), and #8459 dropped the inferred skip-aheads and discarded keys (gone on 11-11).

row night underruns/min gap ms/min player underruns/min skip-aheads/min discarded ms/min player skips/min trimmed ms/min stalled silence target p95 ms
bursty 11-05 0 0 0 0 0 - - 0 0 323.2
bursty 11-06 0 0 0 0 0 - - 0 0 323.2
bursty 11-07 0 0 2.2 0 0 - - 0.005 0.006 323.2
bursty 11-08 0 0 0 0 0 - - 0 0 303.2
bursty 11-09 0 0 0 0 0 0 0 0 0 323.2
bursty 11-10 0 0 0 0 0 0 0 0 0 303.2
bursty 11-11 0 0 0 - - 0 0 0 0 343.2
fixed-250 11-05 0 0 0 0 0 - - 0 0 250
fixed-250 11-06 0 0 0 0 0 - - 0 0 250
fixed-250 11-07 0 0 0 0 0 - - 0 0 250
fixed-250 11-08 0 0 0 0 0 - - 0 0 250
fixed-250 11-09 0 0 0 0 0 0 0 0 0 250
fixed-250 11-10 0 0 0 0 0 0 0 0 0 250
fixed-250 11-11 0 0 0 - - 0 0 0 0 250
mild 11-05 0 0 0 0 0 - - 0 0 209.2
mild 11-06 0 0 0 0 0 - - 0 0 209.2
mild 11-07 0 0 0 0 0 - - 0 0 209.2
mild 11-08 0 0 0 0 0 - - 0 0 209.2
mild 11-09 0 0 0 0 0 0 0 0 0 209.2
mild 11-10 0 0 0 0 0 0 0 0 0 209.2
mild 11-11 0 0 0 - - 0 0 0 0 209.2
near-zero 11-05 0 0 0 0 0 - - 0 0 209.2
near-zero 11-06 0 0 0 0 0 - - 0 0 209.2
near-zero 11-07 0 0 0 0 0 - - 0 0 209.2
near-zero 11-08 0 0 0 0 0 - - 0 0 209.2
near-zero 11-09 0 0 0 0 0 0 0 0 0 209.2
near-zero 11-10 0 0 0 0 0 0 0 0 0 209.2
near-zero 11-11 0 0 0 - - 0 0 0 0 209.2
step 11-05 0 0 2.2 0 0 - - 0.005 0.006 223.2
step 11-06 0 0 2.2 0 0 - - 0.005 0.006 223.2
step 11-07 0 0 2.2 0 0 - - 0.009 0.006 223.2
step 11-08 0 0 2.2 0 0 - - 0 0.003 223.2
step 11-09 0 0 2.2 0 0 0 0 0.009 0.006 223.2
step 11-10 0 0 2.2 0 0 0 0 0.005 0.006 223.2
step 11-11 0 0 2.2 - - 0 0 0.005 0.006 223.2
wide 11-05 0 0 6.5 1.1 47.5 - - 0.014 0.005 283.2
wide 11-06 0 0 15.3 1.1 44.2 - - 0.009 0.021 283.2
wide 11-07 0 0 8.7 2.2 109.7 - - 0.009 0.013 283.2
wide 11-08 0 0 6.5 0 0 - - 0 0.004 283.2
wide 11-09 0 0 7.6 1.1 45.1 0 0 0.005 0.007 283.2
wide 11-10 1.1 0.8 9.8 1.1 44.4 0 0 0.018 0.012 303.2
wide 11-11 2.2 1.4 8.8 - - 0 0 0.005 0.009 283.2

In-order check

The share of each night's chromium-*.arrivals.ndjson frames that arrive behind a newer one: a frame counts when an earlier-arriving frame of the same row has a larger media timestamp.

night AAC rows (range) Opus bursty, for contrast
2026-11-05 0.3 to 2.9% 26.3%
2026-11-06 0.3 to 3.1% 26.5%
2026-11-07 0.3 to 3.1% 23.6%
2026-11-08 0.3 to 3.1% 24.8%
2026-11-09 0.3 to 3.1% 27.0%
2026-11-10 0.3 to 3.1% 25.6%
2026-11-11 0.0 to 3.1% 24.0%

Before the grouping, #7898 measured 82 to 86% of AAC frames behind a newer one (its own, different measure). Nearly every AAC frame now arrives in order, so the quest's claim holds.

Choices made without asking

  • The seven nights span the browser's time stretch. moq-dev/moq#8058 merged 2026-11-09 01:39 UTC, so 11-05 to 11-08 played on the trim player and 11-09 to 11-11 on the stretch. The quest's window is all seven nights after fix(mux): publish each TS audio PES as one group; re-measure browser audio budgets #7898, and calibrate's plan expects this quest to finish on its own window before the stretch model lands, so I kept it. The cost: the worst of the earlier player sets two ceilings (bursty stalled and silence from 11-07, wide silence from 11-06), looser than the stretch nights alone would give, until calibrate re-measures every row. The alternative, grading 11-09 to 11-15, would wait four more nights for ceilings calibrate replaces anyway.
  • Provisional keys untouched, as above.
  • Merged main to edit budgets.json against test(audio-quality): grade steady-state lag, the player's own trims, and recovery from congestion #8459's new key set; the old base still had skip_aheads_per_min and discarded_ms_per_min.
  • Resumed the earlier local attempt (previous round): quest start found a local branch from 2026-10-29; its quest-file conflict was resolved to main's version.
  • In-order definition: timestamp inversion rather than group inversion, which would not compare with the pre-grouping numbers (one group per frame then).

Decisions

  • Grading window: ✅ keep 2026-11-05 to 2026-11-11 as the quest words it, though 11-05 to 11-08 ran the older trim player; calibrate re-measures every row after the stretch model lands (rejected: regrade only 11-09 to 11-15).

Models

Size: XS
Implementer: Claude, claude-haiku-5-5, high effort
Implementer: Claude Code, claude-haiku-5-5, effort not reported (iterate, XS)
Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, M)
Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with low)

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, low effort
  • Session: 01a1cc8b-3f4f-7392-b33b-f1021dee3c74, round 2
  • Head: c1a0db0
  • Base: origin/main at b1ed1ba
  • Score: 9/10, The only change since the previous review clarifies calibration provenance and introduces no actionable defect; independently verifying the nightly measurements remains outside this source-only review.
  • Findings: none

(Written by Claude Opus 5.5 with low)

kixelated and others added 5 commits October 29, 2026 22:54
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The earlier attempt's quest edits describe the pre-#7898 plan; the claim's
quest file, which counts the seven nightlies from the reordered importer, wins.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
kixelated and others added 3 commits November 11, 2026 13:29
…ghts of grouped PES

The six AAC browser rows other than congestion take their ceilings from the
2026-11-05 to 2026-11-11 scheduled nightlies, all after the TS importer began
publishing each PES as one group, by the note's rule. The note drops its
sentence that the AAC rows predate the grouping, and the finished quest goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/github.com/moq-dev/moq into quest/m1/audio-jitter-target/capped-budgets

# Conflicts:
#	quest/m1/audio-jitter-target/capped-budgets.md
@kixelated
kixelated marked this pull request as ready for review November 11, 2026 21:38
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The browser AAC bursty and step underrun ceilings decreased. The bursty stalled-share and silence-share ceilings decreased. The wide silence-share ceiling increased, and its target p95 ceiling decreased. The budget note now identifies the measurement period and runs. The capped-budgets quest document and its link were removed.

Merge Risk: Minimal · up to c1a0d

This change recalibrates audio-quality test ceilings and removes an obsolete calibration doc without leaving dangling links. No actionable merge-blocking risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: re-measuring the AAC browser budgets in test/audio-quality/budgets.json after the TS importer began grouping each PES. The raw summary and objectives confirm that the …
Description check Passed The description is detailed and directly related to the changeset. It explains the re-measurement of six AAC browser rows in budgets.json, the worst-night rule, the table of changed values, the seven …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: re-measuring the AAC browser budgets in test/audio-quality/budgets.json after the TS importer began grouping each PES. The raw summary and objectives confirm that the budget values were updated from seven nightly runs following PR #7898. The title is specific to the primary change and does not describe unrelated work.

Full details: Description check

Explanation

The description is detailed and directly related to the changeset. It explains the re-measurement of six AAC browser rows in budgets.json, the worst-night rule, the table of changed values, the seven nightly runs graded, the deletion of capped-budgets.md with its link from calibrate.md, and the choices made. It matches the raw summary and objectives.

Finishing Touches

Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

Biome (2.5.15)

test/audio-quality/budgets.json

Biome could not lint this file: configuration resulted in errors. Check the repository's Biome configuration and plugins.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c4049464-8514-419c-93dd-5c7650a587e8
Commits

Reviewing files that changed from the base of the PR and between 19fc87d and c1a0db0.

Files selected for processing: 1

  • test/audio-quality/budgets.json
Files skipped from review as they are similar to previous changes: 1

  • test/audio-quality/budgets.json

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ One wording fix in the note; the measurements themselves check out.

Reviewed changes

  • Note gains the AAC measurement window. test/audio-quality/budgets.json now records the seven scheduled runs from 2026-11-05 to 2026-11-11 as the six AAC rows' source, notes that the first four predate the browser's time stretch, and drops the sentence that the AAC rows predate the PES grouping.
  • Three AAC rows re-derived by the standing rule. bursty and step tighten four keys each (underruns_per_min 260→250, underrun_ms_per_min 710→700, plus bursty's stalled_share and silence_share); wide loosens silence_share 0.034→0.042 and tightens target_ms_p95 430→400.
  • Quest closed. quest/m1/audio-jitter-target/capped-budgets.md is deleted and its link leaves calibrate.md's Related section.

I re-downloaded all seven audio-quality-run artifacts and re-derived the table: every per-night value matches the chromium-aac-* summaries, and the rule (worst × 1.25 + hiccup, rounded up to two significant figures) reproduces each new ceiling exactly. bun grade.ts reports "within budget" for the browser rows on 11-10 and 11-11, and the only AAC findings on 11-05 to 11-09 are the four provisional keys reading "unmeasured" because the harness gained them on 11-09. All seven run IDs are schedule Nightly runs on main, and the 11-05 run's head is 27 commits past #7898's merge. Recomputed in-order shares from .arrivals.ndjson land at 0.0 to 3.1% on the six rows (the inversions are the newest-first backfill at subscribe), so the grouping premise holds. bash sh/text.sh passes.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Comment thread test/audio-quality/budgets.json Resolved
…9 and 11-10

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Note clause reworded. test/audio-quality/budgets.json now says the first four graded nights predate the browser's time stretch and that the six AAC rows keep the four provisional ceilings set from 2026-11-09 and 2026-11-10, instead of attaching the provisional keys to the four earliest nights, which never reported them.

This is the whole delta since 19fc87d (budgets.json, one line). The new clause matches the rows themselves: every AAC row still carries player_skips_per_min 3.3, trimmed_ms_per_min 280 and both av_lag_ms ceilings derived from the two stretch nights, and it no longer conflicts with the later sentence naming 11-09 and 11-10 as those keys' source. bash sh/text.sh passes and budgets.json still parses with its tab indentation. The round-1 inline finding is addressed.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit 284c977 Nov 11, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m1/audio-jitter-target/capped-budgets branch November 11, 2026 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant