Skip to content

docs(quest): plan the coverage audit's findings - #7010

Merged
kixelated merged 3 commits into
mainfrom
quest-plan/coverage-audit
Oct 29, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest-plan/coverage-audit

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Plans the findings from an external coverage and CI audit (taken at 82f7159) that nothing in the quest tree already owned. Each claim was checked against the code, and today's scheduled runs were triaged.

What already had an owner

Nightly recovery is m0/nightly, media fuzzing m2/media-fuzz, wire fuzzing plus Miri for net/sock/uring/c m2/wire-fuzz-coverage, the resume model m1/resume-model-test, NVIDIA silent passes m2/gpu-ci, HLS/DASH m2/hls-player-test, coverage interpretation m2/audit-test-coverage, and bench calibration m1/bench-ci plus siblings.

Rejected: differential Rust/JS fuzzing (a heavy harness, little over shared vectors), rotating JS per-file isolation runs (speculative, never reproduced), instrumenting Rust under OBS TSan, and Clippy on platform targets (unverified gap).

Changes

  • m0/nightly: three children for the 2026-10-27 failures with no owner (run 40506938673): OBS TSan's wall-clock bound, the paired bench regressions, and Firefox's unclosed VideoFrame. The plan first had six; on merging main, Loom's log race (moq-dev/moq#7134) and the mechanical breaks (moq-dev/moq#7047, moq-dev/moq#7120, moq-dev/moq#7121) were already fixed, and moq-dev/moq#7119 owns render-sw's DMA-BUF assertion (render-capabilities), the WASM baseline (browser-baseline), and serve-bench (fixed by moq-dev/moq#7138), so those were dropped.
  • m2, new: Miri for in-memory unsafe (kio, moq-net), the opt-in test inventory gate, zizmor plus govulncheck/pip-audit, the worklet matrix and React hooks tests, format gates (gofmt, ktfmt, swift-format, clang-format, Ruff scope), and weekly cargo-mutants.
  • m2, amended: vaapi-render-tests takes the VA-API decode and transcode tests that pass without a device; gpu-ci names moq-transcode's hardware_available tests.
  • m2, reranked: right after release execution come media-fuzz, wire-fuzz, Miri, VA-API, inventory, security, browser, format, and mutants.

Public API: none. Wire: none.

Decision prompts

Goal: plan quests only for the audit gaps nothing owns, and fold extra scope into owned quests?

  • Yes
  • ✅ Yes, and reprioritize

Tooling quests (multi-select)

  • ✅ Formatting gates
  • ✅ Security linting
  • ✅ Test inventory check
  • ✅ cargo-mutants pilot

Coverage quests (multi-select)

  • ✅ Miri for kio/moq-net
  • ✅ VA-API silent passes
  • ✅ Browser test gaps

Priority

  • ✅ Raise in m2 (media-fuzz, wire-fuzz, then the new ones, right after release execution)
  • media-fuzz into m1
  • New quests at m2's end

Formatters

  • ✅ ktfmt + swift-format (+ clang-format with OBS's config)
  • ktlint + SwiftFormat

Inventory scope

  • ✅ Opt-in targets (ignored tests, fuzz/bench targets, test recipes, standalone scripts), in the merge gate
  • Fuzz/bench/ignored only

Security placement

  • ✅ zizmor in the merge gate with reviewed config; govulncheck/pip-audit nightly
  • All nightly

Mutants

  • ✅ Weekly, scoped to moq-auth and moq-net permission/resume/expiry, no score gate
  • One-off survey
  • Nightly, scoped

Nightly children

  • ✅ 6 children (mechanical breaks bundled)
  • One per cause (10)
  • Fix mechanicals here

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Six Nightly children own the 2026-10-27 failures nothing owned (Loom on
kio's log, render-sw, OBS TSan, paired bench regressions, browser legs,
and three mechanical breaks). Six m2 quests add Miri for in-memory
unsafe, a test-inventory gate, workflow and dependency security linting,
the worklet matrix and React hooks tests, format gates for every
language, and weekly mutation testing. VA-API and GPU CI absorb the
remaining silent hardware passes, and the fuzz quests move up in m2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 24 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5212bc6d-98a5-454f-8e39-32e20cb57ba9
Commits

Reviewing files that changed from the base of the PR and between 66bdbb9 and 4052518.

Files selected for processing: 13
  • quest/m0/nightly/README.md
  • quest/m0/nightly/bench-regressions.md
  • quest/m0/nightly/firefox-videoframe.md
  • quest/m0/nightly/obs-tsan.md
  • quest/m2/README.md
  • quest/m2/browser-test-gaps.md
  • quest/m2/format-gates.md
  • quest/m2/gpu-ci.md
  • quest/m2/miri-memory.md
  • quest/m2/mutants.md
  • quest/m2/test-inventory.md
  • quest/m2/vaapi-render-tests.md
  • quest/m2/workflow-security.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T15:26:23.194509Z 4052518 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: bb004fd.

No actionable bugs found in this planning-only change. The direction is sound: prioritize the suspected Loom memory-safety failure, reuse existing quest owners and one shared Miri job, and keep costly mutation testing scoped and outside the merge gate. The worklet and React coverage goals match the inspected harness and hooks; explicit ignored hardware tests are more honest than silent passes. An opt-in inventory is a simpler boundary than trying to infer all test coverage.

These are planned fixes, not implemented coverage or recovered CI. Verification was static review of the changed plans and relevant repository context; I did not run tests, hardware checks, or reproduce the cited scheduled-run failures.

(Written by OpenAI)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Summary: plans the coverage audit's unowned findings. It adds six Nightly children for the failures nothing owned on 2026-10-27, six m2 quests (Miri, test inventory, security linting, browser gaps, format gates, weekly mutants), and VA-API/GPU CI amendments, and moves the fuzz quests up in m2. Every decision is in the PR body. Codex review clean; the maintainer approved the merge in session.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 28, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 28, 2026
kixelated and others added 2 commits October 29, 2026 08:23
Main fixed the Loom log race (#7134) and the three mechanical breaks
(#7047, #7120, #7121), and #7119 planned render-capabilities and
browser-baseline, so remove loom-log, render-sw, and small-breaks, and
narrow browser-legs to the Firefox VideoFrame leak. Drop the Related link
to the finished cargo-deny quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Re-landing after the merge-queue merge_conflict. I merged main (the quest driver resolved the link lists) and lined the plan up with what main has done since:

Codex gave head 4052518 a 👍 with no findings, the required checks pass, and quest check passes.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit c8a2405 Oct 29, 2026
11 checks passed
@kixelated
kixelated deleted the quest-plan/coverage-audit branch October 29, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant