Engineering Preview: remaining product and release evidence

[Ketchio-dev]

This is the single active tracker for the remaining BridgeVM product and release-evidence work. Historical checkpoints remain in the comments; they are evidence records, not the current work list.

The authoritative product state, thresholds, known defects and evidence live in capabilities/windows-hvf.json. Product state remains ENGINEERING_PREVIEW. General Preview and v1 are 3D-off; Vulkan and the D3D11-compatible subset remain experimental Graphics Lab future paths.

Release-blocking work

• A9 — clean-machine product flow: retain passing ISO-install and installed-disk-import journeys through the packaged app on the supported 3D-off path. Existing pilots and failures remain evidence; neither complete journey is proven.
• A11 — exact release-head regression seal: finish current own-HVF app/runtime safety work, then retain the complete exact-SHA release check and hosted verification required by the registry. Deterministic component checks do not close the live product journey.
• A19 — snapshot lifecycle: complete the unchanged powered-off disk-plus-vars criterion, including the remaining live selected-generation export/restore and interruption or physical power-loss coverage. Deterministic publication and export checks remain supporting evidence only.

Product work supporting those gates

• Validate the packaged app's own-HVF create/install/import/start/open/status/stop flow on real hardware without source-tree dependencies.
• Verify the native UI with real user actions and Accessibility dispatch, including recovery and cancellation states. SwiftUI value tests and synthetic AX graphs do not establish rendered behavior.
• Keep lifecycle controls bound to the exact app-owned native session and fail closed during stale registration or ownership transitions. PR Fail closed when an HVF detail session is unavailable #223 merged on 2026-09-20 after exact-head hosted checks; its unavailable-session route is implemented, while live ownership behavior remains within the open product evidence work.
• Install and audit the latest exact-main Apple Development artifact without stopping or mutating the currently running older preview VM; validate a fresh 3D-off launch separately.

Non-blocking evidence still open

• B6: run the declared 3-resolution × 3-scale × 3-run glyph matrix with verified masks and the unchanged frame-time clause. The observed blank sign-in glyphs on an old experimental-3D preview remain a defect, not a passing sample.
• B8: retain install, boot, update and rollback receipts for the declared M4 and M5 matrix. Undeclared generations remain disclosed as uncovered.
• B9: complete the declared 20-workload matrix with retained p50, p95 and p99 frame times and exact result classes.

Integration rules

Retain fixed-sample live receipts and failed experiments. Do not reduce thresholds, substitute smoke tests, infer guest behavior from deterministic tests, or promote the product while a release-blocking row remains open. Deterministic work runs on GitHub-hosted Actions; the physical-Mac queue is reserved for work requiring Apple-silicon HVF, WindowServer/CGL, private Windows media or a real GPU.

A change is integrated only after implementation, local project validation, exact-PR-head hosted verification, normal merge, exact-main push verification and safe branch cleanup. Private disks, vars, vTPM state, recovery material and guest assets never enter git or CI artifacts.

[Ketchio-dev]

Evidence/performance checkpoint (2026-09-01): T16 campaign a40f06e9d0835f324f413371fed57d75 completed 20/20 valid runs as ten predetermined counterbalanced A/A pairs at harness 3ffb66b1. Identical binary bytes produced read-throughput label medians 555.565 and 542.926 MiB/s. The paired directional delta was +2.724%, bootstrap 95% CI -25.526% to +7.796%, so the primary A/A noise bound is 25.526%.

The PRP-prefix candidate ceiling was frozen at 2.94% before metrics were read. Since 25.526% >= 2.94%, the registered decision is STOP_NO_IMPLEMENTATION_OR_AB. No performance improvement is claimed and the invalid earlier campaigns remain recorded as failures. External fail-closed report SHA-256: f90684a99ccb05b3d5fd8d90e58aa0bfc031f151cf8007119ea4d76142a30204.

The exact-main hosted performance-binary workflow 33550480180 is green. Its final binary SHA-256 c10d3373cc2e6a8d9751abbe3158aeeba0de17b4ef2481e4c57a7e7d521d78cd passed GitHub attestation verification against source SHA 34043bdd470a4556feff558ce3e813a4acf737bd, the pinned signer workflow, and the self-hosted-runner denial rule. Future claim-bearing calibration/A-B must start from this attested artifact and a lower-noise workload design; it cannot reuse the locally signed A/A baseline for source attribution.

This issue remains OPEN for A9, glyph, audio, clean-machine, compatibility, and other stated evidence gaps.

[Ketchio-dev]

Exploratory A/A variance decomposition, without changing the stop decision:

• all-run read-throughput mean 535.729 MiB/s, standard deviation 106.540 MiB/s, CV 19.887%
• fixed 2.5 GiB read phase duration ranged 3.317–9.619 s, CV 26.289%
• measured read service duration ranged 1.538–6.501 s, CV 34.231%; service bandwidth CV 28.592%
• phase-minus-service overhead median 1.907 s, CV 15.496%
• read p99 CV 74.577%; Pearson correlation between read throughput and read p99 was -0.832
• Spearman ordinal-versus-throughput correlation was only -0.205, so the data does not support a simple monotonic campaign-time explanation
• median absolute within-pair label delta was 8.715%, maximum 65.518%; pair 6 and pair 8 remain in the record and are not trimmed

This is diagnostic description, not a cause claim and not a post-hoc rescue of the stopped PRP candidate.

Proposed next calibration contract, to freeze before collecting new metrics:

1. Use the exact-main attested baseline (c10d3373cc2e6a8d9751abbe3158aeeba0de17b4ef2481e4c57a7e7d521d78cd).
2. Version the workload rather than rewriting this result. Keep 512 MiB, 128 KiB, QD1, unbuffered and verified I/O, but raise measured read volume from 2.5 GiB to at least 16 GiB per lane so read service lasts roughly 30 seconds at the observed median.
3. Emit fixed-size read epochs and retain every epoch. The independent statistical unit remains the disk-and-vars lane; epochs measure within-lane instability and are not treated as extra samples.
4. Run a small counterbalanced A/A pilot only as a rejection filter, then the unchanged ten-pair full A/A for any noise claim. No outlier deletion, trimming, threshold lowering, or substitution of a host microbenchmark.
5. Require the full A/A read-throughput CI noise bound to be below the candidate-specific ceiling before profiling or implementation. Read p99 and all write/flush/desktop guardrails remain binding.

No new campaign has been authorized by this comment; it records the next pre-registration target after the valid STOP decision.

[Ketchio-dev]

Methodology cross-check against primary/official guidance:

• Google Benchmark documents repeated raw runs plus mean, median, standard deviation and CV, and reports that random interleaving can reduce run-to-run variance: https://google.github.io/benchmark/user_guide.html and https://google.github.io/benchmark/random_interleaving.html
• Kalibera and Jones model multiple non-determinism levels, including repeated VM invocation, and recommend effect-size confidence intervals: https://kar.kent.ac.uk/33611/
• SPEC CPU run/reporting rules treat stated observation conditions and reproducibility as part of a published performance claim: https://www.spec.org/cpu2017/Docs/runrules.html

BridgeVM already retains every lane, separates independent disk-and-vars VM lanes from within-lane operations, publishes paired bootstrap CIs, seals identities, and refuses claims when noise exceeds the candidate ceiling. One refinement for the proposed versioned calibration is a pre-sealed public random seed that chooses balanced first-label order within pairs, while still requiring exactly five baseline-first and five candidate-first pairs. This guards against an unknown periodic confound; it does not assume the microbenchmark variance reduction reported by Google transfers to this Windows VM workload.

[Ketchio-dev]

Evidence checkpoint (2026-09-01):

Exact main SHA 2dfddfb4a2114cffec14d215fb821b9673951b0d is green in hosted CI 33566951683 and Security 33566951658. The newly added T17 product-E2E harness and receipt validators are present and their deterministic/macOS contract suites pass.

This does not close any live Windows criterion. No validated private Windows ISO plus external signed ARM64 driver payload/manifest was available, so no claim-eligible T17 pilot or release campaign has run. A9, glyph/audio/clean-machine/compatibility evidence, and any other live gaps remain OPEN.

The prior performance decision also remains unchanged: STOP_NO_IMPLEMENTATION_OR_AB. The measured A/A noise bound was 25.526%, above the frozen 2.94% candidate ceiling, so there is still no performance-improvement claim. Future work must first pass a pre-registered lower-noise A/A calibration; no threshold was relaxed and no samples were removed.

A non-publishing packaging dry-run for this exact SHA is sealed as run 33567220597 and remains asynchronous. Its final result will be recorded separately.

[Ketchio-dev]

Live-product evidence checkpoint (2026-09-01):

A11 is now sealed at 44e808e7e7178f673429a8ba2058d80b6f83aab4 over tested code head 12ca8c4657fa380ef74e0229a11edc0b42f639d9. Exact seal-SHA T0 job t0-44e808e7-full-check passed the complete project check; hosted CI 33571413958, Security 33571414115, and non-publishing release dry-run 33571419041 are all green. The downloaded package passed SHA256SUMS, deep codesign, third-party notice/runtime checks, and staging of the complete external ARM64 storage/serial/network payload through the helper actually bundled in the app.

The private T17 inputs are now real rather than synthetic: Windows 11 25H2 Arm64 ISO SHA-256 638aa2c88e94385b00f4f178d071e3df0b7d9e335577a83bd533b7f2eb65adf0, official Fedora virtio-win 0.1.302 external payload, and the seal-SHA packaged app. All ten manifest assets passed strict preflight; manifest SHA-256 is c21ea648daf2acc6077638be903ef20a99d7e177d7d7b9f49caabbae61af1aed. Real-hardware pilot t17-44e808e7-pilot is running through the local sealed queue. It may spend up to 30 minutes in installation before exercising boot, keyboard/pointer, Korean clipboard, folder sharing, network/audio, shutdown/reboot, snapshot/restore and guest proof.

No live criterion is closed yet. A passing pilot would still be one ad-hoc-signed, 3D-off lane rather than the unchanged release criterion. A9, signed 3D provenance, clean-machine/release evidence and the other open compatibility gaps remain open until their actual receipts pass.

Performance status is unchanged: the valid T16 A/A noise bound is 25.526%, above the frozen 2.94% candidate ceiling, so STOP_NO_IMPLEMENTATION_OR_AB remains binding. No samples were removed and no improvement is claimed.

[Ketchio-dev]

Live-evidence correction and next boundary (2026-09-01):

The two T17 pilots reported previously did not pass. The first exposed a stale resident-worker dispatch contract and never reached the helper. The second reached the exact packaged helper and launched BridgeVM, then failed closed with accessibility-untrusted before VM creation. Both failures and their receipts remain in the record; they prove neither Windows installation nor any keyboard, pointer, clipboard, sharing, network, audio, reboot, Secure Boot, or snapshot behavior.

The worker recurrence is fixed at tested code SHA 3c46c86788145dc66a9f8fd9727aa7acd8c7e1f1, and exact evidence seal bfddb92e76388f0cd0e735b800933ae34023fbcc is green in full T0, hosted CI 33573244217, Security 33573244119, and release dry-run 33573256223. The downloaded package and its app-local catalog verifier passed checksum, deep-codesign, third-party notice/runtime, forbidden-content, and real three-role external payload staging checks. A fresh strict ten-asset manifest for that exact package is privately retained with SHA-256 7bcacfda86737e9bf8f823a4af1095de85f1f450153a3a2ab2161e9a1ff6d483.

The next live T17 job is intentionally not queued while macOS denies Accessibility access to dev.bridgevm.control; repeating it would deterministically burn another job id without product evidence. That security setting remains for explicit user authorization. Even a future passing pilot will be one ad-hoc, one-lane, 3D-off live signal and will not satisfy a multi-run release criterion.

Performance status is also unchanged: the valid T16 v1 A/A bound is 25.526%, above the frozen 2.94% candidate ceiling, so STOP_NO_IMPLEMENTATION_OR_AB remains binding. The next legitimate step is a separately versioned, pre-registered lower-noise A/A calibration without removing samples or relaxing any threshold. All other live rows named by this issue remain OPEN until their unchanged gates pass.

[Ketchio-dev]

Performance evidence checkpoint (2026-09-01):

T16 v2 is implemented and exact-head sealed at 9037c8b. Studio T0 job 20260901-211304-91774-17183 passed the full deterministic project check (check-log SHA-256 25bfb0f88c95f46cbe0160da247194ef79b43d8aa6f88ba25af56806fd468cad), and hosted CI 33578404234 plus Security 33578404173 are green. The hosted T16 v2 contract passed on Linux, including the portable synthetic fixture path that had failed in an earlier retained CI run.

The final pre-registration is campaign 11a43503d40b71623f0718cf55e1fbb2: 24 A/A pairs / 48 isolated lanes, balanced 12 AB and 12 BA, fixed workload and thresholds, all samples retained. Private schedule receipt SHA-256: 63af5ed99fe59e7d849131f148465a55ada0b488a20ceab827fd86a973b60fb9.

This is measurement readiness, not a result. No live v2 number and no optimization claim exist yet. Submission remains fail-closed because the frozen 320 GiB free-space requirement exceeds the current 156,954,376 KiB availability by 178,589,944 KiB (about 170.3 GiB). Old diagnostic/live VM evidence is being preserved pending explicit archive/delete authorization. T17 remains separately blocked by macOS Accessibility denial before VM creation, so the product/evidence issue stays OPEN.

[Ketchio-dev]

Exact-seal checkpoint (2026-09-01):

The evidence-truth implementation is sealed at main SHA 350a7e5. Studio T0 job 20260902-022402-t0-350a7e55 passed the complete project check; retained check-log SHA-256 is c6cc807d45709233fe265ba2c0ee591d232f5bd62b0a8297c5ab6ebab6da925e. Hosted CI 33583073728 and Security 33583073675 are fully green at the same SHA.

The capability registry now carries B6 glyph correctness, B7 typed audio teardown, B8 clean-machine hosts, B9 the 20-workload matrix and B10 exact hosted QMP stress as separate OPEN rows. The audio workstream wording was corrected from total callback_errors=0 to zero unexpected callback errors with expected stopping statuses separately typed and counted.

Claim-eligible QMP run 33583360863 and attested release-binary build 33583401432 are now running at the exact green seal. They are pending, not passes. T18 has not been submitted; B7 requires its immutable ten-run receipt. A9, B6, B7, B8, B9 and B10 therefore remain OPEN, T17 remains blocked before VM creation by Accessibility denial, and T16 v2 still lacks the required disk headroom. No performance gain is claimed and this issue remains OPEN.

[Ketchio-dev]

B7 live criterion result (2026-09-01):

Studio T18 job 20260902-023340-t18-350a7e55 completed at exact green seal 350a7e5. Its fixed ten independent disk-and-vars lanes passed 10/10 with no replacement: 2,504,031 rendered frames, zero drops, zero unexpected callback errors, zero active/queue-invalidated/unclassified errors, and zero AudioQueue stop/dispose errors. The 30 teardown statuses were all expected EnqueueDuringReset and separately typed. Cleanup was verified.

Manifest SHA-256: 3a8cf8ed9fa5d134fac6ef6b345816df01e70a24276fdbe91e826dd79244091a. Binary SHA-256: 5912a1f291d9935d3d4d2270b37829de543645db2583c3e3f257b9296f1493e8. Run-log-set SHA-256: dd4780eb4a329a8bee97fbab2e307451733584becefc92a12fc8c4fb5176d853. Public receipt SHA-256: 5844b7a086e7a3877ec34c0365331f23ea0f2baa2f62b0eab79d1ea0de491081; the independent verifier and private ordinal/nonce cross-check passed.

B7 is now PROVEN in evidence commit 44fecf8. The receipt deliberately keeps claim_eligible=false and capability_promotion=false, so this closes only B7. Product state remains Engineering Preview; A9, B6, B8, B9 and B10 remain OPEN. Exact evidence-SHA T0 20260902-024538-t0-44fecf8f, CI 33584424375 and Security 33584424350 are running and must be green before the evidence seal is final.

[Ketchio-dev]

B7 evidence seal is now green (2026-09-01): exact main SHA 44fecf8 passed Studio T0 job 20260902-024538-t0-44fecf8f with project check PASS; retained check-log SHA-256 is d124abea2750445300a51b8f4bd0da5521aa80a6b1a0f6914bf720c19e763d6a. Hosted CI 33584424375 and Security 33584424350 both completed successfully at the same SHA. B7 is therefore sealed PROVEN. The issue remains OPEN for A9, B6, B8, B9 and B10; no product-state promotion follows.

[Ketchio-dev]

B10 evidence seal (2026-09-02):

GitHub-hosted QMP run 33583360863 at exact commit 350a7e5 passed the unchanged criterion: under 24 load processes the old setup ordering reproduced EINVAL then ECONNREFUSED 20/20, followed by exactly 60/60 sequential cargo test --workspace --locked rounds with zero failures. The downloaded artifact contained exactly 60 compressed logs; the strict verifier and an independent post-download audit matched all 60 gzip hashes and all 60 decompressed raw hashes. Receipt SHA-256: c62e72fc83393bdb63b45e6f6c86a310f72f497440d3b4f5ceb7f7939a6a1660.

B10 is recorded PROVEN at current evidence head 5bc8438. That exact head passed Studio T0 job 20260902-001100-93183-18835 (project check PASS; check-log SHA-256 259def1235044a61ecc67191b8694e0f831a6fc733c53cbf6b80d64573adac9c), hosted CI 33589814009 and Security 33589814013.

This closes only deterministic QMP workspace stress. It does not prove guest behavior, product E2E or a performance gain, and product state remains Engineering Preview. This issue remains OPEN for A9, B6, B8 and B9.

[Ketchio-dev]

T17 packaged-product correction checkpoint (2026-09-02):

• Exact sealed package pilot t17-86c5fd68-v1.1.0-pilot-r1 is retained as a failure. The package and ten-asset manifest passed preflight, Accessibility was trusted and the real app frontend started, but no product stage completed; the lane stopped with ui-element-missing. Public receipt SHA-256: 086f88b7c6221977025c724ee54a4083f44b976fc6d847f14cecac0f420b74e4.
• The failing state and source contract exposed a fresh-library mismatch: BridgeVM renders FirstRunView for an empty library, while T17 waited for bridgevm.library.empty.create, which exists only in the separate emptyState. Code head 5280ef56cc9f3f017aa36fe1196d9104b8437174 now enters through the always-visible bridgevm.library.toolbar.create control and adds a fixed identifier regression test.
• Focused Swift product-E2E tests pass 10/10, the deterministic T17 contract passes 41 checks, and the complete local project check passed before commit. At the exact code head, CI run 33605274851 passed every independent code/build/test job and failed only the deliberately stale capability/documentation registry; Security run 33605274784 passed all jobs. Studio T0 t0-5280ef56-a11-preseal likewise failed only registry freshness; every other project section passed and retained check-log SHA-256 is 95459b2c9b78189a7d499d6abfb3adf4ded3fd496bad1d329087d8ca218ae0fc.
• A locally derived pilot package for the new helper was not promoted and failed closed as accessibility-untrusted. Its ad-hoc designated requirement has a new CDHash, so macOS does not transfer the older TCC grant. Public receipt SHA-256: f9eb090c5a6bfbcfc852ac98a41553f488dd7fe2be9138f057f92db6e2359bfa.

The user-approved TCC replacement is currently waiting at the macOS Touch ID/password authorization sheet. No repeat is queued until the stale row is removed and the exact new helper is added. A9 remains OPEN, product state remains Engineering Preview, and neither failed pilot is reclassified.

[Ketchio-dev]

Exact release-seal checkpoint (2026-09-02):

• Seal commit 2028ae0084f82df44764e6d7d974f03332691c17 passed Studio T0 job t0-2028ae00-a11-seal; retained project-check log SHA-256 is ae533f2c3acab00e7f520c163af8c78b13dff1ee3b016574be78509b24925c6f.
• Hosted CI 33606943722, Security 33606943998, and release dry-run 33606942048 all completed successfully at that exact SHA.
• The downloaded v1.1.0 dry-run artifact passed an independent audit: every published SHA256SUMS entry matched; the tar contained 117 members under one BridgeVM.app root with no unsafe path/link and no .sys/.inf/.cat; tar and DMG app bundles passed deep strict code-signature, fixed nested-helper, and HVF-entitlement verification; both app and helper are arm64; DMG checksum is valid.
• The offline installer passed dry-run, fresh install, and update with --keep-backup; the update retained exactly one valid backup. Bootstrap pin verification also passed.
• The release contract remains fail-closed and honest: Engineering Preview, 3D-off, no kernel driver, no product injection, ad-hoc signed and not notarized. Gatekeeper rejected it as expected.

This validates packaging and installer behavior only. T17 product E2E has not passed: the exact corrected package pilot is prepared but cannot be queued until the currently visible macOS Touch ID/password sheet completes the Accessibility-row replacement. A9, B6, B8 and B9 remain OPEN; no product-state or performance claim is promoted.

[Ketchio-dev]

Repository cleanup checkpoint, 2026-09-16:

• Normally merged Add app-owned CLI startup with retained runtime supervision #162 (app-owned CLI startup), Bump actions/upload-artifact from 4 to 7 #152 (upload-artifact) and Bump actions/download-artifact from 4 to 8 #153 (download-artifact) after successful exact-head checks.
• Consolidated the duplicate general tracker Project takeover: close open product and evidence defects before release #100 into this issue; unresolved product/release criteria remain open.
• Latest observed main 278142b349e920e92954f31c9f82ef01a00faa63 still has pending postmerge workflows. Retire those merged branches only after that verification.
• Dependabot independently rebased Bump actions/attest from 4.2.1 to 4.2.2 #150, Bump actions/checkout from 4 to 7 #154 and Bump actions/setup-python from 5 to 7 #155 after integration. Their new heads require new checks; old-head green results do not authorize merging the replacements.
• Bump actions/checkout from 4 to 7 #154 run 35094326919 failed with native_failure child deadline: 10.0307493 seconds, confirmed child exit/drained output, zero stdout/stderr. The cause is not established. Preserve the ten-second gate and failure evidence; do not bypass or dismiss it.
• Support UUID-pinned external APFS storage for T17 #149 has four unique commits; the A9 and B6 research branches have one and five unique commits respectively, with retained worktrees/evidence. They are preserved pending deliberate integration or archival.
• Prior responsive GUI checkpoint aa20106ac605dad3b9c30f1e3047b8eae649690c passed all 40 hosted workflows. It will be integrated with the separately reproduced and tested installation-recovery fix, avoiding another parallel GUI PR.