Skip to content

test(mcp): precompile cargo fixtures before the runner deadline phase - #949

Merged
ScriptedAlchemy merged 13 commits into
codex/tracedecay-total-redesign-plan-reopenedfrom
cursor/separate-fixture-compile-from-runner-deadline-dccb
Sep 7, 2026
Merged

test(mcp): precompile cargo fixtures before the runner deadline phase#949
ScriptedAlchemy merged 13 commits into
codex/tracedecay-total-redesign-plan-reopenedfrom
cursor/separate-fixture-compile-from-runner-deadline-dccb

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • Separates cold cargo fixture compilation from the test-runner execution contract in crates/tracedecay-mcp/src/workflow/test_runner.rs tests. The three Windows exact-test runner cases (cargo_runner_executes_every_requested_exact_test_once, cargo_runner_reports_passing_and_failing_exact_tests, cargo_runner_rejects_a_vacuous_exact_test_run) now provision the fixture (compile + discovery) under a separate budget before running the phase that asserts against the genuine 10s production deadline.
  • Adds a distinct cold-build contract test: cancelling while a build script stalls the compile reports Cancelled and reaps the whole cargo process tree (reap assertion on unix).
  • Production timeout semantics are unchanged; no timeout is converted to NoMatch and no production deadline is raised.
  • Also fixes a pre-existing base-branch compile error that failed the hotpath-coverage / slice-tests (hotpath on) check: orphaned #[hotpath::measure] attributes left on mod tests in crates/tracedecay-global-db/src/schema_contract/invariants/triggers.rs after 128fa8fa2 deleted the functions they annotated.

Fixes #935.

Motivation

On Windows the exact-test runner cases hit their operation timeout while stderr still said Compiling a tiny fixture, so the assertions about exactly-once selection, retained failing results, and NoMatch were never actually reached. Cold compilation and the runner's execution contract were conflated. See #935 (filed against the PR #707 CI baseline).

Changes

  • crates/tracedecay-mcp/src/workflow/test_runner.rs
    • Production: split the compile-unit flags out of cargo_test_args into a private cargo_test_build_args(profile); cargo_test_args output is byte-identical (covered by the existing cargo_test_args_* tests in crates/tracedecay).
    • Tests: ProvisionedFixture / provision_fixture writes the package, runs the compile phase (cargo test --no-fail-fast --no-run, i.e. the production flags plus --no-run) and the discovery phase (-- --list) with a 5-minute provisioning budget and a fresh TestRunControl so provisioning output does not consume the execution phase's output budget. Recorded phase timings and the discovered test list are included in every assertion message so a failure is attributed to compile, discovery, execution, or result collection.
    • Tests: ProvisionedFixture::execute runs run_cargo_tests against the unchanged EXACT_TEST_EXECUTION_DEADLINE (10s); a Timeout on a warm fixture panics with the phase record and captured stderr instead of being masked.
    • Tests: each warm case asserts discovery saw exactly the fixture's tests, so NoMatch in the vacuous case is proven to be about selection rather than a fixture that never contained the test.
    • Tests: new cancelling_a_stalled_fixture_compile_terminates_the_cargo_process_tree uses a build.rs that publishes its pid (write + rename) and sleeps; a canceller waits for the marker, then control.cancel(). Asserts the compile actually reached the stall, the result is Cancelled (not Timeout/NoMatch), and on unix that the build-script process is reaped.
  • crates/tracedecay-global-db/src/schema_contract/invariants/triggers.rs
    • Removes two orphaned #[hotpath::measure] attributes above #[cfg(test)] mod tests. With the hotpath feature off the macro is a passthrough; with it on it demands an fn, so the hotpath-on slice tests failed with expected fn.

Test plan

  • cargo test -p tracedecay-mcp --lib workflow::test_runner — 10/10 pass, repeated 3x (Linux)
  • cargo test -p tracedecay --lib cargo_test_args — 2/2
  • Exact CI slice command locally: cargo test -p tracedecay-rusqlite-runtime -p tracedecay-global-db --locked --features tracedecay-rusqlite-runtime/hotpath,tracedecay-global-db/hotpath,tracedecay-global-db/test-helpers — all green
  • cargo clippy -p tracedecay-mcp --lib --tests — no warnings in test_runner.rs (the only warning in the build is a pre-existing GraphProjectionLabelPage dead-code warning in tracedecay-graph-db)
  • cargo fmt -p tracedecay-mcp -- --check
  • npm run lint:commit
  • Windows shards on this PR's CI should show the three exact-test cases passing with warm execution; if provisioning itself is slow the failure message now names the phase.

Checklist

  • CHANGELOG.md updated — test-only change, no entry
  • No secrets, credentials, or .env files included
  • Breaking changes documented (none)

Targets codex/tracedecay-total-redesign-plan-reopened (PR #707). #707 stays draft; this PR does not undraft or merge it.

Open in Web Open in Cursor 

The Windows exact-test runner cases timed out while cargo was still
compiling a tiny fixture, conflating cold compilation with the runner's
execution contract. Provision each fixture (compile with the production
cargo flags plus --no-run, then discover the tests) under a separate
budget before running the phase that asserts against the genuine
production deadline, and attribute failures to the compile, discovery,
execution, or result-collection phase.

The cold-build contract gets its own test: cancelling while a build
script stalls the compile reports Cancelled and reaps the whole cargo
process tree.

Refs #935

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
@changeset-bot

changeset-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 99a66fd

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

ScriptedAlchemy and others added 4 commits September 6, 2026 14:52
…sor/separate-fixture-compile-from-runner-deadline-dccb
128fa8f deleted suspend_immutability_for_canonical_repair and
restore_immutability_after_canonical_repair but left their
#[hotpath::measure] attributes above the tests module. The macro is a
passthrough with the hotpath feature off and demands an fn with it on,
so the hotpath-on slice tests failed to compile.

Refs #935

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
…from-runner-deadline-dccb' into cursor/separate-fixture-compile-from-runner-deadline-dccb

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
…sor/separate-fixture-compile-from-runner-deadline-dccb

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — fixture phase split accepted; HOLD for conflicts and Windows execution

Reviewed 9f5c2cd48cc2d4f0ca1e3432455ceb840cb618b5; source review only. GitHub reports conflicts; exact-head CI is pending. Separate cold provisioning and exact-test execution is correct, and the production cargo_test_args behavior should remain unchanged.

Keep the 10-second execution contract and independently assert discovered test identities before the exactly-once/no-match cases. Provision with the same toolchain/profile/feature/target environment used by execution so the supposedly warm phase does not silently compile a different resolution. A warm execution timeout remains a real failure; never translate it to NoMatch.

P1 acceptance: the new compile-cancellation case asserts process-tree reap only on Unix. Add native Windows evidence that cargo and the stalled build-script child are both gone, file locks are released, and a successor build can run. Otherwise this PR repairs the selection tests while leaving the original platform's cancellation lifecycle unproven.

P2 integration: #953 removes the same orphaned hotpath attributes. Land that deletion once and preserve it while resolving both branches; no duplicated workaround or coverage skip. Retain at least one genuinely cold compile cancellation/timeout test so provisioning doesn't remove the runner's compile-phase contract from coverage.

Final gate: affected exact-runner tests plus their Windows shard and hotpath-on slice on the resolved #707 merge tree. #707 remains draft.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Performance Comparison codex/tracedecay-total-redesign-plan-reopenedcursor/separate-fixture-compile-from-runner-deadline-dccb

Total Elapsed Time: 4.88s → 4.73s (-3.0%)
CPU Baseline: 248.91µs → 246.57µs (-0.9%)
Benchmark ID: index-bench-timing

timing - Execution duration of functions.

+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| Function                                 | Calls                      | Avg                             | P95                             | Total                            | % Total                      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| tracedecay-index-bench                   | 1 → 1 (+0.0%)              | 4.88s → 4.73s (-3.1%)           | 4.88s → 4.73s (-3.1%)           | 4.88s → 4.73s (-3.1%)            | 100.00% → 100.00% (+0.0%)    |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.append_pages              | 13 → 13 (+0.0%)            | 169.73ms → 180.94ms (+6.6%)     | 228.33ms → 257.29ms (+12.7%)    | 2.21s → 2.35s (+6.3%)            | 45.25% → 49.73% (+9.9%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.sqlite              | 13 → 13 (+0.0%)            | 108.42ms → 111.40ms (+2.7%)     | 145.62ms → 158.73ms (+9.0%)     | 1.41s → 1.45s (+2.8%)            | 28.90% → 30.62% (+6.0%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.workers.install               | 83 → 83 (+0.0%)            | 18.75ms → 16.64ms (-11.3%)      | 55.21ms → 64.52ms (+16.9%)      | 1.56s → 1.38s (-11.5%)           | 31.91% → 29.19% (-8.5%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.postings            | 13 → 13 (+0.0%)            | 98.09ms → 100.40ms (+2.4%)      | 133.23ms → 146.41ms (+9.9%)     | 1.28s → 1.31s (+2.3%)            | 26.15% → 27.59% (+5.5%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| domain.canonical.sha256                  | 114526 → 114526 (+0.0%)    | 11.60µs → 9.42µs (-18.8%)       | 16.20µs → 16.14µs (-0.4%)       | 1.33s → 1.08s (-18.8%)           | 27.24% → 22.81% (-16.3%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.build.and_publish             | 2 → 2 (+0.0%)              | 602.88ms → 491.40ms (-18.5%)    | 628.62ms → 503.84ms (-19.8%)    | 1.21s → 982.80ms (-18.8%)        | 24.73% → 20.78% (-16.0%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.prepare_pages             | 13 → 13 (+0.0%)            | 49.20ms → 56.13ms (+14.1%)      | 68.03ms → 82.64ms (+21.5%) ⚠️   | 639.62ms → 729.73ms (+14.1%)     | 13.12% → 15.43% (+17.6%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.chunk.index_file              | 276 → 276 (+0.0%)          | 2.80ms → 2.22ms (-20.7%) 🚀     | 6.19ms → 3.26ms (-47.3%) 🚀     | 772.43ms → 612.57ms (-20.7%) 🚀  | 15.84% → 12.95% (-18.2%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.parallel_prepare    | 13 → 13 (+0.0%)            | 39.65ms → 47.10ms (+18.8%)      | 55.21ms → 71.24ms (+29.0%) ⚠️   | 515.44ms → 612.27ms (+18.8%)     | 10.57% → 12.94% (+22.4%) ⚠️  |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.extract.parser_artifact       | 276 → 276 (+0.0%)          | 2.55ms → 2.04ms (-20.0%)        | 5.80ms → 3.13ms (-46.0%) 🚀     | 704.55ms → 562.10ms (-20.2%) 🚀  | 14.45% → 11.88% (-17.8%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.finalization.advance_wake | 14 → 14 (+0.0%)            | 34.57ms → 35.49ms (+2.7%)       | 217.06ms → 221.51ms (+2.1%)     | 483.94ms → 496.89ms (+2.7%)      | 9.92% → 10.51% (+5.9%)       |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.chunk.build                   | 276 → 276 (+0.0%)          | 1.78ms → 1.37ms (-23.0%) 🚀     | 4.79ms → 1.99ms (-58.5%) 🚀     | 492.47ms → 378.37ms (-23.2%) 🚀  | 10.10% → 8.00% (-20.8%) 🚀   |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| 🆕 code_index.chunk.map_ordered          | 0 → 780 (+100.0%) ⚠️       | 0.00ns → 483.91µs (+100.0%) ⚠️  | 0.00ns → 629.25µs (+100.0%) ⚠️  | 0.00ns → 377.45ms (+100.0%) ⚠️   | 0.00% → 7.98% (+100.0%) ⚠️   |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.chunk.identify                | 276 → 276 (+0.0%)          | 1.77ms → 1.35ms (-23.7%) 🚀     | 4.77ms → 1.98ms (-58.5%) 🚀     | 487.96ms → 373.88ms (-23.4%) 🚀  | 10.01% → 7.90% (-21.1%) 🚀   |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| 🗑️ code_index.build.materialize_full     | 1 → 0 (-100.0%) 🚀         | 422.92ms → 0.00ns (-100.0%) 🚀  | 423.10ms → 0.00ns (-100.0%) 🚀  | 422.92ms → 0.00ns (-100.0%) 🚀   | 8.67% → 0.00% (-100.0%) 🚀   |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+

Generated with hotpath-rs

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System-level re-review at 9f5c2cd48 — source change is appropriate; keep the two contracts separate

Read the complete remaining diff. No new production correctness defect found. cargo_test_build_args is a mechanical extraction and the test-execution arguments retain their existing exact-selection behavior. Provisioning and discovery run under a separate control/output budget, then the actual runner executes with the unchanged 10-second deadline. The new exact parsed-result assertions are stronger than merely checking that a success substring appears.

Keep the distinction explicit: these warm tests prove selection/result collection and vacuous-run refusal; they do not prove that a real cold user invocation can compile inside ten seconds. The separate stalled-build cancellation test preserves cold-phase cancellation coverage. In the production protocol, Timeout, Cancelled, a failing test, and NoMatch must remain different outcomes; no provisioning failure should be translated into a successful empty result.

P2 coverage: assert_reaped is still Unix-only. A Windows Cancelled result alone does not prove that cargo's build-script descendant was terminated. Verify process-tree retirement using an owned child/process handle or the existing Windows process owner, without PID-reuse assumptions. This is a missing platform proof, not a demonstrated Windows leak. Preserve the marker proving the build script actually entered its stalled phase before cancellation.

The two orphaned hotpath attributes should be removed once in the final integration tree; do not preserve a duplicate conflict just because another portability branch contained the same deletion. Avoid introducing production fixture ports or a broad runner redesign for this test split.

Returned CI is pending, profile succeeded and coverage/plugin runs queued. No repository tests were executed here. This source review supports the scoped change, not undrafting #707 or treating the historical CI matrix as current acceptance.

@ScriptedAlchemy

ScriptedAlchemy commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Final review-fix evidence — 8f785b7de

Native Windows CI is queued at run 34106360539. GitHub has not allocated a runner despite multiple hours of waiting, so no native shard receipt exists yet; this comment deliberately does not claim one.

@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review September 7, 2026 11:51
…sor/separate-fixture-compile-from-runner-deadline-dccb
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-07T11:52:54.502975Z 99a66fd Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ScriptedAlchemy
ScriptedAlchemy merged commit 6a191cf into codex/tracedecay-total-redesign-plan-reopened Sep 7, 2026
1 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants