test(mcp): precompile cargo fixtures before the runner deadline phase - #949
Conversation
The Windows exact-test runner cases timed out while cargo was still compiling a tiny fixture, conflating cold compilation with the runner's execution contract. Provision each fixture (compile with the production cargo flags plus --no-run, then discover the tests) under a separate budget before running the phase that asserts against the genuine production deadline, and attribute failures to the compile, discovery, execution, or result-collection phase. The cold-build contract gets its own test: cancelling while a build script stalls the compile reports Cancelled and reaps the whole cargo process tree. Refs #935 Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
|
…sor/separate-fixture-compile-from-runner-deadline-dccb
128fa8f deleted suspend_immutability_for_canonical_repair and restore_immutability_after_canonical_repair but left their #[hotpath::measure] attributes above the tests module. The macro is a passthrough with the hotpath feature off and demands an fn with it on, so the hotpath-on slice tests failed to compile. Refs #935 Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
…from-runner-deadline-dccb' into cursor/separate-fixture-compile-from-runner-deadline-dccb Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
…sor/separate-fixture-compile-from-runner-deadline-dccb
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Review — fixture phase split accepted; HOLD for conflicts and Windows execution
Reviewed 9f5c2cd48cc2d4f0ca1e3432455ceb840cb618b5; source review only. GitHub reports conflicts; exact-head CI is pending. Separate cold provisioning and exact-test execution is correct, and the production cargo_test_args behavior should remain unchanged.
Keep the 10-second execution contract and independently assert discovered test identities before the exactly-once/no-match cases. Provision with the same toolchain/profile/feature/target environment used by execution so the supposedly warm phase does not silently compile a different resolution. A warm execution timeout remains a real failure; never translate it to NoMatch.
P1 acceptance: the new compile-cancellation case asserts process-tree reap only on Unix. Add native Windows evidence that cargo and the stalled build-script child are both gone, file locks are released, and a successor build can run. Otherwise this PR repairs the selection tests while leaving the original platform's cancellation lifecycle unproven.
P2 integration: #953 removes the same orphaned hotpath attributes. Land that deletion once and preserve it while resolving both branches; no duplicated workaround or coverage skip. Retain at least one genuinely cold compile cancellation/timeout test so provisioning doesn't remove the runner's compile-phase contract from coverage.
Final gate: affected exact-runner tests plus their Windows shard and hotpath-on slice on the resolved #707 merge tree. #707 remains draft.
Performance Comparison
|
…ture-compile-from-runner-deadline-dccb
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
System-level re-review at 9f5c2cd48 — source change is appropriate; keep the two contracts separate
Read the complete remaining diff. No new production correctness defect found. cargo_test_build_args is a mechanical extraction and the test-execution arguments retain their existing exact-selection behavior. Provisioning and discovery run under a separate control/output budget, then the actual runner executes with the unchanged 10-second deadline. The new exact parsed-result assertions are stronger than merely checking that a success substring appears.
Keep the distinction explicit: these warm tests prove selection/result collection and vacuous-run refusal; they do not prove that a real cold user invocation can compile inside ten seconds. The separate stalled-build cancellation test preserves cold-phase cancellation coverage. In the production protocol, Timeout, Cancelled, a failing test, and NoMatch must remain different outcomes; no provisioning failure should be translated into a successful empty result.
P2 coverage: assert_reaped is still Unix-only. A Windows Cancelled result alone does not prove that cargo's build-script descendant was terminated. Verify process-tree retirement using an owned child/process handle or the existing Windows process owner, without PID-reuse assumptions. This is a missing platform proof, not a demonstrated Windows leak. Preserve the marker proving the build script actually entered its stalled phase before cancellation.
The two orphaned hotpath attributes should be removed once in the final integration tree; do not preserve a duplicate conflict just because another portability branch contained the same deletion. Avoid introducing production fixture ports or a broad runner redesign for this test split.
Returned CI is pending, profile succeeded and coverage/plugin runs queued. No repository tests were executed here. This source review supports the scoped change, not undrafting #707 or treating the historical CI matrix as current acceptance.
…ture-compile-from-runner-deadline-dccb
Final review-fix evidence —
|
…ture-compile-from-runner-deadline-dccb
…ture-compile-from-runner-deadline-dccb
…ture-compile-from-runner-deadline-dccb
…ture-compile-from-runner-deadline-dccb
…sor/separate-fixture-compile-from-runner-deadline-dccb
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
6a191cf
into
codex/tracedecay-total-redesign-plan-reopened
Summary
crates/tracedecay-mcp/src/workflow/test_runner.rstests. The three Windows exact-test runner cases (cargo_runner_executes_every_requested_exact_test_once,cargo_runner_reports_passing_and_failing_exact_tests,cargo_runner_rejects_a_vacuous_exact_test_run) now provision the fixture (compile + discovery) under a separate budget before running the phase that asserts against the genuine 10s production deadline.Cancelledand reaps the whole cargo process tree (reap assertion on unix).NoMatchand no production deadline is raised.hotpath-coverage / slice-tests (hotpath on)check: orphaned#[hotpath::measure]attributes left onmod testsincrates/tracedecay-global-db/src/schema_contract/invariants/triggers.rsafter128fa8fa2deleted the functions they annotated.Fixes #935.
Motivation
On Windows the exact-test runner cases hit their operation timeout while stderr still said
Compilinga tiny fixture, so the assertions about exactly-once selection, retained failing results, andNoMatchwere never actually reached. Cold compilation and the runner's execution contract were conflated. See #935 (filed against the PR #707 CI baseline).Changes
crates/tracedecay-mcp/src/workflow/test_runner.rscargo_test_argsinto a privatecargo_test_build_args(profile);cargo_test_argsoutput is byte-identical (covered by the existingcargo_test_args_*tests incrates/tracedecay).ProvisionedFixture/provision_fixturewrites the package, runs the compile phase (cargo test --no-fail-fast --no-run, i.e. the production flags plus--no-run) and the discovery phase (-- --list) with a 5-minute provisioning budget and a freshTestRunControlso provisioning output does not consume the execution phase's output budget. Recorded phase timings and the discovered test list are included in every assertion message so a failure is attributed to compile, discovery, execution, or result collection.ProvisionedFixture::executerunsrun_cargo_testsagainst the unchangedEXACT_TEST_EXECUTION_DEADLINE(10s); aTimeouton a warm fixture panics with the phase record and captured stderr instead of being masked.NoMatchin the vacuous case is proven to be about selection rather than a fixture that never contained the test.cancelling_a_stalled_fixture_compile_terminates_the_cargo_process_treeuses abuild.rsthat publishes its pid (write + rename) and sleeps; a canceller waits for the marker, thencontrol.cancel(). Asserts the compile actually reached the stall, the result isCancelled(notTimeout/NoMatch), and on unix that the build-script process is reaped.crates/tracedecay-global-db/src/schema_contract/invariants/triggers.rs#[hotpath::measure]attributes above#[cfg(test)] mod tests. With thehotpathfeature off the macro is a passthrough; with it on it demands anfn, so the hotpath-on slice tests failed withexpected fn.Test plan
cargo test -p tracedecay-mcp --lib workflow::test_runner— 10/10 pass, repeated 3x (Linux)cargo test -p tracedecay --lib cargo_test_args— 2/2cargo test -p tracedecay-rusqlite-runtime -p tracedecay-global-db --locked --features tracedecay-rusqlite-runtime/hotpath,tracedecay-global-db/hotpath,tracedecay-global-db/test-helpers— all greencargo clippy -p tracedecay-mcp --lib --tests— no warnings intest_runner.rs(the only warning in the build is a pre-existingGraphProjectionLabelPagedead-code warning intracedecay-graph-db)cargo fmt -p tracedecay-mcp -- --checknpm run lint:commitChecklist
CHANGELOG.mdupdated — test-only change, no entry.envfiles includedTargets
codex/tracedecay-total-redesign-plan-reopened(PR #707). #707 stays draft; this PR does not undraft or merge it.