Skip to content

test(cli): provision compiled host-CLI fixtures - #954

Merged
ScriptedAlchemy merged 10 commits into
codex/tracedecay-total-redesign-plan-reopenedfrom
cursor/host-cli-fixtures-42d7
Sep 7, 2026
Merged

test(cli): provision compiled host-CLI fixtures#954
ScriptedAlchemy merged 10 commits into
codex/tracedecay-total-redesign-plan-reopenedfrom
cursor/host-cli-fixtures-42d7

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • Provision the deterministic Kiro/Codex fake as a dedicated Cargo example built only by test workflows; ordinary product --bin / --bins builds no longer compile it in build.rs.
  • Resolve the fixture from the executing test's remapped target profile and include the Windows .exe explicitly in the nextest archive contract.
  • Record each invocation as a JSON argv array, preserving arguments that contain spaces, while retaining separate missing-CLI, malformed-output, and ownership-conflict outcomes.
  • Keep real vendor conformance outside the deterministic fixture protocol suite.

Fixes #936.

Implementation

  • tracedecay-host-cli-fixture is a [[example]]; Linux/macOS and Windows test lanes build it explicitly with Cargo, so target linker configuration remains Cargo-owned.
  • .config/nextest.toml archives target/debug/examples/tracedecay-host-cli-fixture.exe for clean Windows shards.
  • Test provisioning derives target/<profile>/examples/... from current_exe() instead of embedding a build-machine OUT_DIR.
  • Unix tests symlink the prebuilt executable onto isolated PATHs, avoiding the ETXTBSY fork/write-descriptor race; Windows copies the native .exe.

Evidence

  • Review-critical Kiro/Codex fixture and lifecycle selection passes, including missing executable, malformed output, ownership conflict, Codex automation, rollback, and retry.
  • Agent-host package: 637/637 passed.
  • Nextest archive -> fresh extraction -> separate checkout remap: 5/5 fixture protocol tests passed; nextest reported one archived non-test binary.
  • cargo clippy -p tracedecay-cli -p tracedecay-agent-hosts --all-targets --locked -- -D warnings: clean on the latest merged feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707 tip.
  • The std-only fixture cross-builds to a PE32+ x86-64 Windows executable locally; native Windows archive/shard checks run on the pushed exact head.

The complete local CLI package run executed 570 tests: 569 passed and the unchanged branch_list_reads_profile_sharded_branch_meta subprocess hit its existing 90-second local deadline. Focused host-CLI acceptance passed again after each latest-#707 merge; per-finding and exact-head receipts are posted below.

Compile a real kiro-cli/codex helper in build.rs so Windows tests no
longer rename a shell script to .exe or depend on ambient installs.

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
@changeset-bot

changeset-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 78cf578

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — REVISE fixture provisioning; keep the native fake CLI

Reviewed 9d51efd120480029b939cd5559c2eb2f548129ae; source review only. GitHub reports conflicts and exact-head CI pending. A compiled deterministic fake is correct for these tests; requiring ambient Kiro/Codex or renaming a shell script to .exe is not.

P1 build regression: tracedecay-cli/build.rs::main unconditionally compiles this test-only executable for every product build. The direct rustc call adds --target but does not inherit Cargo's target linker/configuration, and uses the build-host EXE_SUFFIX. That makes normal release/cross-target builds depend on linking a fixture they never use. Provision it through an explicitly enabled test-support target/feature or a dedicated workflow helper build; don't use cfg(test) in build.rs as a substitute for explicit feature selection. Keep non-test builds free of this work.

P1 distribution acceptance: prove the fixture survives nextest archive -> clean Windows shard extraction/remap and is executable at the path the tests resolve. An OUT_DIR absolute path baked into a binary is not, by itself, evidence that the helper is shipped to the executing machine. Include the helper in the selected test-artifact contract and test without access to the build machine's tree.

Preserve missing-executable, malformed-output and ownership-conflict as different outcomes. Record argv as structured entries rather than a joined string if the assertion is meant to distinguish arguments containing spaces. Keep vendor conformance separate from the fake's protocol tests. Resolve conflicts and run native Windows host-fixture plus Kiro/Codex acceptance before integration into #707.

…plan-reopened' into cursor/host-cli-fixtures-42d7

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System-level re-review at 9d51efd12 — keep test provisioning out of the product build contract

Re-read build.rs, compile_host_cli_fixture.rs and the shared provisioner. The earlier build-boundary objection is still present: build.rs invokes the fixture compiler unconditionally, including non-test builds/checks. The native executable itself is a sound replacement for a shell script named .exe; the ownership of its build/distribution is the problem.

P1 — the raw compiler invocation bypasses Cargo's target toolchain configuration. It forwards --target but not the target linker or applicable Cargo rustflags. A cross-target cargo check that previously needed only target metadata now attempts to link an unrelated test executable, and a product cross-build using a configured linker may fail because this subprocess does not inherit that configuration. The destination suffix is also taken from the build-script host rather than the target. Move this work to an explicit test-fixture provisioning step/artifact, or a test-only build contract that correctly follows the target toolchain. Do not gate it by debug/release profile: release tests and debug product builds are both valid.

P2 — verify the artifact, not only the installed test-local copy. provision_host_cli_fixture.rs obtains the source through compile-time TRACEDECAY_HOST_CLI_FIXTURE, then hard-links/copies it. It has no fallback if the build job's OUT_DIR file is absent. The Windows archive-only lane must explicitly transport that helper or reproduce it in test provisioning before tests run. I have not inspected a produced archive, so I am not claiming the helper is definitely missing; the current source establishes that it is a required external test artifact. Exercise a clean extraction without the original target directory and assert the helper runs, not merely that its bytes have an executable header.

Keep install/list/conflict/malformed/missing-program cases separate. A deterministic fake proves TraceDecay's invocation and parsing contract, not compatibility with the real vendor CLI; do not label it a vendor end-to-end acceptance lane. Preserve exact argument recording and isolated HOME/PATH.

No native compilation or archive execution performed here. Returned CI is pending, coverage in progress, profile succeeded and plugin validation queued. #707 remains draft.

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Addressed the latest review on exact head b542c6ee7.

P1 — production/cross-target build regression

  • Removed the fixture compiler and all TRACEDECAY_HOST_CLI_FIXTURE / OUT_DIR plumbing from build.rs.
  • Declared the fake as a Cargo [[example]], so ordinary product --bin / --bins builds exclude it.
  • Added explicit Linux/macOS and native Windows test-workflow builds in ci.yml. Cargo now owns linker/target configuration.
  • Local Windows-source build evidence: rustc --target x86_64-pc-windows-gnu ... host_cli_fixture.rs produced PE32+ executable (console) x86-64, for MS Windows. The repository's full Windows MSVC dependency graph cannot be cross-checked on this Linux host because ring requires Windows SDK headers; the pushed workflow performs the native MSVC build.

P1 — nextest archive distribution

  • Added the exact Windows helper to profile.ci.archive.include.
  • compiled_host_cli_fixture derives the executable from the running test's target/<profile>/examples directory; no build-machine absolute path is embedded.
  • Archive simulation built a nextest archive, extracted it to /tmp/tracedecay-host-cli-extract-954-post-merge, remapped source metadata to a separate clean checkout, and ran the fixture target. Nextest reported Extracting 2 binaries (including 1 non-test binary) and all 5/5 tests passed from the extracted tree, including the assertion that the resolved helper is beside that test profile.

Preserved contracts

  • 15/15 focused Kiro/Codex lifecycle tests passed after the latest feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707 merge: missing executable, malformed output, ownership conflict, native format, Codex automation, rollback, and retry all remain distinct.
  • Invocation logs are JSON arrays; the protocol test passes /usr/local/bin/Trace Decay/tracedecay and verifies it remains one argv entry.
  • Deterministic fake protocol tests remain in host_cli_fixture; no real-vendor conformance was folded into them.
  • 637/637 tracedecay-agent-hosts tests passed.
  • Strict clippy passed with 0 warnings: cargo clippy -p tracedecay-cli -p tracedecay-agent-hosts --all-targets --locked -- -D warnings.

The complete local CLI run executed 570 tests: 569 passed; the unchanged branch_list_reads_profile_sharded_branch_meta subprocess hit its existing 90-second local deadline. The 15 host-CLI acceptance tests passed again on the merged tip.

…plan-reopened' into cursor/host-cli-fixtures-42d7
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Performance Comparison codex/tracedecay-total-redesign-plan-reopenedcursor/host-cli-fixtures-42d7

Total Elapsed Time: 4.54s → 4.98s (+9.8%)
CPU Baseline: 77.80µs → 77.28µs (-0.7%)
Benchmark ID: index-bench-timing

timing - Execution duration of functions.

+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| Function                                 | Calls                      | Avg                              | P95                              | Total                            | % Total                      |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| tracedecay-index-bench                   | 1 → 1 (+0.0%)              | 4.54s → 4.98s (+9.7%)            | 4.54s → 4.99s (+9.9%)            | 4.54s → 4.98s (+9.7%)            | 100.00% → 100.00% (+0.0%)    |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.append_pages              | 13 → 13 (+0.0%)            | 167.31ms → 192.83ms (+15.3%)     | 230.56ms → 326.11ms (+41.4%) ⚠️  | 2.18s → 2.51s (+15.1%)           | 47.91% → 50.31% (+5.0%)      |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.sqlite              | 13 → 13 (+0.0%)            | 113.51ms → 144.61ms (+27.4%) ⚠️  | 151.78ms → 273.94ms (+80.5%) ⚠️  | 1.48s → 1.88s (+27.0%) ⚠️        | 32.50% → 37.73% (+16.1%)     |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| code_index.workers.install               | 83 → 83 (+0.0%)            | 15.88ms → 12.08ms (-23.9%) 🚀    | 53.74ms → 42.17ms (-21.5%) 🚀    | 1.32s → 1.00s (-24.2%) 🚀        | 29.03% → 20.12% (-30.7%) 🚀  |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.postings            | 13 → 13 (+0.0%)            | 76.40ms → 76.40ms (+0.0%)        | 103.74ms → 104.14ms (+0.4%)      | 993.20ms → 993.20ms (+0.0%)      | 21.88% → 19.93% (-8.9%)      |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.commit              | 13 → 13 (+0.0%)            | 34.34ms → 65.46ms (+90.6%) ⚠️    | 45.09ms → 198.71ms (+340.7%) ⚠️  | 446.40ms → 851.01ms (+90.6%) ⚠️  | 9.83% → 17.08% (+73.8%) ⚠️   |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| domain.canonical.sha256                  | 114526 → 114526 (+0.0%)    | 8.99µs → 7.18µs (-20.1%) 🚀      | 12.04µs → 12.09µs (+0.4%)        | 1.03s → 822.10ms (-20.2%) 🚀     | 22.67% → 16.50% (-27.2%) 🚀  |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| code_index.build.and_publish             | 2 → 2 (+0.0%)              | 504.48ms → 404.40ms (-19.8%)     | 528.22ms → 411.04ms (-22.2%) 🚀  | 1.01s → 808.81ms (-19.9%)        | 22.22% → 16.23% (-27.0%) 🚀  |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.finalization.advance_wake | 14 → 14 (+0.0%)            | 34.40ms → 46.28ms (+34.5%) ⚠️    | 215.74ms → 217.32ms (+0.7%)      | 481.61ms → 647.95ms (+34.5%) ⚠️  | 10.61% → 13.00% (+22.5%) ⚠️  |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| 🆕 query.artifact.create                 | 0 → 2 (+100.0%) ⚠️         | 0.00ns → 282.22ms (+100.0%) ⚠️   | 0.00ns → 282.33ms (+100.0%) ⚠️   | 0.00ns → 564.45ms (+100.0%) ⚠️   | 0.00% → 11.33% (+100.0%) ⚠️  |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.prepare_pages             | 13 → 13 (+0.0%)            | 43.70ms → 38.35ms (-12.2%)       | 69.99ms → 54.23ms (-22.5%) 🚀    | 568.13ms → 498.56ms (-12.2%)     | 12.51% → 10.01% (-20.0%)     |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| code_index.chunk.index_file              | 276 → 276 (+0.0%)          | 2.35ms → 1.80ms (-23.4%) 🚀      | 5.26ms → 2.61ms (-50.4%) 🚀      | 648.90ms → 497.93ms (-23.3%) 🚀  | 14.29% → 9.99% (-30.1%) 🚀   |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| code_index.extract.parser_artifact       | 276 → 276 (+0.0%)          | 2.27ms → 1.65ms (-27.3%) 🚀      | 5.28ms → 2.48ms (-53.0%) 🚀      | 626.83ms → 455.98ms (-27.3%) 🚀  | 13.81% → 9.15% (-33.7%) 🚀   |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.parallel_prepare    | 13 → 13 (+0.0%)            | 35.29ms → 30.47ms (-13.7%)       | 59.34ms → 42.89ms (-27.7%) 🚀    | 458.83ms → 396.08ms (-13.7%)     | 10.11% → 7.95% (-21.4%) 🚀   |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| code_index.chunk.build                   | 276 → 276 (+0.0%)          | 1.40ms → 1.09ms (-22.1%) 🚀      | 4.15ms → 1.59ms (-61.7%) 🚀      | 386.19ms → 299.84ms (-22.4%) 🚀  | 8.51% → 6.02% (-29.3%) 🚀    |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+
| 🗑️ code_index.chunk.identify             | 276 → 0 (-100.0%) 🚀       | 1.36ms → 0.00ns (-100.0%) 🚀     | 4.13ms → 0.00ns (-100.0%) 🚀     | 376.20ms → 0.00ns (-100.0%) 🚀   | 8.29% → 0.00% (-100.0%) 🚀   |
+------------------------------------------+----------------------------+----------------------------------+----------------------------------+----------------------------------+------------------------------+

Generated with hotpath-rs

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Latest-base follow-up: #707 advanced after the first evidence post, so I merged its new tip abf7f47a8 and pushed exact head edb7c3b40.

Fresh evidence on edb7c3b40:

The intervening #707 delta touched search-eval/configuration files only; it did not alter the fixture, archive contract, CLI host lifecycle, or agent-host code. Exact-head GitHub checks have restarted, including the native Windows archive/shards.

…plan-reopened' into cursor/host-cli-fixtures-42d7
…plan-reopened' into cursor/host-cli-fixtures-42d7
…plan-reopened' into cursor/host-cli-fixtures-42d7
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Final base convergence: pushed 78cf57842, which contains current #707 tip abf58b20e.

  • Remote comparison: 0 behind / 10 ahead.
  • GitHub mergeability: MERGEABLE.
  • The final four base commits touched graph-db/session Git-correlation code only; none touched the fixture, CLI host lifecycle, agent-hosts, archive config, or CI workflow. The immediately preceding integrated tip passed 11/11 review-critical tests and strict CLI/agent-host clippy.
  • Exact-head checks restarted and remain queued/pending. The Actions job watcher hit the shared GitHub REST rate limit before native Windows jobs began; it did not report a CI test failure.

@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review September 7, 2026 06:49
@ScriptedAlchemy
ScriptedAlchemy merged commit a8c322a into codex/tracedecay-total-redesign-plan-reopened Sep 7, 2026
1 of 10 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-07T06:51:29.309559Z 78cf578 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants