Skip to content

test(ci): make source-provenance fixtures genuinely offline - #952

Merged
ScriptedAlchemy merged 12 commits into
codex/tracedecay-total-redesign-plan-reopenedfrom
cursor/offline-source-provenance-9f7f
Sep 7, 2026
Merged

test(ci): make source-provenance fixtures genuinely offline#952
ScriptedAlchemy merged 12 commits into
codex/tracedecay-total-redesign-plan-reopenedfrom
cursor/offline-source-provenance-9f7f

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • pin the source-provenance Cargo fixture with a checked-in lockfile
  • vendor only the fixture's serde_json dependency closure
  • force nested Cargo commands through an empty fixture-local Cargo home with --locked --offline
  • preserve vendored bytes on Windows and report missing material during an explicit preflight

Motivation

Archive-only Windows nextest shards do not inherit the build job's Cargo registry cache. The source-provenance tests therefore failed while resolving serde_json, before reaching their provenance assertions.

Fixes #937.

Changes

  • added crates/tracedecay-cli/tests/fixtures/source-provenance-cargo/ with the pinned manifest, lockfile, and vendored dependency closure
  • updated CargoFixture to copy only source inputs into its temporary Git repository
  • isolated CARGO_HOME and CARGO_TARGET_DIR, configured vendored sources cross-platform, and retained offline-only nested builds
  • disabled Git text conversion for vendored bytes so Cargo checksums remain valid on Windows

Test plan

  • cargo test -p tracedecay-cli --test source_provenance_test passes (15 tests)
  • focused Clippy with -D warnings passes
  • vendored package checksums and license presence validated
  • cargo nextest run --workspace --no-fail-fast passes
  • Reproduced the original serde_json resolution failure with an empty Cargo home before adding vendored sources

Checklist

  • CHANGELOG.md updated (not applicable: test fixture provisioning only)
  • No secrets, credentials, or .env files included
  • Breaking changes documented (not applicable)
Open in Web Open in Cursor 

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
@changeset-bot

changeset-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 694ed74

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review September 6, 2026 20:54
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — offline fixture approach accepted; HOLD for conflicts and archive-shard proof

Reviewed 7e892ed5e19e21eaea1332ff3503bbd167148dc7; reviewed the fixture harness, manifest/lock and vendoring boundary, not an independent line-by-line audit of upstream vendored crates. GitHub reports conflicts; exact-head CI pending.

The isolated CARGO_HOME/CARGO_TARGET_DIR, exact lockfile, --locked --offline preflight and -text vendor boundary address the actual archive-only failure. Keep this vendor closure private to the test fixture; do not change production dependency resolution or relax the provenance assertions.

P1 acceptance: run from a fresh Windows shard checkout with an empty Cargo home and the pinned toolchain already provisioned, with no network dependency. Delete one vendored dependency/checksum in a negative test and require a named dependency-preflight failure before provenance assertions. Confirm CRLF checkout cannot mutate the checksum-bound vendor bytes.

P2: cargo_config_directory interpolates an unescaped path into a TOML double-quoted string. Use a proper TOML string encoder/literal strategy for roots containing quotes/backslashes rather than relying on CI's ordinary path. Keep build outputs outside the temporary Git repository so the unchanged build remains clean and does not retrigger its own provenance script.

Retain package checksums/licenses and a reproducible pinned regeneration procedure. Resolve the current base conflicts, then verify all 15 provenance tests and the actual archive execution path. No new compatibility layer, ignored tests or #707 draft change is needed.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System-level re-review at 7e892ed5e — offline dependency ownership is appropriate; encode the config path

Re-read the authored CargoFixture setup, preflight and nested build path. The fixture has its own Cargo home and target directory, pins the dependency closure and uses --locked --offline for fetch/build. That separates dependency provisioning failures from the provenance behavior under test instead of depending on a warm build-job registry cache. Keep vendored-byte preservation and checksums; they are part of reproducibility, not a substitute for reviewing the authored harness. I have not independently rehashed/audited every vendored source file in this pass.

P2 — interpolate a TOML string, not an unescaped path. The generated Cargo config uses directory = "{vendor_directory}"; cargo_config_directory only changes backslashes to slashes. A legal Unix checkout directory containing a double quote makes that TOML invalid before the provenance tests run, and a literal Unix backslash must not be respelled as a separator. Serialize the native path through a TOML-safe encoding and test spaces/non-ASCII plus quote/backslash characters where the host permits them. This is the same producer/consumer boundary principle as the other portability fixes, not a reason to add a universal path normalizer.

The generated test build.rs intentionally imports the actual source-provenance implementation rather than a mirrored implementation; preserve that. The noop-build and tracked/untracked-edit assertions must continue to check the build-script counter and correct source identity, not merely whether cargo exits successfully. Fixture-local Cargo settings are provisioning state, not input that should mark the fixture repository dirty.

For archive-only Windows acceptance, start without the original registry cache and make missing vendor material fail explicitly at preflight without a network fallback. The source checkout/shared build-support file must be available at the path the test embeds; do not claim the test archive is source-independent when the harness intentionally imports that source. This is different from the shipped packaged evaluator's no-checkout contract.

Returned CI is pending, profile succeeded, coverage failed and plugin validation queued. The coverage failure's cause was not inspected here and is not attributed to this patch. No nested Cargo builds executed in this review.

@ScriptedAlchemy

ScriptedAlchemy commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Final review-fix evidence — e2bea65ec

  • Merged current feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707 head 3af6706dc; GitHub reports base 3af6706dc, MERGEABLE, with no conflict/behind delta. The orphaned Hotpath deletion comes only from feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707.
  • Removed the ambient Windows registry seed. Nested Cargo uses a fresh isolated CARGO_HOME, out-of-repository target dir, --locked, and --offline; Cargo config paths are TOML-encoded, and archive relocation uses runtime CARGO_MANIFEST_DIR.
  • Missing serde_json/.cargo-checksum.json now proves a named dependency-preflight failure before Git/provenance setup.
  • Latest owned-file validation: 17/17 provenance tests; former failing Hotpath slice command passed completely after the latest-base telemetry correction; targeted clippy 0 diagnostics; actionlint, hosted-runner tests (7/7), hosted-runner audit, rustfmt, and diff checks passed.
  • Archive simulation extracted 305 files into a separate core.autocrlf=true checkout and ran 17/17 with empty CARGO_HOME plus CARGO_NET_OFFLINE=true. All 475 vendor files remained byte-identical (sha256:af67a8a3cdab044722aebd556e0978a894534db51f7d6396668d8b665eb6807d); cargo vendor --locked reproduced that exact tree; all 11 lock packages retain checksums/licenses; an isolated x86_64-pc-windows-gnu offline fixture cross-check passed. Subsequent feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707 merges did not change those archive-proven fixture bytes.

Native Windows CI is queued at run 34106362273. GitHub has not allocated a runner despite multiple hours of waiting, so no native shard receipt exists yet; this comment deliberately does not claim one.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Performance Comparison codex/tracedecay-total-redesign-plan-reopenedcursor/offline-source-provenance-9f7f

Total Elapsed Time: 5.48s → 5.14s (-6.2%)
CPU Baseline: 81.97µs → 92.57µs (+12.9%)
Benchmark ID: index-bench-timing

timing - Execution duration of functions.

+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| Function                                 | Calls                      | Avg                             | P95                             | Total                            | % Total                      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| tracedecay-index-bench                   | 1 → 1 (+0.0%)              | 5.48s → 5.14s (-6.2%)           | 5.49s → 5.15s (-6.2%)           | 5.48s → 5.14s (-6.2%)            | 100.00% → 100.00% (+0.0%)    |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.append_pages              | 13 → 13 (+0.0%)            | 204.07ms → 189.79ms (-7.0%)     | 279.71ms → 258.87ms (-7.5%)     | 2.65s → 2.47s (-6.8%)            | 48.39% → 47.96% (-0.9%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.sqlite              | 13 → 13 (+0.0%)            | 130.31ms → 129.37ms (-0.7%)     | 175.64ms → 178.26ms (+1.5%)     | 1.69s → 1.68s (-0.6%)            | 30.90% → 32.69% (+5.8%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.postings            | 13 → 13 (+0.0%)            | 115.99ms → 114.49ms (-1.3%)     | 158.33ms → 158.07ms (-0.2%)     | 1.51s → 1.49s (-1.3%)            | 27.51% → 28.93% (+5.2%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.workers.install               | 83 → 83 (+0.0%)            | 18.44ms → 15.26ms (-17.2%)      | 68.55ms → 52.00ms (-24.1%) 🚀   | 1.53s → 1.27s (-17.0%)           | 27.93% → 24.62% (-11.9%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.build.and_publish             | 2 → 2 (+0.0%)              | 552.86ms → 522.52ms (-5.5%)     | 570.43ms → 532.15ms (-6.7%)     | 1.11s → 1.05s (-5.4%)            | 20.17% → 20.31% (+0.7%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| domain.canonical.sha256                  | 114526 → 114526 (+0.0%)    | 9.16µs → 8.78µs (-4.1%)         | 14.90µs → 14.84µs (-0.4%)       | 1.05s → 1.01s (-3.8%)            | 19.13% → 19.55% (+2.2%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.finalization.advance_wake | 14 → 14 (+0.0%)            | 44.67ms → 45.09ms (+0.9%)       | 284.43ms → 307.49ms (+8.1%)     | 625.33ms → 631.29ms (+1.0%)      | 11.41% → 12.27% (+7.5%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.prepare_pages             | 13 → 13 (+0.0%)            | 59.53ms → 47.18ms (-20.7%) 🚀   | 86.25ms → 67.76ms (-21.4%) 🚀   | 773.86ms → 613.38ms (-20.7%) 🚀  | 14.12% → 11.92% (-15.6%)     |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.chunk.index_file              | 276 → 276 (+0.0%)          | 2.29ms → 2.18ms (-4.8%)         | 3.24ms → 3.20ms (-1.2%)         | 630.82ms → 600.36ms (-4.8%)      | 11.51% → 11.67% (+1.4%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.extract.parser_artifact       | 276 → 276 (+0.0%)          | 2.10ms → 2.11ms (+0.5%)         | 3.19ms → 3.16ms (-0.9%)         | 578.55ms → 583.66ms (+0.9%)      | 10.55% → 11.35% (+7.6%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| query.artifact.batch.parallel_prepare    | 13 → 13 (+0.0%)            | 48.84ms → 36.78ms (-24.7%) 🚀   | 73.20ms → 52.72ms (-28.0%) 🚀   | 634.96ms → 478.10ms (-24.7%) 🚀  | 11.58% → 9.29% (-19.8%)      |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.build.assemble                | 2 → 2 (+0.0%)              | 203.67ms → 191.82ms (-5.8%)     | 225.05ms → 204.21ms (-9.3%)     | 407.35ms → 383.65ms (-5.8%)      | 7.43% → 7.46% (+0.4%)        |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| code_index.chunk.build                   | 276 → 276 (+0.0%)          | 1.35ms → 1.36ms (+0.7%)         | 1.97ms → 2.00ms (+1.5%)         | 373.48ms → 375.60ms (+0.6%)      | 6.81% → 7.30% (+7.2%)        |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| 🆕 code_index.chunk.identify             | 0 → 276 (+100.0%) ⚠️       | 0.00ns → 1.35ms (+100.0%) ⚠️    | 0.00ns → 1.99ms (+100.0%) ⚠️    | 0.00ns → 371.23ms (+100.0%) ⚠️   | 0.00% → 7.22% (+100.0%) ⚠️   |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+
| 🗑️ code_index.chunk.map_ordered          | 780 → 0 (-100.0%) 🚀       | 476.78µs → 0.00ns (-100.0%) 🚀  | 594.94µs → 0.00ns (-100.0%) 🚀  | 371.89ms → 0.00ns (-100.0%) 🚀   | 6.78% → 0.00% (-100.0%) 🚀   |
+------------------------------------------+----------------------------+---------------------------------+---------------------------------+----------------------------------+------------------------------+

Generated with hotpath-rs

@ScriptedAlchemy
ScriptedAlchemy merged commit cc4b4e9 into codex/tracedecay-total-redesign-plan-reopened Sep 7, 2026
1 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants