test(ci): make source-provenance fixtures genuinely offline - #952
Conversation
Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
|
Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…sor/offline-source-provenance-9f7f
…sor/offline-source-provenance-9f7f
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Review — offline fixture approach accepted; HOLD for conflicts and archive-shard proof
Reviewed 7e892ed5e19e21eaea1332ff3503bbd167148dc7; reviewed the fixture harness, manifest/lock and vendoring boundary, not an independent line-by-line audit of upstream vendored crates. GitHub reports conflicts; exact-head CI pending.
The isolated CARGO_HOME/CARGO_TARGET_DIR, exact lockfile, --locked --offline preflight and -text vendor boundary address the actual archive-only failure. Keep this vendor closure private to the test fixture; do not change production dependency resolution or relax the provenance assertions.
P1 acceptance: run from a fresh Windows shard checkout with an empty Cargo home and the pinned toolchain already provisioned, with no network dependency. Delete one vendored dependency/checksum in a negative test and require a named dependency-preflight failure before provenance assertions. Confirm CRLF checkout cannot mutate the checksum-bound vendor bytes.
P2: cargo_config_directory interpolates an unescaped path into a TOML double-quoted string. Use a proper TOML string encoder/literal strategy for roots containing quotes/backslashes rather than relying on CI's ordinary path. Keep build outputs outside the temporary Git repository so the unchanged build remains clean and does not retrigger its own provenance script.
Retain package checksums/licenses and a reproducible pinned regeneration procedure. Resolve the current base conflicts, then verify all 15 provenance tests and the actual archive execution path. No new compatibility layer, ignored tests or #707 draft change is needed.
…ce-provenance-9f7f
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
System-level re-review at 7e892ed5e — offline dependency ownership is appropriate; encode the config path
Re-read the authored CargoFixture setup, preflight and nested build path. The fixture has its own Cargo home and target directory, pins the dependency closure and uses --locked --offline for fetch/build. That separates dependency provisioning failures from the provenance behavior under test instead of depending on a warm build-job registry cache. Keep vendored-byte preservation and checksums; they are part of reproducibility, not a substitute for reviewing the authored harness. I have not independently rehashed/audited every vendored source file in this pass.
P2 — interpolate a TOML string, not an unescaped path. The generated Cargo config uses directory = "{vendor_directory}"; cargo_config_directory only changes backslashes to slashes. A legal Unix checkout directory containing a double quote makes that TOML invalid before the provenance tests run, and a literal Unix backslash must not be respelled as a separator. Serialize the native path through a TOML-safe encoding and test spaces/non-ASCII plus quote/backslash characters where the host permits them. This is the same producer/consumer boundary principle as the other portability fixes, not a reason to add a universal path normalizer.
The generated test build.rs intentionally imports the actual source-provenance implementation rather than a mirrored implementation; preserve that. The noop-build and tracked/untracked-edit assertions must continue to check the build-script counter and correct source identity, not merely whether cargo exits successfully. Fixture-local Cargo settings are provisioning state, not input that should mark the fixture repository dirty.
For archive-only Windows acceptance, start without the original registry cache and make missing vendor material fail explicitly at preflight without a network fallback. The source checkout/shared build-support file must be available at the path the test embeds; do not claim the test archive is source-independent when the harness intentionally imports that source. This is different from the shipped packaged evaluator's no-checkout contract.
Returned CI is pending, profile succeeded, coverage failed and plugin validation queued. The coverage failure's cause was not inspected here and is not attributed to this patch. No nested Cargo builds executed in this review.
Final review-fix evidence —
|
…ce-provenance-9f7f
…ce-provenance-9f7f
Performance Comparison
|
…ce-provenance-9f7f
…ce-provenance-9f7f
…sor/offline-source-provenance-9f7f
cc4b4e9
into
codex/tracedecay-total-redesign-plan-reopened
Summary
serde_jsondependency closure--locked --offlineMotivation
Archive-only Windows nextest shards do not inherit the build job's Cargo registry cache. The source-provenance tests therefore failed while resolving
serde_json, before reaching their provenance assertions.Fixes #937.
Changes
crates/tracedecay-cli/tests/fixtures/source-provenance-cargo/with the pinned manifest, lockfile, and vendored dependency closureCargoFixtureto copy only source inputs into its temporary Git repositoryCARGO_HOMEandCARGO_TARGET_DIR, configured vendored sources cross-platform, and retained offline-only nested buildsTest plan
cargo test -p tracedecay-cli --test source_provenance_testpasses (15 tests)-D warningspassescargo nextest run --workspace --no-fail-fastpassesserde_jsonresolution failure with an empty Cargo home before adding vendored sourcesChecklist
CHANGELOG.mdupdated (not applicable: test fixture provisioning only).envfiles included