[Daniel Meppiel (danielmeppiel)]

fix(tests): restore canonical fixture contracts for CI

Description

TL;DR

Repair the 39 bounded fixture/oracle cases in #3335 without changing production code.
Downloader fixtures materialize valid per-package trees, MCP cleanup fixtures obtain real managed-write evidence, and assertions follow current preservation, policy, diagnostic and identity contracts.
The recovery at 125d7c4ba fixes two review-identified Windows newline assumptions with exact LF/CRLF byte controls; historical full affected-file proof remains bound to 6ec1665bf.

Important

This is the sole corrective carrier for #3335, not a claim that all CI is green.
The named-package case qualifies controlled generic Git, not GitHub REST.
Only the designated merge manager qualifies or merges this PR; new-head hosted checks are tracked separately.

Problem (WHY)

• The original 38-case baseline on public 3b9a8503 produced 37 failures and one pass; invalid mock paths and unverified native cleanup prevented meaningful behavioral assertions.
• Several oracles expected obsolete diagnostics, identical cold/warm provenance, or implicit overwrite of deliberately edited skills.
• The subsequently assigned identity case omitted the exact orphan finding produced by mismatched canonical-owner replay.
• Review of 6ec1665bf found two real newline assumptions: translated tamper writes and an LF-only expectation for copied source bytes. These are static causal findings, not reproduced hosted Windows failures.

The repair uses execution evidence instead of accepting mock-error exits as success, following Agent Skills' guidance: "Read agent execution traces, not just final outputs."

Approach (WHAT)

• Use actual PackageInfo/ResolvedReference models and physical sources at the requested install path, including staging.
• Generate managed MCP declarations and native receipts through the unchanged canonical writer; refuse names-only and foreign removal with unchanged bytes.
• Prove ordinary preservation before explicit existing --force repair; retain unforced clean/missing-file behavior.
• Assert exact policy provenance and modes, then compare every remaining payload field.
• Exercise real virtual-file construction, frozen baselines and named Git preflight rather than obsolete mock seams.
• Require the exact identity failed-check set and singleton structured finding; deliberately wrong expected paths must fail.
• Preserve explicit LF/CRLF tamper bytes and compare promoted bytes with the source snapshot taken before promotion.

Implementation (HOW)

Paths below are relative to tests/integration/, except explicitly qualified helper/documentation paths.

File(s)	Correction
tests/utils/materialized_downloader.py	Shared real-model, package-specific materialization at the actual requested destination.
test_selective_install_mcp.py	Preserve selective/deep/diamond graph checks; reject absent/MagicMock sources before MCP effects, then restore the valid graph.
test_global_mcp_lockfile_e2e.py	Genuine sources with global/project lock placement and clearing.
test_update_mcp_lsp_prune.py	Old/new commits, MCP pruning, LSP recording and dry-run protection.
test_policy_install_e2e.py	Successful warn-mode installation of all three blocked packages; I5 untouched.
tests/utils/managed_mcp.py	Actual inert native writes with declaration/hash evidence, including explicit user scope.
test_codex_windows_toml_install.py	Authorized cleanup plus unverifiable/foreign refusal; preserve literal Windows path content.
test_integrators_end_to_end.py	Producer-backed Claude cleanup, refusal bytes and exact survivor.
test_mcp_integrator_characterisation.py	Real project/user Claude and Codex clients, genuine ownership, exact survivors and refusal bytes.
test_mcp_integrator_install_flow.py, test_wave6_integrators_coverage.py	Replace names-only Claude cleanup setup with actual writer evidence.
test_commands_mcp_flow.py, test_integrators_validation_rules.py	Exact canonical local-source diagnostics and genuine valid-manifest acceptance.
test_grok_audit_target_lifecycle.py	Ordinary dirty-file/receipt preservation before explicit repair; recovery uses explicit LF/CRLF bytes across all four targets.
test_govern_core_contract.py	Exact cold file: versus warm org: source, enforcement state/mode and complete remaining semantic equality.
test_github_downloader_phase3w4.py, test_github_downloader_validation_flow.py	Real virtual materialization and persisted metadata/bytes instead of a copied parser.
test_deps_models_validation.py	Existing project: no-work installs create no lock; explicit lockfile-only creates an empty lock.
test_frozen_cold_cache_apm_package_e2e.py	Supported dependencies.mcp, genuine initial install, cold replay and declaration-mismatch rejection.
test_frozen_native_service_lifecycle.py	Hermetic generic-Git route; real Trace2 ls-remote, original commits and native/frozen controls.
test_skill_integrator_hermetic.py	Actual no-logger console fallback; recovery compares native/LF/CRLF promoted bytes to pre-promotion source bytes and verifies source unchanged.
test_hermetic_lifecycle_foundation.py	Only source-identity oracle changed; four other mutations and shared recovery checks retained.
docs/src/content/docs/contributing/integration-testing.md	Separate source/ownership-fixture guidance bullet; no README or generated-model bullet changes.

Recovery details: explicit tamper bytes, source-byte promotion oracle.

Diagram

Dashed nodes are new fixture helpers; production admission and ownership decisions remain unchanged.

flowchart LR
    subgraph Fixtures["Corrected test inputs"]
        S["stub_materialized_downloader"]
        M["install_managed_mcp"]
    end
    subgraph Authority["Unchanged canonical owners"]
        D["Source admission"]
        W["MCPIntegrator._install_for_runtime"]
        R["MCPIntegrator.remove_stale"]
    end
    subgraph Oracles["Load-bearing assertions"]
        G["Transitive graph and exact lock"]
        P["Foreign or unverifiable bytes preserved"]
        C["Authorized stale entry removed"]
    end
    S -->|"PackageInfo and physical staging tree"| D
    D --> G
    M --> W
    W -->|"Declaration and actual native receipt"| R
    R --> P
    R --> C
    classDef new stroke-dasharray: 5 5;
    class S,M new;

Loading

Trade-offs

• Reuse real canonical producers instead of hand-authored ownership hashes; these are hermetic fixtures, not native-client execution.
• Use generic Git for the native-service retention scenario. The selected-package test already passed locally; the original hosted REST failure is not claimed repaired or covered as REST.
• Preserve every other policy field and exact file bytes rather than dropping payloads or normalizing newlines to make comparisons pass.
• Keep fix(ci): align generated skill repair with explicit overwrite consent #3333/fix(ci): use complete skill-result contracts in orchestration fixtures #3334 changes only as already-public main composition. [BUG] apm install records a stale dependency version in apm.lock.yaml when apm_modules/ holds the previous version #3246, I5, production, security, workflows and releases remain outside this correction. No CHANGELOG entry for test/guidance-only changes.

Benefits

1. Original 39-ID accounting remains intact, with explicit renames and five originals expanded into 11 newline variants.
2. Historical 20-module proof remains available; the recovery independently passes both complete changed modules (60 cases).
3. Invalid-source, unauthorized-cleanup, wrong-identity-path and newline regressions have load-bearing refusal or exact-byte controls.

Issue and approved scope

Issue: Fixes #3335.

Human scope-approval comment: existing direct maintainer CI execution direction, recorded in #3335 under Execution authority and ownership: "I do not agree that you park the fix to CI, move full steam ahead, being idle and waiting for me to agree is not acceptable. Your mission is to fix CI". This is not a new approval record or accepted label. Review contact: Daniel Meppiel (@danielmeppiel).

Bounded scope is original 38 plus the explicitly added identity case. Two neighboring MCP characterisation fixtures were corrected under the existing group's all-affected-file requirement, not counted as new original hosted IDs. Recovery changes only the two reviewed fixture paths. Designated merge-manager qualification and actual new-head gates remain separate from local completion and broader main/release recovery.

Type of change

• Bug fix
• New feature
• Documentation
• Maintenance / refactor

Testing

• Tested locally
• All existing tests pass
• Added tests for new functionality (if applicable)

Validation

Recovery head: 125d7c4ba65f20b552b0b7a4de3e384d70ba6d4a, composed with actual public main 4adaa21f5a369298e316356891d2c65181f02560 using both rerere flags disabled. Parent, isolated Python child and actual installed CLI are bound to this owned Python 3.12 checkout.

Evidence	Result and binding
Original 38 baseline on 3b9a8503	37 failed, 1 passed, no skips; retained separately.
First affected-file attempt	1016 passed, 8 failed; six local wiring corrections and two neighboring fixtures subsequently addressed.
Historical 6ec1665bf	1031 passed across 20 modules; 44 passed assigned/control cases; 68 passed quality. Not reattributed to recovery head.
Historical identity negative	Wrong expected orphan path failed exact singleton assertion; restored, all five mutations passed.
Recovery 125d7c4ba	60 passed across both complete changed modules, 104.35s; 68 passed quality, 78.60s; no failures/errors/skips.
Recovery negatives/restoration	Reintroduced translated tamper write and LF-only promotion assertion: 2 expected byte-equality failures; restored before commit and 60-case pass.
Canonical owner proof	Full actual completion schema validated, then strict verified=true, terminal_evidence_required=true; zero touched production owners, functional_tests=[]. Regression evidence is separate.
Independent review	Prior 23-file review missed the two real newline issues. Recovered prior context with the same reviewer persona for only the two-file delta: no findings; static review, not independent runtime or merge approval.

All 43 original sealed artifacts remain unchanged. The historical owner JSON passed only the semantic verifier and was not canonical-schema-valid; additive recovery evidence corrects it without invented owner rows. Earlier pytest cleanup warnings remain in historical logs. Local LF/CRLF controls are not a native Windows run; no full-suite, main-CI or release-green claim is made.

Executed recovery commands and exact results

All validation used python3 "$GATE" --lane ci-contract-fixtures --timeout 1800 -- COMMAND with one admitted validator. $RECOVERY is the additive session evidence directory; original evidence remains immutable.

.venv/bin/python "$RECOVERY/run_checks.py" restored-head ran both complete integration modules:

60 passed in 104.35s (0:01:44)

.venv/bin/python "$RECOVERY/run_checks.py" quality-head tests/quality:

68 passed in 78.60s (0:01:18)

The admitted .venv/bin/python "$RECOVERY/qualify.py" ran these exact checks serially at unchanged recovery head:

Inner command	Result
uv run --frozen --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py	All checks passed!
uv run --frozen --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py	2086 files already formatted
uv run --frozen --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py	Exit 0; 10.00/10
bash scripts/lint-auth-signals.sh	[+] auth-signal lint clean
bash scripts/lint-architecture-boundaries.sh	Exit 0
Current CI YAML I/O, portable-path regex and 2100-line scans	Passed; equivalent Python scans avoid macOS grep -P absence.
uv run --frozen --extra dev python scripts/check_test_assertions.py	[+] assertion-quality ratchet clean: AQ001=4, AQ002=12
uv run --frozen --extra dev python scripts/check_exact_test_duplicates.py	[+] exact test duplicate ratchet clean: 1410 files, 0 allowed duplicate group(s)
.venv/bin/python "$RECOVERY/collect_proof.py"	Original 43 digests intact; runtime binding, inventory, full schema and strict terminal owner proof passed.

Historical 1031/44/68 logs/XML, baseline, failed attempts and original diagram validation remain sealed at 6ec1665bf. The retained Mermaid block is byte-identical to the previously mmdc-validated diagram.

Explicit ID mappings:

• test_dir_with_apm_yml_still_returns_marker_msg -> test_dir_with_valid_apm_yml_returns_none.
• test_warn_overwrite_importerror_path -> test_warn_overwrite_console_fallback, now [native], [lf], [crlf].
• Each of the four original test_installed_target_survives_audit_and_reinstall[...] IDs -> corresponding [...,lf] / [...,crlf] variants (pytest uses a final -lf / -crlf suffix).

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	Selective/transitive installs retain service graphs and reject unusable sources before effects.	Secure by default, Governed by policy	tests/integration/test_selective_install_mcp.py::TestSelectiveInstallTransitiveMCPIntegration::test_lockfile_records_transitive_mcp_servers (regression-trap for #3335), deep/diamond/global/update modules	integration
2	Managed stale cleanup preserves personal settings and refuses unknown ownership.	Secure by default, Multi-harness support	tests/integration/test_codex_windows_toml_install.py::test_codex_stale_cleanup_preserves_windows_literal_paths, test_mcp_integrator_characterisation.py::TestRemoveStale	integration
3	LF/CRLF edits survive reinstall; explicit repair restores exact originals and clean audit.	DevX (pragmatic as npm), Governed by policy	tests/integration/test_grok_audit_target_lifecycle.py::test_installed_target_survives_audit_and_reinstall (eight variants)	e2e
4	Cached policy replay preserves enforcement and truthfully identifies its source.	Governed by policy	tests/integration/test_govern_core_contract.py	e2e
5	Cold frozen replay restores valid declarations but rejects mismatches without rewriting durable bytes.	Governed by policy	tests/integration/test_frozen_cold_cache_apm_package_e2e.py	integration
6	Named generic-Git installs retain unselected service evidence and exact commits.	Vendor-neutral, Multi-harness support	tests/integration/test_frozen_native_service_lifecycle.py::test_selected_package_install_retains_unselected_service_evidence	e2e
7	Identity diagnosis is exact and recovery restores original identity/commit/hash.	Secure by default	tests/integration/test_hermetic_lifecycle_foundation.py::test_generated_artifact_tampering_reports_cause_and_recovers (five parameters)	e2e
8	No-logger promotion preserves source bytes and emits the overwrite message.	DevX (pragmatic as npm)	tests/integration/test_skill_integrator_hermetic.py::TestPromoteSubSkills::test_warn_overwrite_console_fallback (native/LF/CRLF)	integration

How to test

• Use an owned environment and installed CLI from this checkout; set APM_BINARY_PATH="$PWD/.venv/bin/apm" and APM_E2E_TESTS=1.
• Under the required admission gate, run .venv/bin/python -m pytest -n0 -q -p no:cacheprovider tests/integration/test_grok_audit_target_lifecycle.py tests/integration/test_skill_integrator_hermetic.py; expect 60 passes, including eight lifecycle and three console newline variants.
• Inspect retained two-failure negative proof and restoration; do not replace exact byte comparisons with normalized text.
• Run current lint/quality commands above; inspect historical 39-ID mapping separately from recovery's parameter expansion and head-bound results.
• Have the designated merge manager inspect actual new-head PR checks and independently qualify this candidate. Scoped local proof does not establish full main/release success.

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

• Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
  (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
  row).
• Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
  updated.
• Test edit: a @pytest.mark.req("req-XXX") test under
  tests/spec_conformance/ added or extended.
• CONFORMANCE.{md,json} regenerated via
  uv run --extra dev python -m tests.spec_conformance.gen_statement
  and committed.
• N/A -- this PR does not change OpenAPM-observable behaviour.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Daniel Meppiel (danielmeppiel)]

Docs sync advisory

Verdict: no_change * Pages affected: 0 * LLM calls: 1/15 * Took: 1.13s (deterministic recovery)

No additional documentation changes are needed. All 23 changed paths match tests/** or docs/** in the corpus index, and none match user_surface_paths. Production and CLI behavior are unchanged; the separate contributor fixture-guidance bullet is already included.

The requested classifier agent could not access filesystem/shell tools and returned blocked. The orchestrator recovered the deterministic L0 path check directly against the live PR diff and .apm/docs-index.yml, then validated the result against the actual classifier-return schema. No second classifier or documentation panel was spawned.

This advisory does not review prose quality or qualify CI/merge readiness. Confidence: high; source: L0; CDO redrafts: 0.

---

Generated by docs-sync. This comment is AI-generated and may contain errors.

[Copilot]

🟡 Changes recommended

Two new LF-only byte assertions conflict with Windows fixture newline translation and fail despite correct preservation behavior.

2 open issues

• Write tampered fixture as bytes for cross-platform assertion · New
• Use fixture bytes to avoid platform-dependent newline assertion · New

What changed in this PR

Repairs the bounded test-fixture and assertion drift tracked in #3335 without changing production code. The changes align integration tests with current source-admission, managed-ownership, and lifecycle contracts.

Changes:

• Replace incomplete downloader mocks and hand-authored cleanup authority with real models, materialized sources, and writer-generated evidence.
• Update preservation, policy, diagnostic, frozen-replay, and identity assertions while retaining negative controls.
• Document source and ownership fixture requirements.

File	Description
tests/​utils/​materialized_downloader.py	Adds package-specific source materialization.
tests/​utils/​managed_mcp.py	Generates real managed MCP write evidence.
tests/​integration/​test_wave6_integrators_coverage.py	Uses authorized Claude cleanup fixtures.
tests/​integration/​test_update_mcp_lsp_prune.py	Supplies real sources and update references.
tests/​integration/​test_skill_integrator_hermetic.py	Exercises console fallback and promoted bytes.
tests/​integration/​test_selective_install_mcp.py	Materializes dependency graphs and rejects invalid sources.
tests/​integration/​test_policy_install_e2e.py	Requires successful warn-mode installation.
tests/​integration/​test_mcp_integrator_install_flow.py	Verifies cleanup with managed ownership evidence.
tests/​integration/​test_mcp_integrator_characterisation.py	Covers real clients, cleanup refusal, and survivors.
tests/​integration/​test_integrators_validation_rules.py	Updates canonical local-source diagnostics.
tests/​integration/​test_integrators_end_to_end.py	Checks authorized cleanup and unverifiable-entry preservation.
tests/​integration/​test_hermetic_lifecycle_foundation.py	Requires exact identity-tamper findings.
tests/​integration/​test_grok_audit_target_lifecycle.py	Checks preservation before explicit repair.
tests/​integration/​test_govern_core_contract.py	Distinguishes policy provenance while preserving semantic equality.
tests/​integration/​test_global_mcp_lockfile_e2e.py	Materializes sources for global lockfile tests.
tests/​integration/​test_github_downloader_validation_flow.py	Exercises virtual download materialization.
tests/​integration/​test_github_downloader_phase3w4.py	Verifies persisted virtual-file metadata and bytes.
tests/​integration/​test_frozen_native_service_lifecycle.py	Uses controlled generic Git with preflight tracing.
tests/​integration/​test_frozen_cold_cache_apm_package_e2e.py	Builds genuine baselines and checks frozen mismatch refusal.
tests/​integration/​test_deps_models_validation.py	Tests empty projects and explicit lockfile creation.
tests/​integration/​test_commands_mcp_flow.py	Updates diagnostics and valid-manifest acceptance.
tests/​integration/​test_codex_windows_toml_install.py	Adds ownership-backed cleanup and refusal controls.
docs/​src/​content/​docs/​contributing/​integration-testing.md	Documents source and ownership fixture contracts.

_{Review effort: Balanced}

[Daniel Meppiel (danielmeppiel)]

Merge Master assessment: ship_now

At 125d7c4ba65f20b552b0b7a4de3e384d70ba6d4a, the bounded #3335 fixture correction is ready for serialized landing. Production code is unchanged.

Both Copilot findings were legitimate and are fixed: tampering now writes explicit LF/CRLF bytes across all four original target variants; promotion compares a pre-copy source-byte snapshot across native/LF/CRLF inputs. Exact preservation, receipts, failing audit before explicit force, repair, count, path and console assertions remain. Both original threads are resolved.

Binding	Verified evidence
Recovery 125d7c4b	Both complete affected modules: 60 passed, no errors/skips. Quality: 68 passed. Two deliberate old-behavior byte failures were restored before the final commit and passing run. Full canonical lint/static checks passed.
Historical 6ec1665b	Original 1,031 passes across 20 modules / 44 selected passes / 68 quality passes remain historical, not new-head executions. All 39 assigned IDs are accounted for: five now map to 11 passing newline variants; 34 unchanged cases retain their earlier evidence.
Hosted 125d7c4b	Own CI and Merge Gate succeeded. All 23 raw checks are terminal: 20 success, two neutral, one skipped. Neutral/skipped checks are not test passes.

I independently checked all 49 recovery and 43 original artifact digests, all 23 published source-file hashes, runtime binding, JUnit outcomes, review resolution and actual-main compatibility. Canonical completion validation and strict terminal owner-v2 verification pass with zero production-owner touches; functional proof is recorded separately, not fabricated as owner rows. Current main (4adaa21f) is already an ancestor, and object-only composition matches the tested head tree.

The original independent review missed these newline defects. A same-persona, recovered-context review of the two-file delta found no remaining issues; this is not a full-panel/CEO verdict or a claimed hosted-Windows reproduction. This is my separate merge-readiness assessment, not a rewritten source/reviewer verdict.

Only #3335 is completed by this PR. The original 6da nonpublishing run remains failed, and the full actual-main CI mission remains open. Generic-Git evidence is not REST coverage; historical failures and cleanup warnings remain preserved. No #3246/#3339, native-consent, specification or broader-scope hold is waived.

---

Generated by autopilot-pr-merge-worker. This comment is AI-generated and may contain errors.