[Daniel Meppiel (danielmeppiel)]

Description

TL;DR

Raise only Linux x64 full-release integration from 60 to 120 minutes and add a focused audit witness to the existing unprivileged Linux PR binary job. Preserve all audit assertions, full-suite selection, worker grouping, other platform bounds and publication gates. Native Linux passed all six audit cases; this is ready for diagnostic/budget qualification, not an application-fix claim.

Warning

The original full-run Linux x64 [git] assertion remains unexplained. No application code changed and no flake is assumed. Full trusted qualification remains required before publication.

Problem (WHY)

Trusted run 39819494530, exact main 5a9be32972c8600745fc0d4933357fd4a2a9e61c, recorded the audit failure at 17:46:18.921651Z, then exhausted 60 minutes at 78%. Workers were progressing; linear extrapolation is approximately 77 minutes before tail headroom. The timeout cannot explain the earlier assertion failure.

The investigation follows the Agent Skills execution loop: "do the work, run a validator (a script, a reference checklist, or a self-check), fix any issues, and repeat until validation passes."

Approach (WHAT)

• Increase only Linux x64; preserve Linux ARM 60, Windows 20, macOS ARM 120 and focused Intel 30 minutes.
• Run all six audit identity cases against the existing Linux PR build, with a five-minute bound and no repository secrets.
• Preserve isolated HOME/config/cache/local repositories, clean/damage/global-scope assertions and read-only snapshots.
• Return full-context diagnosis to the release owner rather than manufacture a source fix from passing focused runs.

Implementation (HOW)

File	Intent
.github/workflows/build-release.yml	Architecture-selective timeout; all other parsed content equals exact main.
.github/workflows/ci.yml	Six-case witness inside existing pr-binary-smoke; no new trigger/job/secret/permission/required check.
tests/integration/test_audit_workspace_identity_lifecycle.py	Preserve isolated audit output before snapshot/JSON assertions. Credential warmup exposes only return code and output lengths, including under pytest assertion rewriting.
tests/unit/test_platform_contract_workflow.py	Assert exact budget/witness shape; reject old/missing budgets, bypasses and fabricated commands.
docs/src/content/docs/contributing/integration-testing.md	Document platform bounds and the witness's limited role.

Diagrams

Highlighted interactions are new; coverage and release dependencies remain unchanged. Validated with mmdc.

sequenceDiagram
    participant CI as ci.yml
    participant PR as pr-binary-smoke
    participant R as build-release.yml
    CI->>PR: Build Linux x64 binary without secrets
    rect rgb(255, 247, 200)
        PR->>PR: Run all six isolated audit identity cases
        PR-->>CI: Exit status and captured failure diagnostics
    end
    Note over CI,PR: Existing non-required PR witness only
    rect rgb(255, 247, 200)
        R->>R: Linux x64 integration budget 60 to 120 minutes
    end
    Note over R: Same corpus, workers, HOME grouping and publication gates

Loading

Trade-offs

• More runner time on a genuine hang, but still bounded and fail-closed. No coverage reduction or sharding redesign.
• The focused PR job remains non-required; it is evidence, not a replacement release gate.
• No speculative application fix. Original diagnosis and full trusted qualification remain outstanding.

Benefits

1. Only the exhausted Linux x64 budget changes.
2. Six packaged audit cases report separately from the long corpus.
3. Mutation tests reject budget/witness regressions; credential diagnostics cannot render unexpected values.

Issue and approved scope

Issue: #3361.

Human scope-approval comment: none fabricated. Authorization comes from the user conversation relayed by the release owner: "ok so merge it and proceed until release happens, fixing anything that needs fixing on the way and bypass merging the fixes (CI)". The coordinator explicitly authorized a truthful diagnostic/budget repair after native focused evidence and credential-safe requalification. The issue records agent-authored conversation provenance, not a human GitHub approval comment.

Review contact: Daniel Meppiel (@danielmeppiel). This PR does not complete the issue: the original assertion is unexplained and full qualification remains. The parent alone owns merges, dispatches, tagging and publication.

Type of change

• Bug fix
• New feature
• Documentation
• Maintenance / refactor

Testing

• Tested locally
• All existing tests pass
• Added tests for new functionality (if applicable)

Validation

Native Linux: CI run 39832616381, job 120012448221, head e3c24cab4d7df9cb4e1e23f88e6e9ab6b957b017, succeeded. The generated Linux x64 PyInstaller binary ran:

uv run --frozen --extra dev pytest tests/integration/test_audit_workspace_identity_lifecycle.py -vv -ra --tb=long --show-capture=all --no-showlocals

============================== 6 passed in 20.99s ==============================

Final head c76045bf36fa08b4cc5b2bb1f9c48cac0545bc08 changes only credential-diagnostic presentation from that witnessed head. Final local qualification passed; final-head hosted checks are not claimed green.

Local execution	Result
uv run --extra dev pytest -q 'tests/integration/test_audit_workspace_identity_lifecycle.py::test_fresh_clone_audit_preserves_identity_across_cold_repeated_and_warm_runs[git]' -rA --tb=long	Original installed Python CLI: 1 passed in 10.53s.
uv run --extra dev pytest -q tests/integration/test_audit_workspace_identity_lifecycle.py -rA --tb=long --basetemp=<owned-scratch>/audit-before	Before edits: 6 passed in 25.98s.
APM_BINARY_PATH=<exact-main-macos-artifact>/dist/apm-darwin-arm64/apm uv run --extra dev pytest -q tests/integration/test_audit_workspace_identity_lifecycle.py -rA --tb=long --basetemp=<owned-scratch>/audit-frozen-before	Exact run's frozen macOS binary before edits: 6 passed in 28.46s.
uv run --extra dev pytest -q tests/unit/test_platform_contract_workflow.py tests/unit/test_security_workflow_contracts.py tests/quality/test_ci_topology.py tests/unit/scripts/test_lifecycle_smoke_sharding.py --tb=short --basetemp=<owned-scratch>/workflow-green	219 passed in 75.96s.
APM_BINARY_PATH=<exact-main-macos-artifact>/dist/apm-darwin-arm64/apm uv run --extra dev pytest -q tests/integration/test_audit_workspace_identity_lifecycle.py tests/unit/test_platform_contract_workflow.py --tb=long --show-capture=all --no-showlocals --basetemp=<owned-scratch>/audit-final-redacted	Final credential-safe tree: 58 passed in 17.87s.

Paths are abbreviated: <owned-scratch> is the isolated recovery artifact directory; the macOS binary came from exact run 39819494530. A scratch pytest probe injected sentinel stdout/stderr at both unexpected-return-code and unexpected-output boundaries: 2 passed, no sentinel in rewritten assertions. Its initial invocation failed import collection; python -m pytest corrected repository imports. Four scratch-only unknown-marker warnings arose outside repository configuration; repository tests passed without them.

Old/new regression and complete canonical lint

Before workflow edits:

uv run --extra dev pytest -q tests/unit/test_platform_contract_workflow.py -k 'standalone_integration_timeouts_are_platform_specific or linux_and_arm_retain_non_live_corpus_grouped_parallelism or linux_binary_job_runs_isolated_audit_witness or missing_or_old_timeout' --tb=short --basetemp=<owned-scratch>/workflow-red

3 failed, 4 passed, 45 deselected in 0.68s

Expected failures identified the old 60-minute setting (two contracts) and missing witness. All selected contracts passed after edits.

git fetch --quiet origin main && git merge --no-edit origin/main returned Already up to date.. Every final-tree command passed:

uv run --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py
uv run --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py
uv run --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py
bash scripts/lint-auth-signals.sh
bash scripts/lint-architecture-boundaries.sh

All checks passed!
2091 files already formatted
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
[+] auth-signal lint clean

YAML I/O, 2100-line and raw-relative-path guard bodies were executed directly from ci.yml using GNU ggrep on macOS: all passed. The first format check requested formatting of the edited workflow contract; it was corrected before complete successful reruns.

Unchanged boundaries: parsed old/new build-release.yml objects are identical after restoring only the Unix step timeout to 60. Triggers, matrix, permissions, secrets, markers, worker distribution, failure propagation and release dependencies are unchanged. scripts/test-integration.sh, grouping/collection fixtures, merge-gate.yml, pyproject.toml, uv.lock and CHANGELOG.md are unchanged. APM remains 0.35.0; #3353 and #3360 are preserved.

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	Release checks retain coverage and fail closed within platform budgets	Governed by policy	tests/unit/test_platform_contract_workflow.py::test_missing_or_old_timeout_mutation_is_rejected (regression-trap for #3361), parallelism contracts, tests/quality/test_ci_topology.py	unit
2	Fresh clones audit repeatedly without writing live state or hiding damage	Governed by policy	tests/integration/test_audit_workspace_identity_lifecycle.py: six native Linux passes; original full-run failure unresolved	e2e
3	PR audit evidence needs no production credential and cannot ignore failure	Secure by default	tests/unit/test_platform_contract_workflow.py::test_linux_audit_witness_bypass_mutation_is_rejected, tests/unit/test_security_workflow_contracts.py	unit

How to test

• Run the four-file focused command above; expect 219 passing contracts.
• Build with uv sync --frozen --extra dev --extra build && uv run ./scripts/build-binary.sh, select the generated binary with APM_BINARY_PATH, then run the hosted six-case command.
• Inspect native job 120012448221; distinguish its witnessed head from the final credential-diagnostic-only refinement.
• Parent merges and performs full trusted qualification; require passing assertions and gates before tagging, without assuming a flake.

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

• Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
  (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
  row).
• Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
  updated.
• Test edit: a @pytest.mark.req("req-XXX") test under
  tests/spec_conformance/ added or extended.
• CONFORMANCE.{md,json} regenerated via
  uv run --extra dev python -m tests.spec_conformance.gen_statement
  and committed.
• N/A -- this PR does not change OpenAPM-observable behaviour.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

🟢 Approval recommended

The bounded budget and diagnostic changes preserve assertions and release gates, while full trusted release qualification remains explicitly outstanding.

No open issues 🎉

What changed in this PR

This PR improves Linux x64 release qualification diagnostics and runtime headroom for #3361. It does not claim to resolve the original audit failure or complete full release qualification.

Changes:

• Increase only Linux x64 full-release integration from 60 to 120 minutes, preserving other platform budgets and publication gates.
• Add a five-minute, six-case audit witness to the existing unprivileged Linux PR binary job.
• Capture audit diagnostics, restrict credential-warmup diagnostics to status and output lengths, and extend workflow contracts and documentation.

File	Description
tests/​unit/​test_platform_contract_workflow.py	Enforce platform budgets and audit-witness execution, including mutation checks.
tests/​integration/​test_audit_workspace_identity_lifecycle.py	Capture audit output and summarize credential-warmup results without printing credential output.
docs/​src/​content/​docs/​contributing/​integration-testing.md	Document platform limits and the focused witness's limited role.
.github/​workflows/​ci.yml	Run the packaged audit cases in the existing Linux binary job.
.github/​workflows/​build-release.yml	Raise the Linux x64 integration timeout to 120 minutes.

_{Review effort: Balanced}