Skip to content

fix(ci): budget focused Intel release qualification - #3364

Merged
Daniel Meppiel (danielmeppiel) merged 1 commit into
mainfrom
danielmeppiel-intel-release-budget
Oct 22, 2026
Merged

Daniel Meppiel (danielmeppiel) merged 1 commit into
mainfrom
danielmeppiel-intel-release-budget

Conversation

@danielmeppiel

Copy link
Copy Markdown
Collaborator

Description

TL;DR

Give the focused macOS Intel release-qualification step 60 minutes instead of 30 after the existing limit interrupted passing workers at 91%. This changes one workflow value, adds three timeout mutations, and updates the existing budget paragraph; no test selection or release gate changes.

Important

Bounded local qualification is complete at 0d13f5f852311d92027cffd65c8e7e69475c5150, based on 69124dc24d1beb7ca33c85ec92ce240cc94fd963. This is not a claim that hosted release qualification has passed. The coordinating maintainer owns the exact-head merge, fresh full qualification, tag, and publication.

Problem (WHY)

  • Trusted run 39834866538, Intel job 120020394775 failed only Run focused Intel integration tests, from 19:16:02Z to 19:46:16Z on 2026-10-22.
  • Workers were still passing at 91%: the last pass was 19:46:14.5110070Z, followed by the timeout at 19:46:15.9806190Z. This is inadequate observed budget, not a demonstrated hang.
  • [!] 30 / 0.91 projects roughly 33 minutes, but test progress is not linear and the step also includes setup. Sixty minutes provides headroom, not a completion guarantee.

The exact-field regression contract applies Agent Skills' "Be prescriptive" guidance for fragile operations; the hosted evidence above, not that guidance, determines the budget.

Approach (WHAT)

  • Change only the Intel focused-step timeout, retaining lifecycle_smoke and not live and -n 4 --dist loadgroup.
  • Reject missing, old 30-minute, and oversized 61-minute budgets in focused mutation tests.
  • Prove complete parsed workflow equality after reversing only the Intel timeout, and update the existing documentation paragraph.

Implementation (HOW)

File Bounded change
.github/workflows/build-release.yml Intel 30 -> 60 plus observed-progress/rationale comment. Runtime installation, credentials, triggers, downstream gates, and failure handling are identical.
tests/unit/test_platform_contract_workflow.py Require 60 and add missing/old/oversized timeout mutations; preserve existing marker/worker/platform contracts.
docs/src/content/docs/contributing/integration-testing.md Amend the existing budget paragraph with Intel evidence and the headroom caveat.

Diagrams

The dashed budget node is the only workflow behavior change; both success and failure routes are unchanged.

flowchart LR
    subgraph Intel["macOS Intel qualification"]
        S["Unchanged lifecycle_smoke and not live"]
        B["NEW budget: 60 minutes, previously 30"]
        V["Unchanged isolated release validation"]
    end
    S --> B
    B -->|"success"| V
    B -->|"failure or timeout"| F["Publication blocked"]
    classDef new stroke-dasharray: 5 5;
    class B new;
Loading

Trade-offs

  • Choose additional bounded runtime rather than sharding, exclusions, or skipped assertions. A genuine slow failure can consume 30 extra minutes; this does not prove completion or fix a failed assertion.
  • Keep hosted execution separate from static local contracts. No full release run was launched, cancelled, or retried here, and no version, changelog, product source, or lockfile changed.

Benefits

  1. Focused Intel qualification receives 60 minutes without reducing selected coverage.
  2. Three mutations prevent missing, obsolete, or oversized timeout drift.
  3. Complete parsed equality preserves Linux x64/macOS ARM 120, Linux ARM 60, Windows 20, and all other workflow behavior.

Issue and approved scope

Issue: Fixes #3363.

Human scope-approval comment: None claimed. Authority is the maintainer's direct conversation relayed by the coordinating release session: "merge it and proceed until release happens, fixing anything that needs fixing on the way and bypass merging the fixes (CI)." #3363 transparently records that bounded delegation; it is agent-authored tracking, not a human GitHub approval comment or machine-readable approval. Review contact: Daniel Meppiel (@danielmeppiel) through the coordinating release session.

This PR completes the bounded Intel-budget repair. The parent retains all merge, fresh full qualification, tag, and publication actions; this worker performs none of them.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

The full suite and hosted qualification were not rerun. The unchecked full-suite item is intentional.

Validation

uv run --frozen --extra dev pytest -q tests/unit/test_platform_contract_workflow.py::test_intel_integration_is_marker_scoped_and_bounded against the original workflow: failed as expected, AssertionError: assert 30 == 60, 1 failed in 2.59s.

uv run --frozen --extra dev pytest -q tests/unit/test_platform_contract_workflow.py tests/unit/test_security_workflow_contracts.py tests/unit/test_ado_live_workflow_contract.py: passed.

66 passed in 1.68s
Complete parsed workflow equality and old-workflow regression proof: passed
uv run --frozen --extra dev python - <<'PY'
from copy import deepcopy
import subprocess
import yaml
from tests.unit.test_platform_contract_workflow import _assert_intel_focused_integration
from tests.workflow_contracts import load_workflow, workflow_job, workflow_step
from pathlib import Path
base = '69124dc24d1beb7ca33c85ec92ce240cc94fd963'
path = '.github/workflows/build-release.yml'
old = yaml.safe_load(subprocess.check_output(['git', 'show', f'{base}:{path}'], text=True))
if True in old and 'on' not in old:
    old['on'] = old.pop(True)
new = load_workflow(Path(path))
job = 'build-and-validate-macos-intel'
step = 'Run focused Intel integration tests'
assert workflow_step(workflow_job(old, job), step)['timeout-minutes'] == 30
_assert_intel_focused_integration(new)
try:
    _assert_intel_focused_integration(old)
except AssertionError:
    print('PASS: complete old workflow rejected by new Intel contract')
else:
    raise AssertionError('old workflow unexpectedly accepted')
reversed_workflow = deepcopy(new)
workflow_step(workflow_job(reversed_workflow, job), step)['timeout-minutes'] = 30
assert reversed_workflow == old
print('PASS: complete parsed workflows equal after reversing ONLY Intel timeout 60 -> 30')
PY
PASS: complete old workflow rejected by new Intel contract
PASS: complete parsed workflows equal after reversing ONLY Intel timeout 60 -> 30
Canonical local lint and guards: all passed before push

Latest main was fetched and fast-forward checked before qualification; it remained 69124dc24d1beb7ca33c85ec92ce240cc94fd963.

uv run --frozen --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py && uv run --frozen --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py && uv run --frozen --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py && bash scripts/lint-auth-signals.sh && bash scripts/lint-architecture-boundaries.sh

Result: exit 0. Ruff: All checks passed!; formatter: 2091 files already formatted; pylint: Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00); auth: [+] auth-signal lint clean; architecture: silent exit 0.

The CI grep guards used macOS GNU grep (ggrep); the line guard also included the native probe:

set -euo pipefail
VIOLATIONS=$(ggrep -rn --include='*.py' -P 'yaml\.(safe_)?dump\(.+,\s*[a-zA-Z_]\w*\b' src/apm_cli/ | ggrep -v 'utils/yaml_io.py' | ggrep -v '# yaml-io-exempt' || true)
if [ -n "$VIOLATIONS" ]; then printf '%s\n' "$VIOLATIONS"; exit 1; fi
printf '%s\n' 'PASS: YAML file-handle safety guard'
MAX_LINES=2100
VIOLATIONS=$({ find src/ scripts/architecture_linter/ -name '*.py' -print0; printf '%s\0' scripts/lint_architecture_boundaries.py scripts/windows_native_symlink_probe_entry.py; } | xargs -0 -I{} awk -v max="$MAX_LINES" 'END { if (NR > max) printf "%s: %d lines (max %d)\n", FILENAME, NR, max }' {})
if [ -n "$VIOLATIONS" ]; then printf '%s\n' "$VIOLATIONS"; exit 1; fi
printf '%s\n' 'PASS: 2100-line source/architecture/probe guard'
if ggrep -rn --include='*.py' -P 'str\([^)]*\.relative_to\(' src/apm_cli/ | ggrep -v portable_relpath | ggrep -v '\.pyc'; then exit 1; fi
printf '%s\n' 'PASS: portable relative-path guard'
PASS: YAML file-handle safety guard
PASS: 2100-line source/architecture/probe guard
PASS: portable relative-path guard

git diff --check: passed. Added-line ASCII check: passed. Docs impact classifier: L0 no_change (CI/tests/already-updated docs only), no panel fan-out.

mmdc -i <session-artifacts>/intel-budget.mmd -o <session-artifacts>/intel-budget.svg --quiet: passed, exit 0.

Local-only caveat: the initial frozen run failed downloading GitPython 3.2.1 from PyPI with Socket is not connected. Restoring that exact wheel via the prior approved packagefeedproxy.microsoft.io index with --require-hashes --no-deps and the current lockfile SHA-256 made frozen runs work. No lock regeneration or integrity bypass; this is distinct from the hosted timeout.

Scenario Evidence

# Scenario (user promise) Principle(s) Test(s) proving it Type
1 Intel release coverage keeps its selected lifecycle cases and receives exactly 60 minutes Governed by policy tests/unit/test_platform_contract_workflow.py::test_intel_integration_is_marker_scoped_and_bounded; ::test_intel_missing_old_or_oversized_timeout_mutation_is_rejected (regression-trap for #3363) unit
2 Other platform budgets and credential/live-service boundaries stay unchanged Secure by default, Governed by policy tests/unit/test_platform_contract_workflow.py, tests/unit/test_security_workflow_contracts.py, tests/unit/test_ado_live_workflow_contract.py unit

How to test

  • Run the three-file pytest command above; expect 66 passes, including all three Intel timeout mutations.
  • Run the parsed-workflow proof above; expect old-workflow rejection and complete equality after reversing only Intel's timeout.
  • Run the canonical lint and guard commands above; expect exit 0.
  • After the maintainer merges the exact head, the coordinating release session performs fresh full qualification; hosted Intel completion remains required.

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Raise only the focused macOS Intel step to 60 minutes after observed passing progress at 91% exhausted the 30-minute limit. Preserve test selection and release gates; add missing/old/oversized budget mutations and update the existing budget paragraph.\n\nRefs #3363

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 851e86b9-f93e-4efb-a823-9806aba2b112
Copilot AI balanced review requested due to automatic review settings October 22, 2026 19:56
@danielmeppiel
Daniel Meppiel (danielmeppiel) merged commit 17a2fc4 into main Oct 22, 2026
23 checks passed
@danielmeppiel
Daniel Meppiel (danielmeppiel) deleted the danielmeppiel-intel-release-budget branch October 22, 2026 19:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The bounded timeout adjustment, regression tests, and documentation are consistent, with no blocking issues identified.

No open issues 🎉

What changed in this PR

This PR addresses #3363 by increasing the focused macOS Intel release-qualification timeout from 30 to 60 minutes, without changing test selection, worker settings, or release gates.

Changes:

  • Increase the Intel step budget with a comment explaining the observed timeout.
  • Update the timeout contract and add mutations for missing, obsolete, and oversized budgets.
  • Document the additional headroom without claiming guaranteed hosted completion.
File Description
tests/​unit/​test_platform_contract_workflow.py Require 60 minutes and reject three invalid timeout configurations.
docs/​src/​content/​docs/​contributing/​integration-testing.md Update the Intel budget and explain its limits.
.github/​workflows/​build-release.yml Raise the focused Intel timeout to 60 minutes.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ci): give focused macOS Intel qualification a 60-minute budget

2 participants