[samrusani]

Summary

The required check "Read-budget maximum workload smoke (Postgres)" fails when one test reaches the --timeout=180 of its pytest step. The step runs four tests, and in every failure it was the round-three case tests/integration/test_label_round3_read_budget_postgres.py::test_postgres_round3_random_read_budgets[True-3000-extra-rows]. This change gives that step the per-test limit the weekly parent budget test already sets for itself, 600 seconds (pytest.mark.timeout(600), added with the paired read budget gate in #675). No test, read budget or threshold changes.

The failures

Both on the step "Maximum-workload read-budget controls", attempt 1, with the other tests of the step passing:

Run	Head	Where the limit was hit	Result of the step
40225383177 (job 121265526616), pull request #680	110fc02c	in the test, in capture_read_profiles while it waited for the profile subprocess: Failed: Timeout (>180.0s) from pytest-timeout.	1 failed, 3 passed in 444.70 s
40294770146 (job 121478533947), pull request #681	c0f566b6	in the teardown of the same case, in DROP DATABASE IF EXISTS ... WITH (FORCE) of tests/integration/conftest.py: ERROR at teardown ... Timeout (>180.0s)	4 passed, 1 error in 435.14 s

Neither change touched a read path. The job passes on most runs. Of the last 60 completed runs of the Tests workflow, the smoke job failed 4 times, always with Failed: Timeout (>180.0s) on a round-three 3000 case: the two above, run 40094845873 (a pull request, [True-3000-extra-rows]) and run 40121004149 (a push to main, where both [False-3000-all-visible] and [True-3000-extra-rows] timed out; 2 failed, 2 passed in 471.52 s). Of the 60, 54 passed (the step took 222 to 448 s, median 374 s, for the four tests), 4 failed and 2 were cancelled. The timeout covers the setup, the call and the teardown of a test together, and that case seeds 3000 sources, repairs the fixture and then reads three revisions and writes read profiles, so it takes about three minutes on a shared runner and 180 seconds leaves it no margin.

What changed

• .github/workflows/tests.yml: --timeout=180 becomes --timeout=600 in the step "Maximum-workload read-budget controls" of the job python-integration, with a comment that gives the reason. The job's timeout-minutes stays 120: four tests at 600 seconds are 40 minutes at worst, so the setup and the upload of the samples fit with room to spare. The step has no --session-timeout and none is added. The integration shards keep --timeout=180 --session-timeout=1500. A test that hangs still stops after 600 seconds.
• tests/unit/test_ci_integration_job_split.py: a new test, test_the_maximum_workload_smoke_gives_each_test_the_limit_of_the_weekly_parent_budget. It compares the step's --timeout with the number in the pytest.mark.timeout(...) marker of tests/integration/test_weekly_parent_read_budget_postgres.py (both 600), counts the four node ids, requires that four tests of that length plus 15 minutes fit in the job's timeout-minutes, and, if the step ever gets a --session-timeout, that it holds at least one test and ends 5 minutes before the job does. Nothing pinned the step's --timeout before.
• scripts/derived_label_mutations.json (a separate commit): the comment shifts the lines below it, and six cases point at tests.yml. line and sha256 are refreshed. Every before text still occurs once.

The first run on this pull request

The smoke job failed on this head too (run 40424477750, 8 m 16 s), but not on the timeout. The step ended 1 failed, 3 passed in 438.95s, and the failing assertion is the absolute screen limit, not pytest-timeout: for [True-3000-extra-rows], key trusted_local_agent, the smallest wall time of the workspace screen at the head was 1.533 s against the limit of 1.5 s (ALICE_READ_ABSOLUTE_LIMIT_SECONDS). The merge base read the same screen in 1.440 s on the same runner and this branch changes no read path, so this is runner noise on that gate. The four cases of the run read workspace at the head in 1.378, 0.946, 1.533 and 1.369 s, so the screen sits within a few percent of its limit on a shared runner. That limit is a read budget and this change does not touch it.

This gate was out of reach of the earlier failures. In capture_read_profiles and in the teardown the timeout came first, and the assertion on the collected failures runs after capture_read_profiles. A run that no longer times out can therefore meet it. On the run of #683, which has no workflow change, the job passed in 7 m 55 s. All other checks of this run passed.

Validation

• tests/unit in the five shard selections of .github/workflows/tests.yml, each as its own process (python -m pytest <selection> -q -p no:cacheprovider), 658 test files:
  • shard 1: 5,971 passed, 0 failed, 2 skipped
  • shard 2: 4,228 passed, 0 failed, 11 skipped
  • shard 3: 14,997 passed, 0 failed, 23 skipped
  • shard 4: 488 passed, 0 failed
  • shard 5: 2,717 passed, 0 failed, 4 skipped
  • Total 28,401 passed, 40 skipped.
• python scripts/check_commit_authors.py --base origin/main --head HEAD: PASS (2 commits). ruff check apps/api/src/alicebot_api scripts tests eval/longmemeval: clean. python scripts/release_check.py and python scripts/check_control_doc_truth.py: PASS. No protected path is touched (scripts/check_protected_paths.py with an event payload for this head).
• Mutations on the new test, each applied alone with the file restored from saved bytes: 7 of 7 killed. The step's --timeout set back to 180, set to 601, or dropped; timeout-minutes of the job set to 40; the marker of the weekly test set to 300; a --session-timeout=300 added (below one test); a --session-timeout=7200 added (past the job). The unmutated test passed before and after.
• Manifest: the six refreshed cases (budget-round-two-test-in-ci, budget-smoke-job-keeps-the-one-second-limit, required-read-budget-measures-prelabels, required-job-sets-the-absolute-gate, smoke-job-drops-the-round-two-case, smoke-job-drops-the-handoff-case) were replayed with scripts/verify_derived_label_mutations.py --only <name> --output <dir> against the final head (6 names, 6 results): 6 killed, each with a green baseline. test_derived_label_mutation_manifest_matches_tree.py passes.
• CI on this head (run 40424477750): every check passed except the smoke job, which failed on the absolute gate described above. Nothing was re-run.