[brettheap]

Lane: openRepoTools-4
Claim: opensoft/openRepoTools#77 (comment)

What this is

This PR is for opensoft/openRepoTools#77. This leg's tests and tests-no-submodule checks take 58 to 81 minutes, because each runs the whole suite serially in one job. Here each becomes a gate job over a matrix of shard jobs, and all four required check names stay.

This is stage 1 of lane 4's design (opensoft/openRepoTools#165 comment 6724536105).

• Lane 3's ruling 8 gave this leg's CI matrix to lane 4 (Amendment 22 act 2: the SQL register under lanes-edit.sh — mode=primary writers in one transaction each, the estate switch and lanes/STORE, the write journal and the renderer (one mirror commit per write), exit 69, fail-closed readers, lanes-index import/rollback, SERIALIZABLE claim order, and the agreement tests openRepoTools#165 comment 6723967774).
• On 2026-11-08 Brett moved #77 to lane 4:
  • lane 4 relayed his word in CI's Linux test job takes 46–72 minutes a run, because its Python tests and the shell suite run one after the other openRepoTools#77 comment 6726773083;
  • lane 5 released the issue in comment 6726814694;
  • lane 4 claimed it in comment 6726847078.

What changes

• .github/workflows/tests.yml:
  • tests-shard runs pytest-1 to pytest-4 and shell;
  • tests-no-submodule-shard runs pytest-1 to pytest-3 and shell;
  • tests and tests-no-submodule become the gates over them.
• .github/ci/ci_shards.py, new: its plan, run, verify and ids commands.
  • It deals tests/test_*.py to the pytest shards, longest first, by the seed seconds in .github/ci/weights.json. A test file with no weight still runs, so adding one needs no edit.
  • The lane helpers' shell suite is one pytest item of about 27 minutes, so it has its own shard and is deselected from the others.
• .github/ci/weights.json, new.
• tests/test_ci_shards.py, new, and tests/test_tests_workflow.py, updated to the new shape.
• Unchanged: guard-launch-mode, package, release.yml and the rulesets.

The four names, and the count

• The gates keep the names tests and tests-no-submodule. They carry no name:.

• They run with if: ${{ always() }}. A failed or cancelled shard turns its gate red, and a gate is never skipped. A skipped job would pass a required check.

• Each shard keeps the node ids it chose and its JUnit report, in an artifact named by run attempt.

• The gate collects the whole suite as a serial run does. It fails unless:

  • every shard exited 0 and ran what it chose;
  • no id is in two shards;
  • the shards' ids equal the full collection, id for id.

  It then prints the line a serial run would print, N passed, M skipped. A "Re-run failed jobs" attempt is read by its attempt number.

Estimate

• Today: 58 to 81 minutes.
• With this PR: about 30 minutes (28 to 33) for the slowest check.
• The floor is the shell suite, 23 to 32 minutes alone. The planned pytest shards are 638 to 657 s for tests and 649 to 651 s for tests-no-submodule. The shell shard is 1,712 s and 1,506 s.

Credit

This rests on lane 5's measurements:

• The three-wide runs (CI's Linux test job takes 46–72 minutes a run, because its Python tests and the shell suite run one after the other openRepoTools#77 comment 6373986119) show that the shell suite is the whole critical path, and that it slows when it shares a runner. That is why it gets a runner of its own here.
• The suite's section timings come from lane 5's The shell suite times each of its sections and its exit-trap teardown, and prints both in its transcript and in CI's step summary (#77, step 1a) openRepoTools#261 and The shell suite's wrapper prints the suite's timing table and teardown line on a pass, where CI's log can read them (#77, step 1b) openRepoTools#265.

Checks before opening

• actionlint 1.7.12, with shellcheck: clean on the new tests.yml, and on release.yml.
• The pinned reader: tests.yml loads with openRepoShape v1.1.1's repo_shape.load_yaml and lists six jobs.
• ci_shards.py plan and ids, for both variants:
  • all 52 tests/test_*.py files land in exactly one pytest shard;
  • the shell item lands only in shell.
• The two new action pins are the commits of actions/upload-artifact v7.0.1 and actions/download-artifact v8.0.1, checked against those tags. They are the same pins release.yml already uses.
• The updated test_tests_workflow.py's assertions were checked against the committed workflow by script.
• verify was exercised over ten synthetic report sets: pass, skips, a missing shard, a failed shard, a duplicate id, and an id extra to or missing from the full collection.
• Not run yet: the new and changed tests themselves. The local wrapper run is queued behind the workstation's test lock, so this PR's CI is their first execution. It is also the matrix's first real run.

Not in this PR

• The spec leg's contract. specs/004-migrate-to-triad/contracts/workflows.md says the workflow has four jobs, "no fifth", held by tests/test_tests_workflow.py.
  • The four required names stay, but the workflow now has six jobs, so that text needs an amendment in the spec leg.
  • No test in this leg reads that file.
  • The spec-leg change waits for Brett's word.
• A stale docstring: tests/test_lane_helpers_suite.py:5 still says tests.yml runs python3 -m pytest tests -q.
• Stage 2: cutting the shell suite by section, which would bring it to about 13 to 17 minutes. It needs a probe run showing which sections run alone, and is a separate change. The planner refuses more than one shell shard for a suite that cannot be cut.

Review and landing

Lane openRepoTools-3 reviews this PR. It lands only on Brett's word.

🤖 Generated with Claude Code

Summary by Sourcery

Parallelize both CI test suites with validated matrix shards while retaining the four existing required check names.

New Features:

• Shard the main and no-submodule test suites across parallel pytest and shell jobs while preserving the existing required gate check names.

Bug Fixes:

• Ensure failed, cancelled, incomplete, duplicated, or otherwise inconsistent shards make the corresponding gate fail instead of passing or being skipped.

Enhancements:

• Add deterministic, weight-based shard planning with full-suite collection and report validation, including support for rerun attempts and serial-style test summaries.

CI:

• Update the test workflow with matrix shard jobs, artifact reports, and always-running aggregation gates.

Tests:

• Add coverage for shard planning, shell-suite handling, workflow structure, report aggregation, and failure cases.

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 6 days and 16 hours by commenting @sourcery-ai review. Upgrade to get a review now.

[sourcery-ai]

Reviewer's Guide

Uses of the API

Replace the serial test jobs with shard matrices and gate jobs that retain the required check names and verify shard reports.

• .github/workflows/tests.yml L116-140, L182-220, L268-305 (+2 small edits)

Configuration

Add the shard planner and timing weights that define which pytest files and shell suite sections each CI shard runs.

• .github/ci/ci_shards.py L1-375
• .github/ci/weights.json L1-165

Tests

Test shard planning and report verification, and update workflow assertions for the shard jobs and gates.

• tests/test_ci_shards.py L1-239
• tests/test_tests_workflow.py L89-102, L861-877, L903-951 (+6 small edits)

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sonarqubecloud]

Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE