From ebef0b9ff3773fd4c252144251fea55af7eb1e2b Mon Sep 17 00:00:00 2001 From: Nat Budin Date: Tue, 6 Oct 2026 05:20:30 -0700 Subject: [PATCH] Run Minitest in 4 parallel processes in CI Minitest was the slowest CI job (about 290s of tests on one process of a 4-core runner). PARALLEL_WORKERS=4 runs it across all four cores: locally the same 1893 tests (same assertions, same result) take 42-77s instead of 269s. Each worker writes its own Cobertura report (coverage/worker-N), rather than merging with the others as the run exits, which races the workers finishing. The main process's coverage.xml holds what loading the app covered, so the existing coverage report job merges it and the workers' reports with scripts/merge_coverage.rb. Merged that way, the coverage is line-for-line identical to a single-process run (12,993 of 16,179 lines in both, none lost or gained across all files). Parallel runs are opt-in (PARALLEL_WORKERS), so a plain `rails test` is unchanged; the macOS fork crash workaround is in agent-docs/testing.md. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_013mDr1nhgyzST2iPzCzMcCd --- .github/workflows/ci.yml | 4 +++- agent-docs/testing.md | 16 ++++++++++++++++ test/test_helper.rb | 18 +++++++++++++++--- 3 files changed, 34 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8168fd8b3ec..73bd132f571 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -93,6 +93,8 @@ jobs: env: TEST_DATABASE_URL: postgres://postgres:postgres@localhost/intercode_test RAILS_ENV: test + # the runner has 4 cores; each worker gets its own copy of the test database + PARALLEL_WORKERS: 4 services: postgres: image: postgres:18 @@ -352,7 +354,7 @@ jobs: name: vitest-coverage path: vitest-coverage - name: Merge coverage reports - run: ruby scripts/merge_coverage.rb merged-coverage.xml minitest-coverage/coverage.xml vitest-coverage/cobertura-coverage.xml minitest-system-coverage/coverage.xml + run: ruby scripts/merge_coverage.rb merged-coverage.xml minitest-coverage/coverage.xml minitest-coverage/worker-*/coverage.xml vitest-coverage/cobertura-coverage.xml minitest-system-coverage/coverage.xml - name: Generate Coverage Report uses: clearlyip/code-coverage-report-action@v7 id: code_coverage_report_action diff --git a/agent-docs/testing.md b/agent-docs/testing.md index 849f404835b..8755794fbed 100644 --- a/agent-docs/testing.md +++ b/agent-docs/testing.md @@ -10,6 +10,22 @@ When modifying loader/action patterns: 4. Run `yarn run tsc --noEmit` to check for TypeScript errors 5. Test actual navigation flows to ensure data loading works +### Running Minitest in parallel + +Set `PARALLEL_WORKERS` to run the suite in that many processes, each with its own copy of the test database +(`intercode_test-0`, `-1`, ...). CI uses 4, which cuts the Minitest run from about 4½ minutes to about 1½. + +```sh +PARALLEL_WORKERS=4 bin/rails test +``` + +- On macOS the `pg` gem can crash (a segfault in `connect_start`) when the workers fork. Set + `PGGSSENCMODE=disable OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES` as well. +- Each worker writes its own coverage report to `coverage/worker-N/coverage.xml`, and the main process's + `coverage/coverage.xml` has what loading the app covered. Merge them with + `ruby scripts/merge_coverage.rb merged.xml coverage/coverage.xml coverage/worker-*/coverage.xml` (CI's coverage report + job does). The merged result is line-for-line identical to a single-process run's. + ## End-to-End Testing with Playwright Use Playwright (and Rails system tests) sparingly: browser-driven tests have been flaky in this project, so prefer a component test (see [Frontend Testing](frontend-testing.md)) wherever it gets close enough. Reserve end-to-end tests for the few flows that really need a real browser. diff --git a/test/test_helper.rb b/test/test_helper.rb index e57c562d174..8c3463cd5b2 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -58,9 +58,21 @@ class ActiveSupport::TestCase include FactoryBot::Syntax::Methods include ActionMailer::TestCase::ClearTestDeliveries - # Minitest is broken with parallelization on Ruby 3.1 - it's expecting methods to be methods but Rails is - # deserializing them as strings, so you can't actually see the unexpected error output - # parallelize(workers: :number_of_processors) + # Set PARALLEL_WORKERS to run tests in that many processes (each with its own copy of the test database). CI does, + # to use all of the cores on its runners; it's left off by default so that a plain `rails test` stays simple. + if ENV["PARALLEL_WORKERS"].to_i > 1 + parallelize(workers: ENV["PARALLEL_WORKERS"].to_i) + + # Each worker writes its own coverage report (coverage/worker-N/coverage.xml) instead of merging with the others as + # the run exits, which races the workers finishing. The main process's coverage.xml has what loading the app + # covered (class bodies and so on), so CI merges all of them with scripts/merge_coverage.rb. + parallelize_setup do |worker| + SimpleCov.command_name "#{SimpleCov.command_name}-#{worker}" + SimpleCov.use_merging false + SimpleCov.coverage_dir "coverage/worker-#{worker}" + end + parallelize_teardown { |_worker| SimpleCov.result.format! } + end class TestGraphqlContext def self.with_user_con_profile(user_con_profile, **attrs)