Skip to content

Optimize Python test runner - #205

Merged
gloveboxes merged 1 commit into
mainfrom
optimize-runall-runners
Sep 29, 2026
Merged

gloveboxes merged 1 commit into
mainfrom
optimize-runall-runners

Conversation

@gloveboxes

@gloveboxes gloveboxes commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Optimizes the native Python runner while retaining its existing build isolation, baseline verification, and timeout/process-tree cleanup behaviour.

  • Bootstrap the extended-test submodule before main test discovery and dispatch, so a missing checkout is initialized or reported before compiling the main corpus.
  • Discover main and extended cases up front, then submit both to one bounded queue with a shared throttle and global fail-fast boundary. Main cases are queued first, so extended cases fill the main suite’s tail without competing during its highest-throughput phase.
  • Keep extended builds in their dedicated extended-tests build subtree and exclude their results from main performance-baseline and report accounting.
  • Cache tool-path resolution and compiled performance-output patterns.
  • Skip collecting cycle/size metrics for extended tests and extra scenarios where those metrics are discarded.

Validation

  • python3 -m py_compile scripts/runall.py
  • Focused main-app plus complete extended-corpus full-mode run passed.
  • Verbose main-app plus complete extended-corpus run passed.
  • Full --mode full --extended --failures-only run passed: 483 main apps and all runnable extended tests, with no performance regressions.

The full run occurred while host filesystem/indexing activity was elevated, so the PR intentionally makes no performance claim from that final measurement.

@gloveboxes
gloveboxes force-pushed the optimize-runall-runners branch from 6e41af9 to 4f68a41 Compare September 29, 2026 03:40
@gloveboxes gloveboxes changed the title Optimize Python and PowerShell test runners Optimize Python test runner Sep 29, 2026
Reduce Python runner overhead and treat the optional extended suite as part of one bounded test corpus.\n\nBootstrap the extended submodule before main-test discovery, construct its work items up front, and append them to the same worker queue as the main apps. This gives both corpora one throttle and fail-fast boundary while preserving separate build directories and excluding extended results from main performance accounting.\n\nAlso cache tool resolution and compiled output patterns, and avoid collecting metrics that are not consumed.
@gloveboxes
gloveboxes force-pushed the optimize-runall-runners branch from 4f68a41 to e205abb Compare September 29, 2026 03:52
@gloveboxes
gloveboxes merged commit 1dc57ab into main Sep 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant