Repository navigation
test(judge): check the batch overlap with a barrier, not wall time - #135
Merged
harenome merged 1 commit intoNov 16, 2026
Merged
Conversation
The test asserted that 16 calls of 0.1 s end in less than 0.8 s. On a loaded machine the wall time grows, and the test failed although the calls ran at the same time. Each call now waits on a barrier of CONCURRENCY parties. The barrier lets the calls go only when that many are in flight at once. Serial calls break it at its timeout and the test fails. The batch has 2 * CONCURRENCY items, so the barrier fills twice for any value of CONCURRENCY. The count and order checks are the same.
aitianshi
added a commit
that referenced
this pull request
Nov 16, 2026
No linked issue: the owner asked for v0.1.1 as a release, so that the Hugging Face endpoint setting (#129) reaches PyPI. ## What Prepares v0.1.1. The version becomes `0.1.1`, and `docs/releases/v0.1.1.md` gives the highlights since v0.1.0, a Contributors section, the full changelog of every pull request in `v0.1.0..main` (#114, #117, #120, #122, #125, #127, #129, #132, #137, and #135 among the test and CI changes) and the installation notes, in the shape of v0.1.0's. Install examples, the release guide, the docs' release-notes link and menu, and the workflows' default tags name 0.1.1. The release tooling already accepts final `vX.Y.Z` tags and does not change. ## Touches CI and skills (the release and publish workflows' default tags) · docs and site ## Verified - Backend suite (`python3 app/tests/parallel.py`): 1991 tests in 116 modules, OK. Route suite: 428 tests in 41 modules, OK. - `catalogue_docs.py --write` and `diagrams.py --write` left no change. - `python3 app/packaging/release.py build --tag v0.1.1` built and verified `nullius_eval-0.1.1-py3-none-any.whl`, `nullius_eval-0.1.1.tar.gz` and `SHA256SUMS`; `release.py verify --tag v0.1.1` and `verify.py` on the wheel passed, including the installed-tool and profile-preservation checks. - The workflows have not run; they run only by hand on `main`. ## Risk Version strings and docs only. A wrong default tag in a workflow is an input the person running it can change, and `release.py` refuses a tag that does not match `pyproject.toml` or has no notes. Nothing is tagged, drafted or published by this change. Undo with a revert.
2 of 14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #109
What
test_calls_overlap_and_the_count_and_order_holdinapp/ui_tests/test_judge_batch.pyasserted that 16 calls of 0.1 s end in less than 0.8 s. On a loaded machine the batch takes longer, and the test fails although the calls overlap. Now each call waits on athreading.Barrierofjudge.CONCURRENCYparties. The barrier releases the calls only when that many are in flight at the same time. The batch has2 * judge.CONCURRENCYitems, so the barrier fills two times for every value ofCONCURRENCY. If the calls run one at a time, the barrier breaks at its 10 s timeout and the test fails. The timeout only stops a hang, and the test measures no duration. The count and order check does not change. No production code changes.Touches
Routes (tests only).
Verified
Base is
c7af06d5. The load runs are in a rootless podman container,--cpus 1, with busy-loop shell processes in parallel. Each repetition runspython3 -m unittest ui_tests.test_judge_batchin a new interpreter.CONCURRENCYset to 6 and to 3 in a scratch copy:RunBatchpasses for both. With a fixed count of 16 items andCONCURRENCY6, the test fails withthreading.BrokenBarrierError.run_batchchanged to use the built-inmapin place ofpool.map(one call at a time). The test fails withthreading.BrokenBarrierErrorafter 10 s.python3 app/tests/parallel.py app/ui_tests: 322 tests in 40 modules, OK, on this branch and on the base.Risk
Low. The change is in one test. Revert the commit to undo.