Repository navigation
Write HDF5 files of unit tests to tmp_path (Pytest fixture) - #205
Merged
Merged
Conversation
The tests of test_gallery.py shared the file domain.h5 in the working directory, and relied on the xdist_group marker to run on the same worker. pytest 9.2.0 breaks this marker with pytest-xdist 3.8.0, so these tests ran in parallel and failed randomly. Each test now writes to its own tmp_path, and the marker is removed. The same is done for test_topology_1.
yguclu
changed the title
tmp_path (Pytest fixture)
No test uses the xdist_group marker anymore, so the default --dist load scheduling of pytest-xdist is enough.
Member
Author
Thanks for the review! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Write the HDF5 files of the unit tests to the
tmp_pathfixture instead of the working directory, so that the tests can run in parallel without thexdist_groupmarker (fixes #204).Problem
The six export/import tests of
sympde/topology/tests/test_gallery.pyall wrote and read the same filedomain.h5in the working directory. They relied on@pytest.mark.xdist_group('h5py')and--dist loadgroupto run one after another on the same worker.pytest 9.2.0 (released 2026-10-27) changed how node IDs are stored, and pytest-xdist 3.8.0 can no longer assign tests to their group (see pytest-dev/pytest-xdist#1386, fixed in pytest-dev/pytest-xdist#1388 but not released yet). The grouped tests were therefore spread over the workers and failed randomly with
BlockingIOErrororOSErrorwhen two of them openeddomain.h5at the same time. This is what currently makes CI fail in #201.Changes
test_gallery.pythat exports a domain writes it to its owntmp_path, and thexdist_group('h5py')markers are removed.test_topology_1intest_topology.pywritesomega.h5totmp_pathtoo.teardown_moduleis no longer needed and is deleted.--dist loadgroupfrom the pytest commands in CI and in the README, since no test uses thexdist_groupmarker anymore.[DEVELOPER]entry to the CHANGELOG.Validation
With pytest 9.2.0 and pytest-xdist 3.8.0,
pytest -n auto --dist loadgroup --pyargs sympde -rapassed 5 times in a row (180 passed, 2 skipped) and left no files in the working directory. Onmaster, the same command failed in all 3 runs, with 2 or 3 failing tests each time. CI passes in all jobs with the final commandpytest -n auto --pyargs sympde -ra.