Skip to content

loopguard: count hatch test as a test run - #305

Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix-loopguard-hatch-test
Open

Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix-loopguard-hatch-test

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

Hatch 1.10 and later run a project's tests with hatch test: pytest in Hatch's own test environment (src/hatch/cli/test/__init__.py in pypa/hatch). The loop guard read hatch run pytest as a test run but not hatch test, so a failing hatch test after the last edit left the turn counted as ended green.

Fix: hatch test is a verification, with its options and any path. With -s/--show it only shows the environments and runs nothing, so it is still not a test run, also when the flag is combined with others (-ps, -cs). After -- the flags are pytest's, so hatch test -- -s counts. hatch test --help, hatch tests and hatch build are still not runs. The alternative is added at the end of VERIFY_RE, apart from the lines #284 and #294 change; the branch merges with both, and with #295 and #296.

Tests, in a new tests/test_loopguard_hatch_test.py:

  • 9 hatch test commands, each a verification, and a failing run after an edit is not ended green; all 9 fail on main;
  • 8 commands that stay non-runs, the --show forms among them.
python -m pytest tests -n 4 --dist worksteal -rfEs --tb=line
1418 passed, 41 skipped

The skips need Windows, a real agent CLI (Codex, Hermes, opencode) or aider, or a non-root user.

Fixes #301

Hatch 1.10 and later run a project's tests with `hatch test`: pytest in Hatch's own test
environment (`src/hatch/cli/test/__init__.py`). The loop guard read `hatch run pytest` as a test
run but not `hatch test`, so a failing `hatch test` after the last edit left the turn counted as
ended green.

`hatch test` is now a verification, with its options and any path. With `-s`/`--show` it only
shows the environments and runs nothing, so it is still not a test run, also when the flag is
combined with others (`-ps`); after `--` the flags are pytest's, so `hatch test -- -s` counts.
The alternative is added at the end of `VERIFY_RE`, apart from the lines other changes edit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

loopguard: hatch test is not counted as a test run

1 participant