Skip to content

Run the tests on pull requests - #11

Merged
greenido merged 1 commit into
mainfrom
ci/test-on-pull-requests
Sep 25, 2026
Merged

greenido merged 1 commit into
mainfrom
ci/test-on-pull-requests

Conversation

@greenido

Copy link
Copy Markdown
Owner

Stacked on #10 → which is stacked on #9. Stacked rather than branched from main only because it rewrites the test:* scripts in package.json that #9 and #10 each added to; branching separately would conflict with both. GitHub retargets it as they merge.

The gap

deploy.yml triggers only on push: branches: [main]. So a pull request's checks were a GitGuardian secrets scan and nothing else — a regression was caught by the deploy after the merge, not by the PR that introduced it. Both PRs in this stack show exactly one check.

ci.yml now runs lint, typecheck, test and build on every pull_request. Each step runs even when an earlier one fails (if: !cancelled()), so one push surfaces every problem instead of just the first, with an explicit gate at the end so the job can't go green with a red step. Pushes to main stay covered by deploy.yml, which now also typechecks — splitting them means no commit gets tested twice.

Both workflows read the Node version from .nvmrc rather than hardcoding '22', which had already drifted from the pinned 22.13.0.

Two things that made CI worth less than it looked

The suites were chained with &&. The first failure hid the other eight — one round trip through CI per broken suite. scripts/run-tests.js runs all of them and prints a summary:

────────────────────────────────────────────
  ✓ elevation
  ✗ zza-fails  (exit 1)
  ✓ zzb-runs-anyway
────────────────────────────────────────────
1 of 11 suite(s) failed.

Verified with throwaway probes named so the passing one sorts after the failing one — it ran, and the job still exited 1.

Each suite had to be registered by hand. Easy to forget, and two never were: tests/new-features.test.ts and tests/weather-api.test.ts had gone unrun for their entire existence. Suites are now discovered from tests/, so a *.test.ts file is picked up by existing. Network-hitting suites declare themselves with an @network marker in their header and are excluded from yarn test — CI must not go red because NOAA is having a morning.

What running those two for the first time found

weather-api.test.ts could not compile at all:

tests/weather-api.test.ts(616,5): error TS1343: The 'import.meta' meta-property is
only allowed when the '--module' option is 'es2020' | ... | 'nodenext'

It guarded its entry point with import.meta.url === \file://${process.argv[1]}`, which is invalid under the CommonJS module setting tsconfig.test.jsonuses. Note this passedyarn typecheck, because the app's tsconfig.jsonsetsmodule: esnext— it only broke when actually executed. Fixed torequire.main === module`.

All three network suites now pass: new-features, providers, weather-api (4 tests).

Script changes

Nine near-identical test:* entries collapse into one filtered entry point:

yarn test              # all 9 offline suites
yarn test scoring      # one suite, matched by substring
yarn test:network      # the 3 live-API suites
yarn typecheck         # new — tsc --noEmit, covers the test files too

README updated for all of it.

Verification

yarn lint, yarn typecheck, yarn test (9 suites, 176 assertions), yarn test:network (3 suites) and yarn build all pass locally. Exit codes confirmed in both directions: 1 with a failing suite, 0 without.

This PR validates itself — being a pull_request, ci.yml runs against its own branch, so the checks on this PR are the new workflow.

Note

ci.yml uses pull_request, not pull_request_target, so a fork's PR runs with a read-only token and no secrets. That's the right default here; it does mean a first-time contributor's run may need approval under your repo settings.

🤖 Generated with Claude Code

@greenido
greenido added this pull request to stack #12 September 25, 2026 15:03
@greenido
greenido force-pushed the ci/test-on-pull-requests branch from a6e7550 to 7026783 Compare September 25, 2026 15:03
Base automatically changed from fix/abort-and-cache to main September 25, 2026 15:04
Nothing did. deploy.yml triggers only on a push to main, so a PR's checks
were a secrets scan and nothing else: a regression was found by the
deploy that followed the merge, not by the PR that caused it.

ci.yml runs lint, typecheck, test and build on every pull request. Each
step runs even when an earlier one fails, so one push surfaces every
problem rather than the first, with an explicit gate at the end so the
job cannot go green with a red step. Pushes to main stay covered by
deploy.yml, which now also typechecks. Both workflows read the Node
version from .nvmrc instead of hardcoding a major.

Two things had to change for that to be worth much.

The suites were chained in package.json with `&&`, so the first failure
hid every suite after it -- a round trip through CI per broken suite.
scripts/run-tests.js runs all of them and lists the results.

And each suite had to be registered by hand, which is easy to forget:
tests/new-features.test.ts and tests/weather-api.test.ts never were, and
had gone unrun for their whole existence. Suites are now discovered from
tests/, so a file is picked up by existing. Ones that reach the network
mark themselves `@network` and are excluded from `yarn test`, since CI
must not fail because NOAA is having a morning.

Running those two for the first time found weather-api.test.ts could not
compile at all: it guarded its entry point with `import.meta.url`, which
is invalid under the CommonJS module setting the test config uses. Fixed
to `require.main === module`; all three network suites now pass.

Nine per-suite scripts collapse into `yarn test <name>`, which filters by
substring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@greenido
greenido force-pushed the ci/test-on-pull-requests branch from 7026783 to 6462368 Compare September 25, 2026 15:04
@greenido
greenido merged commit 5cddae4 into main Sep 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant