Skip to content

chore(ci): run the full suite on sync and release PRs - #490

Merged
shuv1337 merged 3 commits into
integration-v2from
sync-pr-tests
Nov 13, 2026
Merged

shuv1337 merged 3 commits into
integration-v2from
sync-pr-tests

Conversation

@shuv1337

@shuv1337 shuv1337 commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Why

Until now the full suite in test.yml (unit and e2e) ran only from publish.yml, pushed v* tags and workflow_dispatch. Pull requests got only typecheck.yml. As a result, the upstream v2.3.0 sync (#489) merged without unit tests, and its first unit failure (the PowerShell ShellTool timeout) surfaced in the 2.3.0-shuv.1 publish run.

What

test.yml now also runs on pull_request (opened, synchronize, reopened, edited, labeled). Both unit and e2e share one job-level if through a YAML anchor. On pull requests they run only when one of these is true:

Both jobs run because both gate publish. The aim is that publish never finds a new failure.

Concurrency. This mirrors typecheck.yml. Pull requests share test-pr-<number> with cancel-in-progress: true. Every other run gets test-run-<run_id>, so release runs never share a group and are never cancelled. The test- prefix keeps the group apart from any caller's group: publish.yml's is publish-<ref>-<version>, and typecheck.yml's is typecheck-….

Description edits restart a gated PR's suite (accepted and documented). My first commit skipped body-only edits and unrelated labels, and gave those runs a group of their own. On this PR that showed a real problem. A PR's checks list (statusCheckRollup), and branch protection, keep only the newest unit and e2e check runs. The skipped run from a description edit replaced the real, still-running suite, and gh pr checks reported unit skipping. After a real failure, that would hide the failure. GitHub cannot filter pull_request triggers by edited field or label name, so there is no cheap way to avoid creating that run. Every event on a gated PR now restarts the suite instead. The comment in test.yml and the AGENTS.md note explain why.

Cancelled unit runs stop promptly. unit's three build smoke steps (compiled service lifecycle, Node build runtime, Node build) were if: always(), and always() also runs on cancellation. On this PR, a superseded run kept building after cancellation and held test-pr-490, so its replacement stayed pending until I force-cancelled the old run about 10 minutes later. Those steps now use if: ${{ !cancelled() }}, the idiom e2e already uses. They still run after a unit-test failure, but stop on cancellation. Release runs are never cancelled by concurrency, so nothing changes for them; a manual cancel of a publish run now stops promptly. e2e's artifact upload keeps always() because it is quick.

Docs. The header comments in test.yml and typecheck.yml now describe this. The sync procedure in AGENTS.md now requires sync and release PRs to pass unit and e2e before merge, and mentions full-tests for other risky PRs. The CI line in PLAN-fresh-v2-rewrite.md is updated too. The dated docs/sync-*.md audits are historical, so they are left unchanged.

workflow_call still runs both jobs

In a reusable workflow, the github context is the caller's. publish.yml is triggered by workflow_dispatch, so github.event_name in test.yml is workflow_dispatch when publish.yml calls it. The gate's first clause, github.event_name != 'pull_request', is therefore true, and unit and e2e run unconditionally, just as before. Tag pushes (push) and direct dispatches pass the same clause. The concurrency expression's pull_request branch is false there too, so those runs get the unique group test-run-<run_id> and nothing cancels them. Only a caller that is itself triggered by pull_request would hit the PR gate, and none exists.

Verification

  • actionlint 1.7.12 is clean on test.yml. Its only finding is a pre-existing shellcheck SC2016 info note in typecheck.yml's inline bun -e script, which this PR does not change. Prettier and git diff --check are clean on the changed files.
  • Trigger proof: this PR's title does not match the gate, so it was opened without labels.
    • The opened run 43504043883 skipped both jobs.
    • Adding full-tests started 43504123544, with unit and e2e in progress.
    • A description edit under the first design gave 43504207604, which skipped and hid the running suite as described above.
  • Final check runs on the head commit are listed below.

Notes

  • No feature flag: this is a CI-only change.
  • Coordination: the PowerShell timeout fix is in flight separately on t3/shell-timeout-regression. This PR does not touch packages/core/test/tool-shell.test.ts.

Check runs

Results on head 910bb86 (the synchronize run with full-tests), run 43504720265:

  • e2e: pass (20m).
  • unit, attempt 1: failed on one test, ShellTool stopped > stops a command the user stopped, keeps its file and fails the call with the tail, file and shell ID (spawned) (packages/core/test/tool-shell.test.ts:2285). It is a timing race: the killed command's file held first\n because second had not been written yet. This is not the known PowerShell case, which did not fail here. It is pre-existing, in the file the in-flight shell fix owns, and unrelated to this PR, so I left it alone.
  • unit, attempt 2 (rerun of the failed job): pass (12m46s).
  • typecheck, hygiene, plugin-browser and Socket all pass.

This body edit itself starts a fresh unit + e2e run, as documented above; its result is in a comment below.

Co-authored-by: Cursor <cursoragent@cursor.com>
@shuv1337 shuv1337 added the full-tests Run the full test suite (unit and e2e) on this PR label Nov 13, 2026
shuv1337 and others added 2 commits November 13, 2026 12:45
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@shuv1337

Copy link
Copy Markdown
Owner Author

The description edit restarted the suite as documented: run 43511819370 (event pull_request / edited, head 910bb86) passed unit on the first attempt and e2e. The ShellTool stopped race from the earlier attempt did not recur.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-tests Run the full test suite (unit and e2e) on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant