Skip to content

Flaky on Windows arm64: review prompt shim tests time out after 10 seconds #367

Closed

Description

@mubeda

What happened

On main (commit 706d63b), a rerun of "Native desktop - Windows arm64" in CI run 43523947929 failed in the step "Run review prompt argument tests (Windows)".

Four tests in apps/server/tests/windows_terminal_shims.rs failed, all at line 172:

  • review_prompt_shim_passes_ampersand
  • review_prompt_shim_passes_backslash_before_quote
  • review_prompt_shim_passes_caret
  • review_prompt_shim_passes_bang

Each one reported called Result::unwrap() on an Err value: Elapsed(()). That is the 10-second tokio::time::timeout that waits for the marker in the terminal output. Result line: 6 passed; 4 failed.

Why this looks flaky

The same commit passed this step in the first attempt of the same run, and the same code passed it on the pull request run 43496452314. The failing attempt had a full Rust cache hit.

Measured so far: 1 failure in 3 runs of this step on Windows arm64.

Not known

  • Whether the terminal never printed the marker or printed it late. The test unwraps the timeout before it can print the collected output.
  • Whether Windows x64 is affected.

Suggested first step

Make the timeout failure print the collected output and the exit state, so the next failure shows which case it is.

Activity

  1. mubeda commented on Nov 14, 2026

    @mubeda
    OwnerAuthor

    Status after pull request 370 (2026-11-14): not reproduced, so this stays open.

    • In the failing attempt the four tests that timed out were the first four libtest started; each waited the full 10 s and the six started after them passed within 5 s. The step normally takes 2.4 to 4.6 s. That points at a slow first spawn, and it is not proven.
    • Load test in a Windows arm64 VM (6 cores): 30 runs of the step under four busy processes, 0 failures. Those runs were warm, so they do not recreate a cold first spawn on a 4-core runner.
    • The pull request makes a timeout print the terminal output collected so far and the exit state, so the next failure shows whether the marker was late or never printed.
  2. mubeda commented on Nov 14, 2026

    @mubeda
    OwnerAuthor

    Reproduced and fixed in pull request 370 (2026-11-14).

    • A temporary workflow ran the ten review prompt tests as the first terminal spawn on fresh Windows arm64 runners. With the 10-second deadline, 1 of 80 cold runs failed with this issue's symptom: the first four tests timed out and the six started after them passed.
    • The timeout message, added by the same pull request, read output="\u{1b}[?9001h\u{1b}[?1004h", exit=None: the pseudoconsole was up and the shim had neither printed nor exited. That run's batch took 19.8 s; the other 79 took 3.8 to 6.7 s and a warm rerun about 1 s. So the runner stalled process start-up for about 20 seconds.
    • The shim helper's deadline is now 60 seconds. These tests check which arguments a shim receives, not how fast a runner starts processes, and a shim that never prints still fails with the terminal output.
    • 240 cold runs with the new deadline all passed, the count chosen in advance for a 1 in 80 rate.

    Not known: what stalls the runner. No stalled run occurred among the 240, so the fix was not seen turning a stalled run into a pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions