Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .claude/skills/babysit-pipeline/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,16 @@ nothing failed.
put the spec on `rescue_specs.txt`: the driver's dispatch
auto-closes every open PR of the pair, so a regeneration fired while
a repair is mid-flight throws that work away.
- **A `Review: PR #N` run failing at "Verify both theme renders exist"
with staging complete is a lost download, not a missing render.** The
PR keeps no verdict label, so no watchdog case matches and the driver
waits an hour before calling the spec stalled. Check
`gs://anyplot-images/staging/<spec>/<lang>/<lib>/` — 18 objects means
the generate/repair upload was fine — then `gh workflow run
impl-review.yml -f pr_number=N` once, only when no `Review: PR #N`
run is active. The download step retries since the 2026-09-09 fix;
a repeat on the same PR after that means the staging folder really is
empty and the pair needs a fresh generate.
- **A `Merge: PR #N` run that fails AFTER "Merge PR to main" leaves a
silent hole: the squash is on main (metadata pointing at production
URLs, the driver counts the pair as done) but the images are still in
Expand Down
17 changes: 16 additions & 1 deletion .github/workflows/impl-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,22 @@ jobs:
LANGUAGE: ${{ steps.lang.outputs.language }}
run: |
mkdir -p plot_images
gsutil -m cp "gs://anyplot-images/staging/${SPEC_ID}/${LANGUAGE}/${LIBRARY}/*" plot_images/ 2>/dev/null || true
# Retried: a single lost transfer used to be swallowed here (`2>/dev/null
# || true`), and the next step then failed on an empty directory with
# no label set — no watchdog case matches that, so the PR sat until
# someone re-dispatched the review by hand (#11360 on 2026-09-05,
# #11678 on 2026-09-09; staging was complete both times). Keep stderr:
# a transfer blip and a missing staging folder need different answers.
SRC="gs://anyplot-images/staging/${SPEC_ID}/${LANGUAGE}/${LIBRARY}/*"
for attempt in 1 2 3; do
if gsutil -m cp "$SRC" plot_images/ 2> /tmp/gsutil.err \
&& [ -f plot_images/plot-light.png ] && [ -f plot_images/plot-dark.png ]; then
break
fi
GERR=$(head -c 300 /tmp/gsutil.err | tr '\n' ' ')
echo "::warning::staging download incomplete (attempt ${attempt}/3): ${GERR:-no error output}"
[ "$attempt" -lt 3 ] && sleep $((attempt * 10))
done
ls -la plot_images/

- name: Verify both theme renders exist
Expand Down
10 changes: 10 additions & 0 deletions changelog.d/impl-review-staging-download-retry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
### Fixed

- **`impl-review` retries the staging download instead of reviewing an empty directory** — the
step swallowed a lost `gsutil cp` (`2>/dev/null || true`), the render check right after it
then failed on nothing, and because that failure set no `ai-review-failed` label, no watchdog
case picked the PR up: it waited for a manual re-dispatch (#11360 on 2026-09-05, #11678 on
2026-09-09, staging complete both times). The download now tries three times with a short
backoff, stops as soon as both theme renders are on disk, and keeps `gsutil`'s stderr in a
warning per failed attempt, so a transfer blip heals in the same run and a genuinely empty
staging folder still fails loudly at the render check. (#11697)
Loading