Skip to content

fix(execution): preserve finished kit states in shared steps - #193

Open
aitianshi wants to merge 2 commits into
mainfrom
fix/kit-run-status
Open

aitianshi wants to merge 2 commits into
mainfrom
fix/kit-run-status

Conversation

@aitianshi

@aitianshi aitianshi commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #192

Depends on #188. Merge #188 first, then #193.

What

Conformance writes several kit jobs in one execution step. The shared running state and exit code made already completed kits remain running, then labelled all of them failed when a later kit failed. For standard jobs sharing a step, execution now reads a complete, error-free result as that job's own finish, with its own timestamp. This also works after restoring an execution from disk. The overall execution keeps its failure.

Incomplete, errored, unreadable or partially written results retain runner status. Harbor and single-job steps keep their existing exit semantics. Scores and execution commands are unchanged.

Touches

Backend (execution) and backend and route regression tests.

Before and after

Synthetic three-kit fixture, with two clean completions and one failing kit:

Moment Before After
Last kit running 3 running 2 finished, 1 running
Last kit failed 3 failed 2 finished, 1 failed

Before/after screenshots were captured and reviewed locally at 1280 and 390 pixels in light and dark themes. Image attachments are not uploaded to GitHub; this PR records the visible labels above.

Verified

  • 86 backend tests passed: test_execution_job_state, test_execution, test_execution_stop, test_execution_stop_restart.
  • 31 route tests passed: test_runs_by_launch, test_routes.
  • Focused Chromium check used the regression fixture through Flask's test client, comparing the original and changed job_state implementations. Both running and failed states checked at 1280 and 390 pixels, light and dark; no page JavaScript errors. At mobile width the existing layout hides the State column; the summary shows the corrected counts.
  • Regression coverage includes restored executions, incomplete/error/cancellation counts, malformed files, and preservation of Harbor and single-job state behavior.
  • git diff --check passed.
  • No paid conformance run or Docker benchmark was launched; validation uses synthetic results written by the standard's own result writer.

Risk

The state reader now reads one small result file for a standard job sharing a step. A wrong completion decision could hide a failed process behind a finished job row. It requires a matching job ID, a finish timestamp, a positive planned count fully completed, and zero errored, cancelled, running or pending trials; otherwise it falls back to runner state. No record format or migration changes. Revert this commit to restore the previous display.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown

🟡 PR judge · 7 of 7 yes · #193 ↔ #192

Merge risk 🔴 MAJOR the judge's reading of the diff; app/nullius/execution.py is in the path of every run
Priority 🔴 EXTREME the deepest path is app/nullius/execution.py
Question Verdict
Changes do what the issue asks · 5 of 5 criteria ✅ yes
Issue and pull request describe the same change ✅ yes
Description matches the diff, no more, no less ✅ yes
Diff stays in scope ✅ yes
Tests cover the change, or Verified says how ✅ yes
Docs follow the change ✅ yes
Risk named in the description ✅ yes
Before and after shown –
Acceptance criteria, 5 of 5 met
# Criterion Verdict
1 A completed, error-free standard job in a shared execution step reads finished while another job continues. ✅ met
2 A later failure does not relabel completed, error-free jobs as failed; the failed job and overall execution retain thei… ✅ met
3 Restoring an execution preserves those per-job states from its existing result files. ✅ met
4 Incomplete, errored or unreadable records retain runner status; single-job and Harbor steps retain current behavior. ✅ met
5 Backend and route regression tests cover the reported running and failure displays. ✅ met
Checks the script runs itself, 4 of 4 pass
No machine attribution trailer in the commits or the description ✓
New code under app/, docs/, .github/ or .claude/ ✓
No Flask import under the backend package ✓
No key-shaped string in the diff ✓

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runs: completed kits inherit a shared step’s running or failed state

1 participant