Skip to content

Runs: completed kits inherit a shared step’s running or failed state #192

Description

@aitianshi

Priority

extreme: the cause is execution state attribution. Native priority: Medium (P2): this changes displayed states, not execution or recorded scores.

Components

  • app/nullius (backend)
  • app/web (routes)

Evidence

A conformance launch has six kit jobs in one execution step. The supplied Runs screenshots show conversation 2/2, search 3/3, task 3/3, tools 5/5 and workspace 2/2 at 100%, but each remains running until the computer kit ends. Afterwards all six rows say failed, although only computer failed (0/8).

Execution.job_state uses the shared step exit; live_state uses the shared running step. Completed individual kit results cannot override either.

Screenshot

Two screenshots supplied with the report show the running and failed states described above. They are not hosted in GitHub; the counts and labels are transcribed in Evidence.

Reproduction

  1. Start an agent's conformance kits from its page.
  2. Let at least one kit finish cleanly while another kit is still running.
  3. Let the later kit fail.
  4. Read each kit row on Runs during and after the launch.

Expected

Completed, error-free kits read finished independently of the shared step. The failed kit and overall launch retain the failure.

Acceptance

  • A completed, error-free standard job in a shared execution step reads finished while another job continues.
  • A later failure does not relabel completed, error-free jobs as failed; the failed job and overall execution retain their failure.
  • Restoring an execution preserves those per-job states from its existing result files.
  • Incomplete, errored or unreadable records retain runner status; single-job and Harbor steps retain current behavior.
  • Backend and route regression tests cover the reported running and failure displays.

Dependencies and related

Implementation PR #193 depends on #188. Merge #188 first, then #193. PR #188 adds the computer kit; this state attribution defect is in Nullius execution. No matching issue or PR was found among open and closed reports before filing.

Implementation: #193. Validation: 86 backend and 31 route tests passed; focused Chromium checks covered running and failed states at 1280/390 pixels in light/dark themes. Awaiting review and merge.

Activity

  1. self-assigned this
    on Nov 17, 2026
  2. added
    bugSomething isn't working
    P0Fix now: unsafe, loses money or data, or nothing runs
    on Nov 17, 2026
  3. added theissue type on Nov 17, 2026
  4. added
    P2Planned: limited in scope, has a way around it, or a missing feature
    and removed
    P0Fix now: unsafe, loses money or data, or nothing runs
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P2Planned: limited in scope, has a way around it, or a missing featurebugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions