Repository navigation
feat(traces): rework the trial page around what a trial recorded - #173
Merged
Merged
Conversation
The strip repeated what the tabs and the Reward card already say. The time bar loses its paragraph: the total moves to the heading, a phase that took no time has no chip, and each chip's tooltip says what that phase holds on a Harbor or a standard run. Other on a standard run is the cleanup after the check, not a wait for the judge, which is in Verify.
This comment has been minimized.
This comment has been minimized.
A check that took under 0.05 s lost its chip, so an instant check read as none. A recorded phase now keeps its chip; only a segment needs time to draw, and Other, which is computed, still shows only when there is some.
This comment has been minimized.
This comment has been minimized.
One decimal turned 0.05 s into 0.1s and hid differences the clock had measured. Seconds and minutes now carry two decimals.
The "Shape of the run" title goes, and the time bar becomes a tile in a grid that other summaries can join.
This comment has been minimized.
This comment has been minimized.
A trajectory with no per-step usage, no reasoning total and no tool call opened the panel and left it empty.
This comment has been minimized.
This comment has been minimized.
The Artifacts tab lists only the records a trial wrote, each with Read and Download. A record's page gains Copy and Download, both of the whole record redacted as the reader shows it, and its back button returns to the Artifacts tab rather than the overview.
This comment has been minimized.
This comment has been minimized.
…ting or empty parts The Artifacts and Conversation tabs lose the heading their tab already gives. The benchmark's judge card goes: nothing writes the record it read. The judge tabs keep their experimental badge, with who answered and when as its tooltip, and drop the description, the clean hygiene line and the flag summary; a flag's question lines up under its name.
…al's name The trial's name gets smaller. The benchmark and its runner share a card, the agent with its mark, version and model another, and the reward, the agent's time and the steps a strip beside them. Tool calls leave the head, and a value the trial does not have is not drawn.
This comment has been minimized.
This comment has been minimized.
- The head's strip let its cells shrink under their text: six cells on a SWE-bench trial ran into each other between 820 and 1180 px and at phone width. Cells keep their width now; the strip moves under the cards, then wraps. - The Artifacts tab and a record's tools scrolled the page sideways at phone width: the buttons wrap. - An imported trial was said to be Harbor's, its importer's name labelled a benchmark. It says where it was imported from, and nothing else. - A value the trial has not has no cell, as the comment claimed: no Reward, check or total shown as a dash, no strip with nothing in it. A graded score has a cell of its own. - Two decimals were not everywhere: the run page's script redrew stage times with the old rule a moment after load, a span's minutes kept one decimal beside the head's two, and the shell's show kept one. - Download and Copy cut a record at the reader's 16 MiB with no word of it: the saved name says partial. The route answers nosniff. - Copy could say copied with nothing on the clipboard where a browser drops the click across the fetch: the clipboard is handed the pending text inside the click, and a failure is said. - The judge tab ran its notes together (2 refused state truncated). - A runner's Other was called the work after the check; on an older runner's trial it is the wait for the judge. It is what the phases leave of the trial, on both. - Left behind and now gone: the second judge.json fixture, the card and summary styles, four values the judge panel computed for a line that is not drawn. - Tests said more than they held: one renamed, one docstring made true, the page half of an absent trajectory restored. The head, the cut download, the duration twins and the judge tab's notes have tests.
The segments' flex weights were the phases' seconds: under a second in all they summed below 1, and the bar drew only that share of its width. The weights are whole milliseconds now.
This comment has been minimized.
This comment has been minimized.
Ten commits landed on main since this branch left it; three files conflicted. - traces/trial.html: main put the attempt and try tabs of a task run several times between the subtitle and the cards, both of which this branch replaced with the trial head. The tabs stay, above the head: which attempt, then what ran and how it went, the order main reads them in. - web/traces.py: both sides added an import on the same line; both stay. - docs/field-guide.md: main added the n_retries row above the row this branch rewords; both stay.
main keeps a retried attempt's earlier tries, each with a page of its own. The record page reads them; the download route this branch adds did not, so Download answered 404 on a try's Artifacts tab and Copy said it failed. It resolves a try as the record page does.
… on All attempts main's attempt tabs came in above the trial's head, and its All attempts page kept the large name and the subtitle this branch replaced on the trial page: the tabs sat 60 px lower there than on an attempt's page, and the name changed size from one tab to the next. Both pages now read the same way: the name, what ran (one macro, traces/_head.html), then the attempt and try tabs. On a trial's page that attempt's numbers follow the tabs; a trial with no tab keeps its one row. What is above the tabs is the same for every attempt of the task, so a tab changes what is under it and the tabs stay where they are. The title row is as tall with its outcome tag as without.
This comment has been minimized.
This comment has been minimized.
aitianshi
added a commit
that referenced
this pull request
Nov 17, 2026
#173 merges first and words each phase's hover by the kind of run, a standard run's setup included; this branch's own wording of the same line would only conflict with it.
🔴 PR judge · 7 of 8 yes · #173 ↔ #179
Acceptance criteria, 8 of 8 met
Checks the script runs itself, 4 of 4 pass
some of the diff was cut before the judge read it |
aitianshi
added a commit
that referenced
this pull request
Nov 17, 2026
Closes #180 ## What A standard trial of an agent driven over ACP had no agent setup: the agent's process start, `initialize` and `session/new` (which also starts the benchmark's MCP servers) ran inside the window written as `agent_execution`. The trial page read "Setup —" and its Agent time carried the start. The run now notes when the agent's session opens and writes Harbor's `agent_setup`, from the end of the environment's setup to the open session. `agent_execution`, and the window of the trial's trace, start there. The second commit corrects what an independent review of the first found: - A start that ends with no session (the agent never answered, exited, or the run was stopped) was still written as the agent's run. It is its setup, with no `agent_execution`, as Harbor writes a failed agent setup. - A capability got a setup of a millisecond: the time to spawn a process that answers nothing until its first request. A capability, a command agent, an Original setup and a control have no setup phase. - A task's internet door opened inside the agent's setup and closed inside its run. The agent's phases begin once it is open and end before it closes. - The live feed still sent one duration, timed from before the setup, and no `agent_setup` phase, so the live board disagreed with the trial page. It emits the phase, starts the agent's limit when the session opens, and takes every duration from the trial's result. - The trace-window test could not fail. It pins the window now, and the run's side has tests of its own. The last two commits take `main` (ten commits: a unit's VM, forced time limits, attempts and retries among them): - The unit loop reads: the unit's VM, the task's door, then the agent's setup and its run. - `main` renamed the turn's limit to the agent's whole work on a unit (`agent_limit`) and starts its clock once the session is open. The board's `agent_run` phase is emitted there with that same limit, a run's `--timeout agent` included; the setup keeps ACP's two start limits. The line this branch had added still called the old name and merged without a conflict: every ACP unit would have ended in an error. - The trial page's Setup hover is left to #173, which merges first and words it for a standard run. Installing the agent stays a stage of the run, not of a trial. ## Touches - the standard: `standard/run.py` (the unit loop, `drive_agent`), `standard/job.py` (`write_trial`, `feed_durations`), `standard/acp.py` (its two start limits, named) - docs: `field-guide.md` (the phase rows, shorter) - tests: `test_standard_trace.py`, `test_standard_run.py` ## Before and after The same recipe (`agent-task-v1`, the reference agent, a stand-in endpoint answering in 2 s) run on #173, which merges first, and on #173 with this branch: the trial page this lands on. Before, the agent's start sits in Agent; after, it is Setup. | Before | After | | --- | --- | | <img width="440" alt="Before: no Setup phase" src="https://github.com/user-attachments/assets/0bb4e268-444d-43ce-8ac4-737e3809b048" /> | <img width="440" alt="After: Setup 0.06s apart from Agent" src="https://github.com/user-attachments/assets/429ff985-957c-4ff0-b124-41308ce3fe15" /> | ## Verified - On the merge with `main` (`49d32d1`): backend suite 2236 OK, route suite 547 OK. With #173 merged first: 2236 and 562 OK. - Seven of the new or extended tests fail on the first commit and pass on the second: the four windows on shared stamps with the feed told the same seconds; a start that never answers; a capability; a conversation the upstream starts again; the task's door (a stand-in door that takes 0.4 s to open and to close); a failed start and a trial with no clock in `write_trial`. The trace's window is pinned as well: it starts at the open session, and a model call made during the setup widens it and is never clamped. - After the merge, a new test forces `--timeout agent` and reads the feed: the setup carries ACP's 300 s, the run the forced limit. With the old limit's name left in, the setup test ends in an error. - A real local run on the merged code (`nullius standard run agent-task-v1 reference --limit 1 --unsandboxed --timeout agent=45`, stand-in endpoint): `result.json` holds `environment_setup` 0.046 s, `agent_setup` 0.054 s, `agent_execution` 2.056 s, `verifier` 0.001 s, back to back; `events.jsonl` holds the phases `env_setup` (600 s), `agent_setup` (300 s), `agent_run` (45 s), `verify` (600 s) and a `trial.end` with the same four durations; the trace's session is 2.056 s. - Not verified: the docker tier and the Boat backend were not run, so a real container's internet door and a unit's VM were not timed; the door's order is held by the test with the stand-in door, and the VM wraps the same block. ## Risk - Agent time on standard runs written before and after this differs by the agent's start; a comparison across that line reads slightly different. Stored results are not rewritten, and the field guide says so. - On a task with an internet door, the environment's time grows by the door opening and the agent's run shrinks by its opening and closing. - A failed start has no `agent_execution`. Readers already meet a trial without one (a unit that fails in its environment's setup). - The live board shows "agent setup" while the agent starts, with a limit of 300 s (ACP's two start requests), then the agent's limit from its open session. - Merges after #173. Merged alone, the Setup hover on a standard trial would still read "installing the agent" until #173 lands. - Undone by a revert.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #179
What
A trial's page spent its first screen on parts that said nothing or said it twice, and hid some of what the trial recorded. This reworks it into a summary of the trial.
secsfilter (every page that uses it), the run page's script, a span's minutes (nsdur),nullius runs show. One decimal turned 0.05 s into "0.1s"./trial/…/artifact/<key>/download), and a back button to the Artifacts tab. A record past the reader's 16 MiB is saved as<name>.partial.<ext>.verifier/judge.json: nothing in the application ever wrote that file. Its two hand-made fixture copies are deleted with it, which is why the judge's script floors this at severe.Commits nine and ten correct what an independent review of the first eight found: cells colliding in the head at mid widths, sideways scroll at phone width, an imported trial said to be Harbor's, dashes where the page claimed none, three places still at one decimal, a silent cut of large downloads, Copy saying copied when a browser refused the write, and tests whose names no longer matched what they asserted.
The last three take
main(ten commits, among them a task's attempts and an attempt's tries, each with tabs on this page):traces/_head.html) in the place of its large name and subtitle.Touches
traces/trial.html,traces/_head.html(new),traces/attempts.html,_judgement.html,_conversation.html,artifact.html,runs/_execution.html(its script'ssecs),base.html(the copy handler),style.css,traces/conversation.cssweb/traces.py(the download route, the importer's name),web/filters.py(secs,nsdur),web/judge.pyruns/artifacts.py(whole),runs/cli.py(two decimals)field-guide.md(n_rate_limited_retries),cli/pages.md(the download route), the agent marks READMEtest_attempt_routes.pyamong them), and a newui_tests/test_trial_head.pyBefore and after
mainand this branch serving the same job trees, dark, 1280 px wide, rail collapsed.Overview, a SWE-bench trial
Artifacts tab
A record
Judge tab
Verified
main(49d32d1): backend suite 2229 OK, route suite 562 OK. With fix(standard): time the agent's setup apart from its run #174 merged on top as well: 2236 and 562 OK.secs; the run page's script against the template (run in node); a bar shorter than a second; the shell's phases against the page's.Risk
secsgains a second decimal, on every page that uses it. A stage time recorded to one decimal reads "41.20s".429 ×Nbadge keeps it.judge.jsonfixtures removes the only trace of a card nothing could fill; the judge's script floors a deleted test file at severe.traces/trial.html: the two merge cleanly in that order.