feat: AgentV Studio — local explorer hardening

[christso]

Status Update (2026-04-08)

AgentV Studio is already implemented as a local browser for eval results via:

• apps/studio/ (React + Vite)
• Hono-backed agentv serve / agentv studio
• run, eval, experiment, target, project, and settings views

This issue should no longer be treated as a greenfield platform build.

Revised Objective

Use #563 for hardening and light polish of the existing Studio explorer, not for building a full eval management platform.

In Scope

• Small UX improvements to the current Studio explorer
• Tightening run/project browsing flows
• Lightweight reuse of existing data primitives where clearly useful
• Optional lightweight trend/regression surfacing if it cleanly reuses existing CLI/data capabilities

Out of Scope / De-scoped

• Separate history repo architecture
• Studio as the primary home for quality gate management
• Studio as an orchestration command center
• Building a deep trace-analysis platform inside Studio
• Replacing external observability tools
• Public leaderboard / submission workflow
• Trajectory publishing / shareable links

Why scope changed

• Observability: AgentV already exports OpenTelemetry to OTLP backends including Langfuse, which covers much of the heavier trace-analysis use case.
• Quality enforcement: the current workflow is primarily CLI + GitHub Actions CI, not dashboard-driven quality gate management.
• Platform status: the React/Vite/Hono Studio foundation already exists.

UX Gaps (from SWE-bench competitive analysis)

Competitive analysis of the SWE-bench leaderboard identified specific explorer gaps. See agentevals-research#55 for full analysis.

P1 — High impact, reuses existing CLI data primitives

• Run comparison view — Select 2 runs, see side-by-side per-test scores with win/loss/tie deltas. Visual equivalent of agentv compare (data already computed by CLI). SWE-bench has a "Compare results" button with checkboxes on every leaderboard tab.
• Cost column in runs list — Surface total/avg cost per run in the home Runs table. Cost data already exists in JSONL results (cost_usd) but is only visible after drilling into individual eval detail. SWE-bench prominently shows "Avg. $" as a column on every tab.

P2 — Polish, improves explorer usability

• Column sorting — Click any column header (score, cost, duration, date) to sort. No columns in Studio are currently sortable.
• Trend visualization — Show agentv trend data as a line chart (score over time by target/suite). CLI already computes linear regression and degrading/improving/stable classification — Studio could render this visually. This would be an AgentV differentiator (SWE-bench has no trend feature).
• Run filtering — Filter runs list by target, experiment, date range. SWE-bench has agent type and model type dropdowns. Studio currently has tab-based grouping (runs/experiments/targets) but no within-tab filtering.

Implementation notes

All P1 items reuse existing data primitives — no new data collection needed. The run comparison view can call the same logic as agentv compare. Cost is already in JSONL results. This aligns with #788's guidance to keep Studio as a lightweight local explorer that reuses CLI/data primitives.

Remaining Work

• Audit current Studio UX for obvious explorer gaps (done — SWE-bench competitive analysis, 2026-04-08)
• Implement P1 gaps (run comparison view, cost column)
• Implement P2 gaps (column sorting, trend chart, run filtering)
• Decide whether any additional lightweight trend/regression surface is worth adding
• Keep docs aligned with Studio's role as a local explorer

[christso]

Status Clarification

The eval-viewer skill added in plugins/agentv-dev/skills/agentv-bench/eval-viewer/ does not address this issue. It is a single-run review viewer (ported from skill-creator) that renders results.jsonl + optional grading.json into a static HTML page with feedback save/load.

What eval-viewer covers

• Reads results.jsonl from a workspace directory
• Optionally reads grading.json and benchmark.json
• Serves a self-contained HTML report via a zero-dep Python HTTP server
• Supports static HTML export (--static)

What remains unaddressed from this issue

• History repo architecture (runs/, index.json, history.json, meta.json)
• .agentv/config.yaml history.repo integration
• agentv eval run auto-saving snapshots to history repo
• Progressive SSE visualization (real-time updates during eval runs)
• Historical trend charts with campaign grouping
• Trace explorer with collapsible tree from trace.jsonl
• Cross-run and cross-campaign comparison
• Dataset browser with filtering and import
• agentv serve / agentv results reindex / agentv results prune CLI commands
• Campaign/iteration tracking via history.json

Of the 15 acceptance signals defined here, 1 is met, 2 are partially met, and 12 are unaddressed. This issue remains fully open.

[christso]

Dashboard Scope Upgrade: Viewer → AgentV Studio

Based on research into melagiri/code-insights and analysis of the current agentv issue landscape, this issue should be upgraded from a read-only viewer to AgentV Studio — a full eval management platform.

What changes

Tech stack upgrade:

• Current plan: Vanilla JS, CSS bar charts, zero dependencies
• Proposed: React 19 + Vite SPA, Recharts/Plotly for charts, TanStack Table for data grids, shadcn/ui + Tailwind for components
• Justification: The capabilities below (quality gate management, orchestration monitoring, pattern synthesis) cannot be built with vanilla JS. Code-insights demonstrates this stack works well for a similar use case (React + Vite + Hono).

New management capabilities (beyond viewing):

• Run actions: rerun, compare, delete, tag, annotate
• Quality gate configuration UI (extends feat(eval): composable quality gates with auto-remediation triggers #334) — define/edit gates from the dashboard
• Regression alert feed (extends feat(eval): iteration tracking, termination taxonomy, and cross-run regression detection #335) — see and act on regressions
• Orchestration monitoring (NEW feat(dashboard): orchestration monitor — campaign management UI #785) — watch and control active campaigns
• Root cause explorer (NEW feat(dashboard): root cause explorer — trace-driven failure diagnosis #786) — trace-driven failure diagnosis
• Pattern synthesis (NEW feat(dashboard): code-insights integration — pattern synthesis from eval sessions #787) — code-insights inspired insight extraction
• Cost attribution views (extends feat: compute costUsd from token usage via model pricing table #635) — cost per evaluator, category, target

Architecture:

agentv/
├─ apps/cli/              # Lightweight CLI (unchanged, keeps html-writer.ts)
├─ apps/studio/           # NEW: React SPA (AgentV Studio)
│  ├─ src/pages/
│  │  ├─ Overview.tsx     # Runs overview + stat cards
│  │  ├─ Trends.tsx       # Historical trend charts
│  │  ├─ Gates.tsx        # Quality gate configuration (#334)
│  │  ├─ Alerts.tsx       # Regression alerts (#335)
│  │  ├─ Campaigns.tsx    # Orchestration monitor (#785)
│  │  ├─ RootCause.tsx    # Root cause explorer (#786)
│  │  ├─ Insights.tsx     # Pattern synthesis (#787)
│  │  └─ TraceExplorer.tsx
│  └─ vite.config.ts
└─ apps/server/           # Hono backend (extends existing agentv serve)

What stays the same:

• History repo storage model (filesystem, git-based)
• agentv serve CLI command as entry point (now launches AgentV Studio)
• Progressive SSE visualization during runs
• All existing views from the original spec (trends, run history, trace explorer, dataset browser, comparison)
• html-writer.ts remains for lightweight --format html CI reports

Tracking issue

• project: AgentV Studio — lightweight local results explorer (scope reduced) #788 — AgentV Studio epic

Related new issues

• feat(dashboard): orchestration monitor — campaign management UI #785 — Orchestration monitor (campaign management UI)
• feat(dashboard): root cause explorer — trace-driven failure diagnosis #786 — Root cause explorer (trace-driven diagnosis)
• feat(dashboard): code-insights integration — pattern synthesis from eval sessions #787 — Code-insights integration (pattern synthesis)

Research source

• melagiri/code-insights — React + Vite + Hono dashboard with real-time session analysis, activity charts, cost tracking, pattern detection, dark/light themes
• Supabase Studio — open-source local-first management UI pattern

[christso]

Studio UX gaps identified from SWE-bench competitive analysis

From agentevals-research#55 — SWE-bench leaderboard UX analysis reveals these lightweight explorer gaps in Studio:

P1 — High impact, aligns with existing CLI primitives

1. Run comparison view — Select 2 runs, see side-by-side per-test scores with win/loss/tie deltas. This is the visual equivalent of agentv compare, which already produces the data. SWE-bench has a "Compare results" button with checkboxes on the leaderboard.

2. Cost column in runs list — Surface total/avg cost per run in the home Runs table. Cost data already exists in JSONL results (cost_usd) but isn't shown in the runs list — only visible after drilling into individual eval detail. SWE-bench prominently shows "Avg. $" as a column on every leaderboard tab.

P2 — Polish, improves explorer usability

3. Column sorting — Click any column header (score, cost, duration, date) to sort. SWE-bench's % Resolved column is clickable for sorting. Currently no columns in Studio are sortable.

4. Trend visualization — Show agentv trend data as a line chart (score over time by target/suite). CLI already computes linear regression and degrading/improving/stable classification — Studio could render this visually. SWE-bench doesn't have this (it's leaderboard-only), so this would be an AgentV differentiator.

5. Run filtering — Filter runs list by target, experiment, date range. SWE-bench has agent type and model type (open source/proprietary) dropdowns. Studio currently has tab-based grouping (runs/experiments/targets) but no within-tab filtering.

Out of scope (per #788 revised scope)

• Public leaderboard / submission workflow — not a Studio feature
• Trajectory publishing / shareable links — requires infrastructure beyond local explorer
• Benchmark variant tabs — this is dataset management, not Studio UX

Implementation note

All P1 items reuse existing data primitives — no new data collection needed. The run comparison view can call the same logic as agentv compare. Cost is already in JSONL results. This aligns with #788's guidance to keep Studio as a lightweight local explorer that reuses CLI/data primitives.

[christso]

Thanks, this older Dashboard tracking slice is now folded into the public Dashboard results inspection and scale epic: #1466. Related trace-dashboard work remains under #1451 where applicable. Closing this issue as consolidated so public updates stay in one place.