Skip to content

feat: preview images in tool output cards - #75

Merged
johnnyhuy merged 2 commits into
mainfrom
agent/issue-74-tool-images
Nov 14, 2026
Merged

johnnyhuy merged 2 commits into
mainfrom
agent/issue-74-tool-images

Conversation

@johnnyhuy

@johnnyhuy johnnyhuy commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Tool output cards now preserve image results from saved history and live events, with separate Show image controls and bounded raster previews. Image-only results no longer appear empty, and text copies, exports and handoff drafts explain that image bytes are omitted.

Part of #2.
Closes #74.

Verification

  • bun install --frozen-lockfile passed.
  • mise exec -- bun run lint passed.
  • mise exec -- bun run typecheck passed.
  • mise exec -- bun run test:unit passed, 98 tests across 10 files.
  • mise exec -- bun run test:integration passed, 102 tests across 13 files.
  • mise run build passed, including the production bundle.
  • git diff --check passed.

Proof

$ mise exec -- bun run test:unit
 Test Files  10 passed (10)
      Tests  98 passed (98)

$ mise exec -- bun run test:integration
 Test Files  13 passed (13)
      Tests  102 passed (102)

$ mise exec -- bun run test:e2e
 Error: Library not loaded: @rpath/Electron Framework.framework/Electron Framework
 Error: Electron failed to install correctly
 Test Files  7 failed (7)
      Tests  62 skipped (62)

Local app launch and screenshot capture are blocked by the incomplete Electron runtime installation. Hosted macOS Playwright capture succeeded on commit 43318bb, with every Electron test passing.

$ mise run test:e2e  # hosted macOS
 tests/e2e/tool-preview.test.ts (8 tests) passed
 previews saved tool images on demand at desktop and minimum window sizes passed
 explains broken and unsupported saved images while keeping the tool result readable passed
 keeps an opened image visible when a live tool result replaces its partial snapshot passed
 Test Files  7 passed (7)
      Tests  62 passed (62)

The artefact contains five new PNGs under Users/runner/work/pidex/pidex/test-results/tool-preview/.

  • tool-image-controls.png shows the read card, accompanying text, image notice and disclosure control.
  • tool-image-desktop.png and tool-image-narrow.png show the decoded image at native desktop and 900 by 600 minimum window sizes.
  • tool-image-unavailable.png shows both unsupported-format and decoding-failure explanations.
  • tool-image-live.png shows the decoded image after a live tool replaces its partial result with the final result.

All five screenshots were downloaded, visually reviewed and copied to local .tmp/proof/. Separate control and preview frames keep the proof readable on the hosted display, which clamps the requested window height. No scratch captures are committed. Inline attachment upload is blocked because the browser access host is unavailable and native browser access returned cgWindowNotFound; the screenshots are uploaded through the existing CI artefact workflow and linked above.

New unit coverage

  • keeps text and numbered image notices in saved tool results without copying image bytes.
  • explains unsupported and malformed images without treating them as empty output or URLs.
  • replaces image snapshots during streaming and clears them when the final result has text only.
  • exports image result notices and accompanying text without inline image data.

New component coverage

  • loads images on demand, keeps them independent and restores disclosure after row recycling.
  • explains a broken image and allows a replacement streamed image to load.
  • explains unsupported image results without mounting a preview.

New Electron cases cover saved images at native desktop and minimum window sizes, real clipboard text for output and portable context, broken and unsupported images, and a live partial result replaced by its final snapshot. PNG proof is written to .tmp/proof/ and test-results/tool-preview/ for CI upload. No scratch captures are committed.

PNG, JPEG, GIF and WebP inline data are supported. Previews fit the card width and 320 pixels in height. SVG, tool-supplied URLs and missing or malformed data receive a text explanation; decoding failures retain the readable result. Full-size viewing, image copying and image-bearing exports remain possible follow-up work. All earlier tool-card children are merged, so this slice completes the original #2 scope of read, write, edit and bash cards with streaming states and output inspection. The parent will be closed with its child history recorded after merge.

johnnyhuy and others added 2 commits November 14, 2026 22:37
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
@johnnyhuy
johnnyhuy marked this pull request as ready for review November 14, 2026 15:45
@johnnyhuy
johnnyhuy merged commit f2ae5ff into main Nov 14, 2026
15 checks passed
@johnnyhuy
johnnyhuy deleted the agent/issue-74-tool-images branch November 14, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show on-demand image previews in tool output cards

1 participant