Skip to content

feat(media): add generated images, sources and MediaViewer - #125

Merged
TheGreatAxios merged 6 commits into
conversation/10-attachmentsfrom
conversation/11-media
Oct 28, 2026
Merged

TheGreatAxios merged 6 commits into
conversation/10-attachmentsfrom
conversation/11-media

Conversation

@pratikbuilds

Copy link
Copy Markdown
Collaborator

GeneratedImages holds each tile at its final size under the dither grain while it generates, with the elapsed time beside the label. An image made while watched is revealed by the grain; one already there loads plainly; a failed one shows a message and a retry. The grain reads its colours from existing tokens and draws one still frame under reduced motion. Sources lists citations, and MediaViewer shows images on the native dialog with arrow keys between items and focus returned to the opener. Media addresses from agent data are checked before use.

Stacked on #124: review only this commit. Part of #94.

Notes for review

  • Grain colours: ground --muted, field --foreground, band --primary (read via getComputedStyle, like DitherCanvas); no new token. Reduced motion draws one still frame and skips the reveal; off screen it stops drawing.
  • Every image/frame/download/link address from agent data passes safeMediaSrc (PDF frames: http, https, blob only); source links use toSafeHref.
  • MediaViewer takes useMediaViewer's result as viewer; focus returns to the element passed to viewer.open(index, opener). Backdrop is bg-background/90 (no literal scrim colour).
  • BAYER_4 duplicates DitherCanvas's private matrix (existing module left untouched); follow-up: have DitherCanvas import it.
  • The elapsed counter restarts on remount: PartImage carries no start time.

Verification

  • This commit passes bun run typecheck and bun run lint on its own (the stack is checked commit by commit).
  • At the top of the stack: bun run check, bun run build, bun run dep-guard, bun run contrast-test and bun run test:e2e (Playwright specs and the Storybook axe sweep in light and dark) pass.

Closes #105

GeneratedImages holds each tile at its final size under the dither
grain while it generates, with the elapsed time beside the label. An
image made while watched is revealed by the grain; one already there
loads plainly; a failed one shows a message and a retry. The grain reads
its colours from existing tokens and draws one still frame under reduced
motion. Sources lists citations, and MediaViewer shows images on the
native dialog with arrow keys between items and focus returned to the
opener. Media addresses from agent data are checked before use.

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated multi-agent review: independent review + second-agent validation; only confirmed items listed (the family itself is complete per task 11 and well covered).

  • should-fix src/lib/media-src.ts:21 + src/ui/media-viewer.tsx:125 — data: documents are filtered out before the viewer ever sees them, so they silently render as broken tiles with actions attached ("Open" does nothing), and there's no unloadable-tile state to communicate the failure. Either surface them or render an explicit unloadable state.
  • minor src/ui/media-viewer.tsx:258-336 — loaded/failed flags never reset when src changes on the same open element (swap a URL in place and the old state can stick).
  • minor src/ui/sources.tsx:70 — the Link icon renders on citations that aren't links (candidates verified); gate by type.
  • test gap — the 7 e2e and 4 unit tests only cover generated images: the non-image viewer branches, sources rendering, and unloadable tiles have zero coverage.
  • nits confirmed by git-grep — list-style: none on an <ol> in the sources list, prompt-less tile labels, src: "" sentinel usage, a duplicated BAYER_4/recipe block, a re-implemented toSafeHref, magic spacing multipliers, and a few speculative public exports with zero consumers at the series tip.

An image tile that could not be shown still offered Download, Copy
and Open, handing the host an index to an image it cannot load and
opening an empty viewer. Broken tiles now show only host-supplied
action children, and viewer items for such images carry no src
instead of an empty-string sentinel, so the viewer says there is no
preview and offers no download.

A tile whose address changed in place kept its failed or loaded
flag, so the new address was never tried and skipped its reveal.
The flags now reset when the source changes.

Citations shown as plain text no longer carry the link icon, the
sources list keeps list semantics without bullets, and an image
without alt text is named after the prompt's opening words.

Adds viewer tests for video, audio, PDF, text and unloadable items,
and the first tests for Sources.
@pratikbuilds
pratikbuilds force-pushed the conversation/11-media branch from fb3faf7 to 9660d62 Compare October 27, 2026 08:38
@pratikbuilds

pratikbuilds commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks for the review. Here's what happened to each point.

Fixed

1. Images that can't load still offered actions that do nothing (should-fix)

  • The problem: an unloadable tile already said "Couldn't load this image". The real problem was that it still offered Download, Copy and Open.
  • The fix: it now shows only actions the host passes in, and it can't open the viewer. In the viewer, such an image has no src, so it shows "No preview" with no download. The src: "" placeholder is gone, and MediaViewerItem.src is now optional.

2. Changing src kept the old loaded or failed state (minor)

  • The fix: the stuck state was in GeneratedImage, not in MediaViewer, which already starts fresh for each src. GeneratedImage now resets when its source changes.

3. The link icon showed on citations that aren't links (minor)

  • The fix: it now shows only on real links.

4. Only generated images were tested (test gap)

  • The fix: new tests cover video, audio, PDF, text and items without a source in the viewer. A new sources.test.tsx covers links, plain-text citations and the list.

5. Nits

  • The sources list has role="list".
  • An image without alt is named from the start of its prompt.
  • The spacing numbers have comments.
  • generatedImageFileName is no longer exported.

Not changed, and why

  • The duplicated BAYER_4: the other copy is in dither-canvas.tsx, which isn't part of this PR.
  • toSafeHref: Sources already uses the shared helper from lib/url.ts.

Commit: fix(media): drop actions on unloadable tiles and reset on src swap

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review · Request changes

Adds generated-image tiles, source citations, and a media viewer dialog.

Findings

  • src/ui/media-viewer.tsx:89 — Closing the dialog returns focus to the first opener, not the latest. The layout effect snapshots openerRef.current once ([openerRef] does not change) and the cleanup focuses that snapshot. A later viewer.open updates the ref (use-media-viewer.ts:66) while the dialog stays mounted, so open(0, buttonA) then open(1, buttonB) then Close focuses buttonA. Read openerRef.current in the cleanup, or re-run the effect when the opener changes.

  • src/ui/generated-images.tsx:423 — A watched tile that has already revealed keeps the previous image fully visible when its address changes. The src-swap reset clears loaded and failed only. It does not reset developing or remount the <img>. After the grain finishes, developing is false, so the overlay at :493 does not return, and the develop branch forces opacity-100 (:453-454). The new request starts on the same node. Reset developing, or key the image on src, so the old bitmap is not shown at full opacity while data-state is loading.

  • src/ui/media-viewer.tsx:334 — The PDF <iframe> is in the tab order, and Escape is handled only on the dialog (onCancel at :142, onKeyDown at :151). A key delivered to the frame does not reach those handlers, so keyboard focus inside the PDF viewer cannot close the dialog. tabIndex={-1} keeps the frame out of the tab order; the Close button stays the keyboard exit.

@pratikbuilds
pratikbuilds force-pushed the conversation/11-media branch from d86517e to c9738df Compare October 28, 2026 08:41
MediaViewer took the opener when it mounted, so if a host opened
another item while the viewer was up, closing it focused the first
opener. The cleanup now reads the latest opener and falls back to
whatever had focus on open.

A watched image that had already revealed kept the browser's old
picture at full opacity after its source changed, under a loading
state, until the new image arrived. The src swap now puts a watched
tile back under the grain, which reveals the new image once loaded.

Also document that Escape inside a PDF reaches the PDF viewer rather
than the dialog, and that Tab leaves the PDF for Close.
@pratikbuilds
pratikbuilds force-pushed the conversation/11-media branch from c9738df to 1433e3d Compare October 28, 2026 11:28
@pratikbuilds

Copy link
Copy Markdown
Collaborator Author

Thanks for the review. Two points are fixed. The PDF frame is documented instead, for the reason below.

1. Closing the viewer focused the first opener, not the latest

  • The fix: the cleanup now reads the opener when it runs, so it uses the one from the latest viewer.open. If that open was given no opener, focus goes back to whatever had it when the viewer opened.
  • Tests:
    • "on close, focus returns to the latest opener" failed before the fix.
    • "on close without an opener, focus returns to what had it on open" covers the fallback.

2. A watched tile that had revealed showed the old picture at full opacity when its src changed

  • The fix: the existing reset on a src change now resets developing too. A watched tile goes back under the forming grain, which covers the old picture until the new one has loaded. Unwatched tiles and reduced-motion tiles are unchanged. I didn't key the image on src, because that would drop the reveal.
  • Test: "a revealed watched tile goes back under the grain when its source changes". It failed before the fix.

3. Escape inside the PDF frame doesn't close the viewer

  • Docs only. The frame stays in the tab order.
    • tabIndex={-1} would take the whole PDF out of the tab order. A keyboard user would then have no way to scroll it, page through it, or reach its zoom and download controls. That's worse than needing a Tab before Escape.
    • Focus isn't trapped in the PDF. In Chromium, with its built-in PDF viewer, Tab steps through the viewer's controls and then reaches Close.
    • The PDF viewer runs in a frame the page can't attach a key listener to.
  • The MediaViewer docs now say that Escape inside a PDF goes to the browser's PDF viewer, not the dialog, and that Tab leaves the PDF and reaches Close.

Commit: fix(media): return focus to the latest opener; regrain on src swap

Without onOpen, a tile's picture took its alt from the image alone, so
an image with no alt was marked decorative and screen readers skipped
it, though the tile already had a name from the prompt's opening
words. A tile that failed to load likewise read "Couldn't load this
image". Both now use the tile's name, so a plain grid names every
picture and every failure.
The opener named in viewer.open stayed in the hook after the viewer
closed, and MediaViewer prefers it over what had focus on open. Once
any tile had opened the viewer, a later opening through the
controlled index sent focus back to that old tile instead of to the
control that opened it. MediaViewer now clears the opener when it
returns focus there, so an opening without open falls back to what
had focus at the time.
MediaViewer read ref from its props, which only React 19 passes to a
function component; under React 18, inside the peer range, a host's
ref never reached the dialog. MediaViewer is now a forwardRef
component that hands the ref to the <dialog>, and its props type
leaves ref to forwardRef.
@pratikbuilds
pratikbuilds force-pushed the conversation/11-media branch from 1433e3d to bf91f4c Compare October 28, 2026 14:47
@pratikbuilds

Copy link
Copy Markdown
Collaborator Author

Round 3 fixes

A fresh review pass over the stack turned up the points below; each is fixed in a new commit at the end of this branch, and the stack is rebased onto it.

  • 40b73d2: The opener was never cleared after close, so a later open through the controlled index returned focus to an old tile. MediaViewer clears it on close.
  • 04d70b6: Without onOpen, an image with no alt got alt="". It now uses the prompt-based label, and a broken tile says which image failed.
  • bf91f4c: React 18 refs: components that read ref from props only receive it on React 19, while the peer range allows ^18.2.0. They are now wrapped in forwardRef, so a host ref reaches the element on both; the peer range is unchanged. Covers MediaViewer.

bun run check and dep-guard pass on every branch of the stack, and the full e2e suite passes at the top of the stack (163/163, including the stories axe sweep). Each behaviour fix comes with a test that fails without it; the forwardRef tests can only guard, since the suite runs on React 19.

@corbits-triage

Copy link
Copy Markdown
Contributor

Triage: Fail

Waiting on:

  • checks are still running: check, policy-check

Stacked on #124: only this pull request's own changes were checked.

Notes:

  • added code matches a code pattern: e2e/media-viewer.spec.ts:132 Invented fallback `?? 0`; handle the missing value instead., e2e/media-viewer.spec.ts:133 Invented fallback `?? 0`; handle the missing value instead., src/ui/dither-field.tsx:76 Invented fallback `?? 0`; handle the missing value instead., src/ui/dither-field.tsx:140 Invented fallback `?? []`; handle the missing value instead., src/ui/dither-field.tsx:143 Invented fallback `?? []`; handle the missing value instead., src/ui/generated-images.tsx:206 Nested ternary; use if statements or a lookup., src/ui/generated-images.tsx:472 Nested ternary; use if statements or a lookup., src/ui/generated-images.tsx:480 Nested ternary; use if statements or a lookup., stories/chat/media.stories.tsx:130 Invented fallback `?? []`; handle the missing value instead.

@TheGreatAxios
TheGreatAxios dismissed their stale review October 28, 2026 21:49

Superseded by the completed stacked review; blocking fixes were addressed and the rebased stack is being landed.

@TheGreatAxios
TheGreatAxios merged commit e234719 into main Oct 28, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add MediaViewer, generated images and sources

2 participants