Skip to content

feat(scene): overlays load their own images and fonts - #62

Merged
kixelated merged 4 commits into
mainfrom
quest/m1/overlay-assets
Nov 6, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m1/overlay-assets

Conversation

@kixelated

@kixelated kixelated commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Finishes quest/m1/overlay-assets.md (from moq-dev/studio#34).

Problem

An overlay could not show a logo or set text in a brand face: check refused <img>, @font-face, and every url(), and its Blitz document had no net provider. A relative <img src> also panicked inside Blitz's parser during check, because the document's default base URL (a data: URL) cannot resolve one.

Approach

  • overlay::check(html, dir) now returns a Content: the HTML plus every asset it names, read and checked on the source's task, off the render thread. URLs resolve against the scene file's directory as a file: base, exactly as the document will resolve them.
    • data: URLs decode per the Fetch standard (percent-decoding, forgiving base64).
    • file: URLs must canonicalize under the canonical scene directory, which refuses .., absolute paths, and symlinks out of it.
    • Any other scheme is refused, naming the URL.
    • Images must be PNG or JPEG at most 4096 px a side (the overlay limit), and they are fully decoded here, so a corrupt one fails the source. Fonts must decompress (WOFF/WOFF2, via wuff, as Blitz does) and register with fontique the way Blitz will register them.
  • <img> (with src and alt) and top-level @font-face (font-family, src, font-weight, font-style, font-display) are now allowed. url() is found by walking each declaration's CSS tokens. It is allowed only in background, background-image, and @font-face src, the places Blitz draws or loads it, so content: url() or a url() in a custom property is refused by name. local(), image-set(), and src() are refused, as are descriptors Blitz ignores, such as unicode-range, and @font-face inside @media (Blitz reads only top-level ones).
  • On the overlay thread, open puts every decoded image in the document's image cache before the first resolve, using Blitz's public load_resource. Blitz checks that cache before any fetch, so an <img>, a background, or a background an animation brings in later draws with no fetch. A NetProvider (Assets) serves fonts from memory, and they land in the open's resolve. The first frame on program therefore has every asset. If the document ever fetches a URL check did not read, its paint fails, naming the URL.
  • Picture::Overlay carries the Content, so the HTML and assets the compositor opens always match.
  • Blitz's woff feature is on, so it does not skip a src that names a .woff/.woff2 file or format. It receives the already decompressed bytes either way.

Measured on this Mac (release build), with a full-frame 1920x1080 PNG: the overlay thread's open and first resolve take 0.3 ms. Before images were preloaded, Blitz decoded the PNG on that thread, which took 24 ms, and the compositor's next paint waits on that thread. The 23 ms decode now runs in check on the source's task, where an image source already decodes its picture. A TTF costs 0.4 ms in check and 0.17 ms at open.

Impact

  • An overlay may use <img src>, url() in CSS, and @font-face. These load from data: URLs or from files under the scene file's directory. Anything else fails the source, naming the URL.
  • A relative URL in an overlay used to panic Blitz's parser. It now loads or fails with a message.
  • Public API and wire: none. overlay and Picture are crate-private. The scene file format is unchanged. Existing overlays behave the same.
  • New dependencies: base64 and percent-encoding, both already in the lockfile, and wuff, which Blitz's woff feature also brings in, along with brotli-decompressor.
  • README documents where assets go and what they may be.

Choices made without asking

  • local() in @font-face is refused, not skipped. Blitz ignores it, and the machine's fonts are deliberately left out, so a src: local(X), url(y) list must drop local(X).
  • Every url() in a src list is read and must load, including fallbacks Blitz would never fetch. This fails loud rather than letting a broken fallback through.
  • url() in a custom property is refused, even when a var() would only carry it into background. Tracking where each var() lands would be needed to allow it; writing url() in background directly covers the same use.
  • Image sides are capped at 4096 px (browser::MAX_SIDE, the largest an overlay can be), which bounds what Blitz's decode can allocate.
  • WOFF/WOFF2 are supported (Blitz's woff feature) because brand kits usually ship web fonts as WOFF2.
  • <img> takes no width/height attributes. CSS sizes it, consistent with the existing attribute rule.
  • Blitz still panics while parsing an element whose src or href is not a URL (such as http://[). check contains this with catch_unwind and reports it as an error. The overlay thread never sees such HTML.

Alternatives

  • A provider that reads files lazily on the overlay thread. That would move disk I/O onto the thread the compositor waits on, and errors would only surface at paint time. Reading in check reports errors when the source starts.
  • Fetching over https was rejected in the quest's recorded decision (2026-11-04).

Follow-ups

  • Blitz paints a <body> background color across the canvas but not a background image, so body { background: url(logo.png) } on an empty body draws nothing. This is upstream (DioxusLabs/blitz). Until then, an overlay can put the image on a sized element.
  • Blitz panics in its parser on an unresolvable src/href (see above). An upstream fix would let check drop catch_unwind.

Validation

  • just ci (via quest gate) passed on 4b7b203: lints, and 240 tests.
  • GPU tests on Metal: cargo nextest run -p moq-scene --lib --run-ignored only -E 'test(/overlay/)' passed 4/4, including the new an_overlay_draws_its_image_on_its_first_frame.
  • New unit tests cover assets being in place once the overlay opens: an <img> from data:, a background from a file, a background a @keyframes brings in at its end, and a TTF face via @font-face. They also cover refusals (other schemes, .., an absolute path, a symlink out, a missing file, a directory, a wrong type, an oversized image, a truncated PNG, a WOFF2 that does not decompress, bad base64, url() in content or a custom property, local(), image-set(), unicode-range, and nested @font-face) and data: decoding. With the image preload disabled, the open test fails.

Review

Codex (quest review), gpt-6.1-sol, medium effort. Last round: round 2 on head 4b7b203595c88fb90f9a8eb7086bcc8b3ea473a4, 0 findings.

Round 1 (head 97c4128), all fixed in 4b7b203:

  • A background an animation introduced after time zero was fetched during a resolve but applied only at the next, and could stay missing once the animation stopped. Fixed: images are decoded in check and preloaded into Blitz's image cache, so no image is ever fetched.
  • Header-only validation let a truncated image or a non-decompressing WOFF2 through, which then silently failed to draw. Fixed: images fully decode, and fonts decompress and register in check.
  • content: url(logo.png) passed but Blitz never draws image content. Fixed: url() is allowed only in background, background-image, and @font-face src.

(Written by Claude Opus 5.5)

Quest-Claim: a97c81d77de75a2c
@pullfrog

pullfrog Bot commented Nov 6, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

kixelated and others added 2 commits November 5, 2026 23:02
An overlay names images and fonts as data: URLs or files under the scene
file's directory. The source reads and checks each before the overlay
opens, and the overlay's document fetches from those bytes alone, so its
first frame has them and it never touches the network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Overlay assets feat(scene): overlays load their own images and fonts Nov 6, 2026
… Blitz draws them

Images decode in check and go into the document's cache before its first
resolve, so a background an animation brings in later draws at once, and
a corrupt one fails the overlay. Fonts decompress and register in check.
url() is allowed only in background, background-image, and @font-face src.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 6, 2026 07:42
@coderabbitai

coderabbitai Bot commented Nov 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 45 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 07bc70c8-c32c-4443-bd0e-06960f991755
Commits

Reviewing files that changed from the base of the PR and between c6b1fb7 and 4b7b203.

Files ignored due to path filters: 1

  • Cargo.lock is excluded by !**/*.lock
Files selected for processing: 8

  • README.md
  • quest/m1/README.md
  • quest/m1/overlay-assets.md
  • rs/moq-scene/Cargo.toml
  • rs/moq-scene/src/collection.rs
  • rs/moq-scene/src/compositor.rs
  • rs/moq-scene/src/overlay.rs
  • rs/moq-scene/src/source.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 4b7b203

No actionable findings in the reviewed changes.

Overall direction: The validated asset snapshot carried with the overlay keeps source preparation and compositor rendering consistent. Decoding images before the render-thread handoff and preloading Blitz’s image cache addresses first-frame and later-animation asset availability without render-thread file I/O.

Verification limits: GitHub-only static review of the loader, source/compositor handoff, regression tests, and the pinned Blitz resource-handling code. I did not run quest context, quest check, tests, or GPU/platform validation. The author reports passing local checks; the latest GitHub Check run was still in progress when inspected.

@kixelated
kixelated merged commit acbcb5e into main Nov 6, 2026
3 checks passed
@kixelated
kixelated deleted the quest/m1/overlay-assets branch November 6, 2026 15:24
kixelated added a commit that referenced this pull request Nov 6, 2026
A restart lays the overlay out again through #62's open, so its preloaded
images and served fonts survive it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant