Skip to content

Decode imported images once and reuse the result - #140

Merged
vycdev merged 2 commits into
developfrom
perf/issue-131-import-decoding
Nov 10, 2026
Merged

vycdev merged 2 commits into
developfrom
perf/issue-131-import-decoding

Conversation

@vycdev

@vycdev vycdev commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Refs #131

Rebased onto develop after #142 moved the import check into validateImageFile (src/lib/imageFile.ts). This replaces the earlier version of this PR.

What changed

validateImageFile used to load each file into an <img> and then decode it again with createImageBitmap, throwing that bitmap away. Stack Matrix then drew the <img>, which decoded the photo a second time.

  • Raster files decode once, with createImageBitmap alone. SVGs, which Chrome rejects in createImageBitmap, and engines without it still decode in an <img>. The structural isTruncatedImageData check still runs first, so a truncated file is refused before it is decoded.
  • validateImageFile returns the decoded image as a DecodedImage (source, width, height, close()) instead of an HTMLImageElement. The caller owns it and closes it, just as it already owns and revokes the URL.
  • Stack Matrix draws the decoded image directly, then closes it. It draws with imageSmoothingQuality = 'medium'; see Stack Matrix colors below.
  • The main image import (useImageImport) closes it right away. CanvasPreview and useSwatches still decode the committed URL themselves; handing them this decode is the next step and is not in this PR.

Accepted files are unchanged

In Playwright Chromium and WebKit I compared the old rule (<img>, then createImageBitmap for non-SVG) with the new one (createImageBitmap for raster files, <img> for SVG). Each file was tested with its MIME type and with an empty type.

Format Chromium WebKit
JPEG, PNG, BMP, GIF, WebP, AVIF accepted (both rules) accepted (both rules)
HEIC, TIFF rejected (both rules) accepted (both rules)
JP2, PSD, TGA, PDF rejected (both rules) rejected (both rules)
SVG with type / without type accepted / rejected accepted / rejected

The old and new rules gave the same result in every case. A PNG and a JPEG with damaged data but an intact end marker are rejected by createImageBitmap in Chromium (<img> and img.decode() both accept them) and accepted by both in WebKit, as before. Firefox and the Windows WebView2 were not tested.

EXIF orientation: Stack Matrix now draws a bitmap instead of an <img>, so rotated phone photos matter here. With no options, createImageBitmap applies the EXIF orientation exactly as <img> does in Chromium 148 and WebKit 26.4 (same size and the same corner on top). I call it with no options on purpose: passing imageOrientation: 'from-image' throws on older engines that don't recognize that value, which would wrongly reject valid files. Much older engines whose createImageBitmap ignored EXIF tags are untested.

Stack Matrix colors

Stack Matrix measures colors from the photo's pixels, so I checked that drawing a bitmap instead of an <img> doesn't change them:

  • At full size the pixels are identical on both engines, for PNG, JPEG, and both fixtures.
  • Color profiles convert the same way. Display P3-tagged PNG and JPEG files give the same sRGB values through <img> and through the bitmap.
  • Downscaling differed. Photos with a longer side over 1200 px are shrunk, and at the default smoothing quality a bitmap is resampled more coarsely than an <img>. I measured cells the way sampleStackMatrixPhoto does (9×9 points in the middle 32% of each cell, trimmed mean) on the 3888×2916 fixture. The bitmap path ended up further from a true area average of the full-resolution pixels. medium quality matches or beats the old path on both engines (high is worse on WebKit):
Mean error vs true area average (levels per channel) 8×8 cells 16×16 cells
Chromium: old <img> / bitmap at default / bitmap at medium 3.09 / 3.30 / 3.00 2.37 / 2.42 / 2.21
WebKit: old <img> / bitmap at default / bitmap at medium 3.08 / 3.28 / 3.08 2.33 / 2.50 / 2.18

With medium, re-measuring the same photo gives cell colors about 0.5 levels per channel different from before, on average. This was measured on one natural photo, not a printed calibration board.

Cost (3888×2916 JPEG fixture, medians in the app)

Old New
Stack Matrix photo, Chromium (file chosen → photo shown) ~343 ms ~134–222 ms
Stack Matrix photo, WebKit ~206–211 ms ~219–236 ms
Main import check, Chromium (file chosen → check done) ~94–112 ms ~67–69 ms
Main import check, WebKit ~103–109 ms ~90–92 ms
Main import, file chosen → size shown ~700 ms Chromium, ~577 ms WebKit unchanged within noise

Stack Matrix photos load about 35–60% faster in Chromium (and so in the Windows app). On WebKit they are unchanged to about 25 ms slower; the spreads of the runs overlap. The main import's check is shorter, but the total time to show the image doesn't change, because most of that time is spent after the commit, in CanvasPreview and useSwatches.

Tests

  • Unit (tests/imageFile.test.ts): the stubs now count decodes. A raster file is decoded once by createImageBitmap and never by <img>. A truncated PNG is refused before any decode. createImageBitmap rejections, unreadable bytes, SVGs (<img> only), engines without createImageBitmap (<img> fallback), and non-images are each covered.
  • e2e @smoke raster imports are decoded once before they are committed: holds createImageBitmap for imported files and checks that it runs exactly once and that no <img> loads the imported file before the check finishes. It fails on the old code.
  • e2e Stack Matrix photo test: now also loads a JPEG with an EXIF "rotate 90°" tag and checks that it arrives as 200×400, not 400×200.
  • image-import-error.spec.ts and stack-matrix-photo-import.spec.ts: 9/9 on Chromium and 9/9 on WebKit, run with KROMACUT_E2E_FRESH_SERVERS=1.
  • CI on bc5f973: lint, build, unit tests, @smoke, docs, landing, not-found, i18n, and i18n build all pass.
  • npm run lint, npm run build, and npm test (1112 pass, 1 skipped) all pass.
  • The privacy and terms CI jobs are expected to fail until Privacy and terms e2e footer tests still expect 7 links after the Awards link #143 lands. That failure is unrelated to this change.

The changelog note is folded into the existing unreleased Unreadable image imports entry. The docs don't describe decode steps, so they are unchanged.

🤖 Generated with Claude Code

validateImageFile loaded each file into an <img> and then decoded it
again with createImageBitmap, which it threw away. Stack Matrix then drew
the <img>, decoding the photo a second time.

Raster files now decode with createImageBitmap alone. In Chromium and
WebKit it accepts the same formats as <img>, applies the same EXIF
orientation, and still rejects damaged files in Chrome. SVGs, which
Chrome rejects there, and engines without createImageBitmap still decode
in an <img>. The structural truncation check is unchanged.

validateImageFile returns that decoded image as a DecodedImage (source,
width, height, close). Stack Matrix draws it directly; the main image
import closes it, since the preview and swatches decode the committed URL
themselves.

Refs #131

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vycdev
vycdev force-pushed the perf/issue-131-import-decoding branch from ef9732c to bc5f973 Compare November 10, 2026 08:44
@vycdev vycdev changed the title Decode raster imports once while checking them Decode imported images once and reuse the result Nov 10, 2026
Drawing the decoded bitmap at the default smoothing quality downscaled
large photos more coarsely than the <img> did, which moved measured cell
colors further from the true area average. Medium quality matches or
beats the previous accuracy in Chromium and WebKit.

Refs #131

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vycdev
vycdev merged commit c3c0618 into develop Nov 10, 2026
8 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant