Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,9 @@ jobs:
- name: Setup Emscripten
uses: mymindstorm/setup-emsdk@ab889da2abbcbb280f91ec4c215d3bb4f3a8f775
with:
version: 'latest'
# Pinned: the version that built the checked-in web/public/codec.wasm.
# 'latest' moved to 6.x and broke the build; bump deliberately.
version: '5.0.1'

- name: Build WASM
# We use 'emmake' to ensure the Makefile sees the Emscripten environment
Expand Down
77 changes: 77 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Codec Explorer

An interactive lab for transform image compression (DCT vs Haar vs CDF 9/7 wavelets). A C++17 codec core is compiled to WebAssembly and driven by a Svelte 5 + TypeScript web app. Everything runs in the browser.

## Layout

- `core/` — the codec, plain C++17 with no dependencies. `inc/` headers, `src/` sources, `tests/` GoogleTest.
- `ImageCodec` — colour conversion, chroma subsampling, 8×8 DCT or whole-image wavelet, quantization, bit estimate.
- `wavelet`, `transform`, `entropy`, `colorspace`, `CodecAnalysis` (PSNR, SSIM, artifact maps).
- `web/cpp/codec_web.cpp` — the WASM binding layer: one global session, C functions exported to JS.
- `web/public/codec.{js,wasm}` — the **checked-in** WASM build. Edit C++, then rebuild it (below), or the app keeps running the old code.
- `web/src/` — the app.
- `App.svelte` switches between three full-page modes: `ViewerMode`, `InspectorMode` (one 8×8 block through the pipeline), `ArtifactInspectorMode` (error, edge and blocking maps).
- `lib/state.svelte.ts` — shared reactive state (`appState`, a Svelte 5 `$state` class).
- `lib/wasm-bridge.ts` — typed wrappers over `Module._*` calls; `types/wasm.d.ts` declares them.
- `lib/inspection.ts`, `lib/grid-renderer.ts`, `lib/basis-popover.ts` — the Block Inspector's imperative DOM rendering (see "Inspector DOM" below).
- `lib/components/` — shared UI: `QualityControl`, `CodecSettings` (transform + chroma), `ResultSummary`, `ImageViewer`.
- `styles/` — global CSS bundled through `app.css`. Colours are tokens in `variables.css`; dark mode is `[data-theme="dark"]`.
- `apps/native/` — an OpenCV desktop app for poking at the core (needs OpenCV).

## Commands

Web, from `web/`:

```bash
npm run dev # Vite dev server
npm run check # svelte-check (types)
npm run test # unit tests (Vitest + jsdom, fast)
npm run test:browser # integration tests in Playwright Chromium
npm run test:all # both
```

From the repo root:

```bash
make web # rebuild web/public/codec.{js,wasm} (needs Emscripten 5.0.1, as pinned in CI)
make test # C++ GoogleTest suite (CMake; configures apps/native too, so needs OpenCV)
make sanitize # C++ tests under ASan + UBSan (falls back to UBSan where ASan can't start, e.g. macOS 26)
```

Before calling a change done, run `npm run check` and `npm run test:all` (plus `make test` for C++ changes).

## Contracts across the WASM boundary

Nothing links C++ and TypeScript at compile time, so these pairs are kept in sync by hand and **guarded by tests**:

| What | C++ side | TS side | Test |
|---|---|---|---|
| Exported functions | `WEB_EXPORTS` in `Makefile`, or `EMSCRIPTEN_KEEPALIVE` | `web/src/types/wasm.d.ts` | `tests/unit/wasm-exports.test.js` |
| View mode numbers | `enum ViewMode` in `codec_web.cpp` | `ViewMode` in `lib/state.svelte.ts` | `tests/unit/wasm-contracts.test.js` |
| Transform numbers | `map_transform_mode` in `codec_web.cpp` | `Transform` in `lib/transforms.ts` | same |
| Block buffer layout | `BlockDebugData` in `core/inc/ImageCodec.h` | `BLOCK_GRID` in `lib/inspection.ts` | same |
| Histogram bins | `CoefficientStats::NUM_BINS` | `COEFF_BINS` in `lib/wasm-bridge.ts` | same |

Change both sides together. Buffers returned by `malloc` in C++ are freed by the TS caller (see each binding's comment).

## Inspector DOM

The Block Inspector renders its grids imperatively and finds elements by ID. Those lookups skip missing elements **silently**, so:

- Refer to inspector element IDs only through `INSPECTOR_IDS` in `lib/inspector-dom.ts`, never as string literals.
- To add one: give the element an `id` in `InspectorMode.svelte` and add it to `INSPECTOR_IDS`.
- `tests/unit/inspector-dom.test.js` fails if an ID isn't rendered, or if lib code bypasses the list.

## Conventions

- Svelte 5 runes (`$state`, `$derived`, `$effect`, `$props`); mount with `mount()`. Use `untrack` when an effect calls code that reads state it shouldn't depend on.
- Hover help: put explanations in plain `title` attributes (SVG: a `<title>` child). `lib/hover-hint.ts` shows them quickly and styled, app-wide. Add the `has-hint` class to a text label to give it the dotted underline.
- Reuse the shared components (`QualityControl`, `CodecSettings`, `ResultSummary`) instead of copying controls into a mode.
- Tests are `.js` files that import `.ts` sources with a `.js` extension; a small Vite plugin in both Vitest configs resolves them. The WASM `Module` global is mocked in `tests/setup/`.
- `@sveltejs/vite-plugin-svelte` is pinned to `^4` because Vitest 2.1 needs Vite 5. Don't bump it alone.
- Comments explain why, not what. Match the density of the surrounding code.
- Commits: conventional style with a scope, e.g. `feat(inspector): …`, `fix(core): …`, `refactor: …`, `docs: …`. Work on a feature branch and open a PR to `main`; Vercel deploys `main`.

## Related work

Motion estimation lives on the paused `feature/motion-estimation` branch. Its bindings (`*_me_*`, `MotionEstimator.*`) are on `main` too; leave them alone unless working on that feature.
4 changes: 3 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# --- CONFIGURATION ---
WEB_COMPILER = emcc
# em++ (not emcc): newer Emscripten releases only link the C++ standard library
# when invoked as the C++ driver.
WEB_COMPILER = em++
# C functions exported to JavaScript (keep in sync with web/src/types/wasm.d.ts)
WEB_EXPORTS = _init_session _process_image _preview_image _rd_encode _get_rd_bit_estimate \
_get_view_ptr _set_view_tint _set_artifact_gain \
Expand Down
5 changes: 4 additions & 1 deletion core/inc/ImageCodec.h
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ class ImageCodec {
* zeroFraction — fraction that quantize to zero at this codec's quality
*/
struct CoefficientStats {
static constexpr int NUM_BINS = 12;
static constexpr int NUM_BINS = 12; // mirrored by COEFF_BINS in web/src/lib/wasm-bridge.ts
double histogram[NUM_BINS] = {};
double zeroFraction = 0.0;
};
Expand Down Expand Up @@ -107,6 +107,9 @@ class ImageCodec {
Image upsampleChannel(const Image& channel, int targetWidth, int targetHeight, ChromaSubsampling cs) const;

public:
// Returned to JS by inspect_block_data as one flat buffer; the field order
// is mirrored by BLOCK_GRID in web/src/lib/inspection.ts (checked by
// web/tests/unit/wasm-contracts.test.js).
struct BlockDebugData {
double original[8][8];
double coefficients[8][8]; // Transform coefficients (DCT or DWT)
Expand Down
5 changes: 4 additions & 1 deletion web/cpp/codec_web.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ static MotionEstimator g_me;
static std::vector<MotionVector> g_mvs;
static std::vector<std::pair<int,int>> g_search_steps;

// Enum to match view modes in JavaScript.
// View modes for get_view_ptr. Mirrors ViewMode in web/src/lib/state.svelte.ts;
// web/tests/unit/wasm-contracts.test.js checks the two agree.
enum ViewMode {
RGB = 0,
Artifacts = 1,
Expand All @@ -57,6 +58,8 @@ static ImageCodec::ChromaSubsampling map_cs_mode(int mode) {
}
}

// `transform_mode` arguments: mirrors Transform in web/src/lib/transforms.ts
// (checked by web/tests/unit/wasm-contracts.test.js).
static ImageCodec::TransformType map_transform_mode(int mode) {
switch (mode) {
case 1: return ImageCodec::TransformType::DWT_HAAR;
Expand Down
96 changes: 19 additions & 77 deletions web/src/ArtifactInspectorMode.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
import { onMount, onDestroy } from 'svelte';
import { fade } from 'svelte/transition';
import { appState, ViewMode, openBlockInspector } from './lib/state.svelte.js';
import { TRANSFORMS, isWavelet } from './lib/transforms.js';
import { isWavelet } from './lib/transforms.js';
import { processImage, previewImage, getViewPtr, getStats, free, setArtifactGain } from './lib/wasm-bridge.js';
import QualityControl from './lib/components/QualityControl.svelte';
import CodecSettings from './lib/components/CodecSettings.svelte';
import ResultSummary from './lib/components/ResultSummary.svelte';
import ImageViewer from './lib/components/ImageViewer.svelte';

let artifactCanvas: HTMLCanvasElement | undefined = $state();
Expand Down Expand Up @@ -34,12 +36,6 @@
{ val: ViewMode.BlockingMap, label: 'Blocking Map', desc: 'Detects discontinuities at 8×8 block edges, showing where the macroblock structure is visible.' },
];

const CS_MODES = [
{ id: 'art_cs_444', val: 444, label: '4:4:4' },
{ id: 'art_cs_422', val: 422, label: '4:2:2' },
{ id: 'art_cs_420', val: 420, label: '4:2:0' },
];

// Wavelets have no 8×8 grid, so the blocking map should stay dark — say so.
const artifactModes = $derived(ARTIFACT_MODES.map(m =>
m.val === ViewMode.BlockingMap && isWavelet(appState.transformType)
Expand All @@ -50,10 +46,6 @@
ARTIFACT_MODES.find(m => m.val === appState.currentViewMode) ?? ARTIFACT_MODES[0]
);

type MetricState = 'good' | 'moderate' | 'poor';
function getPsnrState(v: number): MetricState { return v >= 40 ? 'good' : v >= 30 ? 'moderate' : 'poor'; }
function getSsimState(v: number): MetricState { return v >= 0.95 ? 'good' : v >= 0.9 ? 'moderate' : 'poor'; }

// --- Viridis false-colour LUT ---
// Stops: [intensity 0-255, R, G, B]
// Black → purple → blue → teal → yellow-green → yellow → white
Expand Down Expand Up @@ -385,8 +377,15 @@
<summary class="sidebar-section-title"><span>Controls</span></summary>
<div class="sidebar-section-body">
<QualityControl idPrefix="artifactQuality" onPreview={previewQuality} />
<CodecSettings idPrefix="art" />
</div>
</details>

<details class="sidebar-section visual-tools-section" open>
<summary class="sidebar-section-title"><span>Display</span></summary>
<div class="sidebar-section-body">
{#if appState.currentViewMode === ViewMode.Artifacts}
<div class="control-group" transition:fade={{duration: 150}}>
<div class="control-group gain-group" transition:fade={{duration: 150}}>
<label class="gain-label">
Error Gain
<span class="gain-value-display">{gainValue.toFixed(1)}×</span>
Expand All @@ -399,43 +398,8 @@
<p class="gain-hint">Amplifies pixel error before clamping to 255. Higher = faint errors more visible.</p>
</div>
{/if}

<div class="control-group">
<div class="group-label">Chroma Subsampling</div>
<div class="toggle-group">
{#each CS_MODES as cs}
<input type="radio" id={cs.id} name="art_chroma"
checked={appState.currentCsMode === cs.val}
onchange={() => appState.currentCsMode = cs.val}>
<label for={cs.id}>{cs.label}</label>
{/each}
</div>
</div>

<div class="control-group">
<div class="group-label">Transform</div>
<div class="toggle-group">
{#each TRANSFORMS as t}
<input type="radio" id="art_transform_{t.slug}" name="art_transform_type"
checked={appState.transformType === t.value}
onchange={() => appState.transformType = t.value}>
<label for="art_transform_{t.slug}" title={t.fullName}>{t.label}</label>
{/each}
</div>
</div>
</div>
</details>

<details class="sidebar-section visual-tools-section" open>
<summary class="sidebar-section-title">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round">
<path d="M12 2L2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5"/>
</svg>
<span>Visual Tools</span>
</summary>
<div class="sidebar-section-body">
<div class="tools-grid">
<button class="tool-btn" class:active={isBlinking} aria-pressed={isBlinking} onclick={toggleBlink} title="Blink Test">
<button class="tool-btn" class:active={isBlinking} aria-pressed={isBlinking} onclick={toggleBlink} title="Blink test: flip repeatedly between the original and the map, so the changes jump out">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"
stroke-linecap="round" stroke-linejoin="round">
<path d="M1 12s4-8 11-8 11 8 11 8-4 8-11 8-11-8-11-8z"/>
Expand All @@ -444,15 +408,15 @@
<span>Blink</span>
</button>

<button class="tool-btn" class:active={showHeatmap} aria-pressed={showHeatmap} onclick={() => showHeatmap = !showHeatmap} title="Toggle Heat Map">
<button class="tool-btn" class:active={showHeatmap} aria-pressed={showHeatmap} onclick={() => showHeatmap = !showHeatmap} title="Colour the map from dark (no change) to yellow and white (large change), instead of grayscale">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"
stroke-linecap="round" stroke-linejoin="round">
<path d="M12 2C6.5 2 2 6.5 2 12s4.5 10 10 10c.926 0 1.648-.746 1.648-1.688 0-.437-.18-.835-.437-1.125-.29-.289-.438-.652-.438-1.125a1.64 1.64 0 0 1 1.668-1.668h1.996c3.051 0 5.555-2.503 5.555-5.554C21.965 6.012 17.461 2 12 2z"/>
</svg>
<span>Heat Map</span>
</button>

<button class="tool-btn" class:active={showBlockGrid} aria-pressed={showBlockGrid} onclick={() => showBlockGrid = !showBlockGrid} title="8×8 Block Grid">
<button class="tool-btn" class:active={showBlockGrid} aria-pressed={showBlockGrid} onclick={() => showBlockGrid = !showBlockGrid} title="Overlay the 8×8 block grid the DCT works on, to line artifacts up with block edges">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"
stroke-linecap="round" stroke-linejoin="round">
<rect x="3" y="3" width="7" height="7"/>
Expand Down Expand Up @@ -489,34 +453,9 @@
</details>

<details class="sidebar-section" open>
<summary class="sidebar-section-title"><span>Metrics</span></summary>
<summary class="sidebar-section-title"><span>Result</span></summary>
<div class="sidebar-section-body">
<div class="compact-metrics-grid">
<div class="metric-header-row">
<div class="metric-col-label">Channel</div>
<div class="metric-col-label">PSNR (dB)</div>
<div class="metric-col-label">SSIM</div>
</div>
{#each [
{ channel: 'Y', psnr: appState.psnr.y, ssim: appState.ssim.y },
{ channel: 'Cr', psnr: appState.psnr.cr, ssim: appState.ssim.cr },
{ channel: 'Cb', psnr: appState.psnr.cb, ssim: appState.ssim.cb },
] as row}
{@const ps = getPsnrState(row.psnr)}
{@const ss = getSsimState(row.ssim)}
<div class="metric-row">
<div class="metric-channel-name">{row.channel}</div>
<div class="metric-cell psnr-cell"
class:stat-good={ps==='good'} class:stat-moderate={ps==='moderate'} class:stat-poor={ps==='poor'}>
{row.psnr.toFixed(2)}
</div>
<div class="metric-cell"
class:stat-good={ss==='good'} class:stat-moderate={ss==='moderate'} class:stat-poor={ss==='poor'}>
{row.ssim.toFixed(4)}
</div>
</div>
{/each}
</div>
<ResultSummary showSize={false} />
</div>
</details>

Expand Down Expand Up @@ -774,6 +713,9 @@
}

/* ===== Error Gain slider ===== */
.gain-group {
margin-bottom: 14px;
}

.gain-label {
display: flex;
Expand Down
Loading
Loading