Skip to content

implementation.md

Latest commit

 

History

History
193 lines (160 loc) · 11.6 KB

implementation.md

File metadata and controls

193 lines (160 loc) · 11.6 KB
Raw

Hexlode implementation plan

Updated: 2026-09-27 Product: idea.md. Vocabulary: CONTEXT.md. Decisions: docs/adr/.

Goal

Version 1 is the complete local product: the home page, six quick tools, pipeline tools, and the Studio with all 33 nodes from the catalogue in idea.md. It ships in three phases. Phase 1 builds the whole application and node batch 1. Phases 2 and 3 only add node batches 2 and 3. Each phase is deployable and passes its exit gate before the next phase starts.

Active phase: Phase 1.

Phase 1: the application and node batch 1

Cleanup

  1. Remove the old canvas UI, recipe dialog, privacy modes (Private Session and Airgap), Codec Tournament, macro presets and debugger.
  2. Keep code that fits the new engine: input validation, the worker message protocol, the serial batch queue and streaming ZIP output. Move each piece into the new layout when the engine needs it.
  3. Remove the MCP demo (src/routes/mcp.ts, src/mcp-todos.ts, src/utils/mcp-handler.ts).
  4. Keep Drizzle and Better Auth compiling and unused (ADR 0006).

Foundation

  1. Replace node:test with Vitest, with browser mode on Playwright Chromium for worker, OPFS and codec tests.
  2. Set up PostHog and Sentry as described in Analytics below, and add the privacy page.
  3. Add a Dockerfile and deploy on Dokploy. CI publishes the image to GHCR, and the server reads the public VITE_POSTHOG_KEY, VITE_POSTHOG_HOST and VITE_SENTRY_DSN from its environment at runtime (ADR 0008).

Engine

  1. Pipeline model, node definitions with accepts and produces, compatibility checks, worker pool, cancellation and estimates.
  2. jSquash codecs, loaded per format on first use.
  3. Step cache in OPFS with the 5 GB budget, and incremental runs.
  4. Output storage in OPFS and ZIP or folder delivery.
  5. Live previews on a sample image.

Application

  1. Home page and the six quick tools: Convert, Compress, Resize, Crop, Rotate, Strip metadata. The home page shows a Studio screenshot taken from a real run. Its Studio scenes are GSAP timelines started by ScrollTrigger (useScene in src/features/home/scene.ts); interface motion elsewhere uses Motion (motion/react). See ADR 0009. The site frame lives in the root route so the top bar animates between pages. The Hexlode theme in src/features/theme/, with dark, light and system colour modes and a self-hosted Figtree font.
  2. Studio: node library with every category, drag, search, category filter and a folded rail, inspector, template picker, live previews, run statistics on nodes and connections, undo and redo, right-click menus, a per-tab draft that survives a reload, narrow-screen message.
  3. Save in browser storage, .hexlode export and import, pipeline tools.

Node batch 1

  1. Files, Filter, Inspect, Resize, Crop, Rotate / Flip, Strip metadata, Convert, Compress to size, Optimize PNG, Rename, Output, Compare.

A template appears in the template picker once all its nodes exist. Phase 1 ships Web-ready photos, Photos for email, Remove location, Square thumbnails, WebP and AVIF, and Blank.

Exit gate

  • The app starts from a fresh clone with no .env.local.
  • Every quick tool and template runs on real JPEG, PNG, WebP, AVIF, JPEG XL and QOI fixtures.
  • A batch of 500 generated 12-megapixel images completes without memory growing with batch size.
  • Changing a node's setting and running again executes only that node and the nodes after it.
  • A .hexlode file round-trips without changes.
  • The node pair matrix passes (see Testing).
  • pnpm validate passes and the app runs in the Docker image.

Phase 2: node batch 2

Auto-trim, Pad / Extend, Pixel-art upscale, Split / Tile, Adjust, Filters, Sharpen / Blur, Background, Text watermark, Image watermark, Border / Rounded corners, Deduplicate. Adds the Watermark and compress and Instagram carousel templates.

Exit gate: each node has output tests on real fixtures, the node pair matrix passes with the new nodes, and the new templates run end to end.

Phase 3: node batch 3

Best format, Responsive set, Favicon / App icons, Placeholder, Palette, Contact sheet, Images to PDF, Set copyright / author. Adds the Responsive image set template.

Exit gate: same as phase 2, plus tests for nodes that change the item count (one image to many, many images to one) and nodes that produce data or documents.

Engine design

  • A pipeline is a directed graph without cycles. Items flow through it one by one, so an item can reach the last node while later items are still at the first node.
  • Nodes that combine items (Contact sheet, Images to PDF, near-duplicate Deduplicate) wait until every upstream item has arrived.
  • Decoding happens at most once per item, the first time a node needs pixels; nodes that only read headers or metadata (Filter by format, Inspect, Rename, Strip metadata) never decode. Nodes pass pixels between them. Encoding happens at Convert, Compress to size and Optimize PNG. An image that reaches Output without an encoding node keeps its source format, and its original bytes when nothing changed its pixels.
  • Metadata is rewritten in the container without re-encoding for JPEG, PNG, WebP and JPEG XL. AVIF metadata is read and can be removed in place but not added; QOI holds none. The run reports a warning when a format cannot keep metadata.
  • Each source item travels through the whole pipeline inside one worker, so pixels never cross threads. The pool admits as many items as it has workers, which bounds memory by pool size, not batch size. Combining nodes spill their inputs to OPFS and run after everything upstream is done.
  • Items carry their metadata. Encoders write it back where the format supports it. Strip metadata is the only node that removes metadata; when an encoder cannot keep metadata, the run reports a warning.
  • A worker pool runs the tasks. The pool size depends on CPU cores and the memory estimate of the largest item in flight.
  • An item a node cannot accept skips that branch (ADR 0003).
  • The step cache stores each node's last results in OPFS, keeping the encoded form of an item instead of its pixels when both exist. Nodes that only change metadata or names store a reference to their input instead of a copy. The cache key combines the node's settings and the cache keys of its inputs, so a settings change invalidates that node and every node after it (ADR 0004).
  • The step cache has a 5 GB budget by default, which the user can change in settings. When it is full, the least recently used results are deleted. A node whose results were deleted runs again from the nearest earlier node that still has a step cache, and the estimate includes that work.
  • The engine is plain TypeScript with no React. The UI subscribes to engine events.

Testing

We write tests first, using the tdd skill. A test must fail when the behaviour it covers is removed.

  • Node tests decode the file the node produces and check format, dimensions, sample pixel values and metadata.
  • Fixtures are small files in the repository, generated or with a licence that allows it, covering every input format, transparency, EXIF orientation, location metadata and a malformed file.
  • The node pair matrix connects every pair of node types and checks that the Studio's accept or refuse decision matches what the engine does when it runs that pair.
  • Worker, OPFS, codec and ZIP tests run in real browsers through Vitest browser mode. Locally they run in Chromium with PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH; VITEST_BROWSERS=firefox or webkit picks another engine. CI runs Chromium, Firefox and WebKit. Playwright's WebKit has no working OPFS, so WebKit skips the OPFS suites listed in vitest.config.ts.
  • pnpm test:coverage runs the unit and browser tests with coverage. The floors in vitest.config.ts sit a few points under current coverage; raise them as coverage grows.
  • Test-only node types with data and document items, one-to-many and many-to-one behaviour prove what batches 2 and 3 need. They join the node pair matrix but never the product registry.
  • The batch of 500 images of 12 megapixels runs with pnpm test:scale. It takes minutes, so it is outside pnpm validate; run it before closing a phase.
  • pnpm validate passes before every commit.
  • CI (.github/workflows/) runs on every pull request: Biome, types, commit messages, the generated theme and route tree, actionlint and hadolint; unit tests on Linux, macOS and Windows; browser tests in three engines; coverage; the build with a smoke test of the server; and the Docker image with a smoke test of the container, its health check, runtime settings and an ARM build. CodeQL, pnpm audit, dependency review and PR titles run in their own workflows. A push to main only publishes the main image and deploys staging, since the pull request already ran the checks; CodeQL, the audit and the scale test also run on a schedule.
  • Release Please keeps a release pull request open on main from the Conventional Commits merged there; merging it tags the version, writes the changelog, publishes latest and the version tags, and deploys.

Analytics

  • PostHog starts as its TanStack Start guide shows, with cookieless_mode: 'always' and person_profiles: 'never'. It captures pageviews, page leaves, clicks, heatmaps and web vitals with element text and attributes masked; session replay is off. The app also sends its own events from one analytics module (ADR 0005).
  • Sentry starts as its TanStack Start guide shows: src/client.tsx in the browser, instrument.server.mjs on the server, src/server.ts and the global middlewares in src/start.ts. It sends errors, logs and a fifth of traces, with sendDefaultPii: false and no replay, through a same-origin tunnel route. File names are removed from messages, exceptions, breadcrumbs and logs before sending. Source maps upload at build time when SENTRY_AUTH_TOKEN, SENTRY_ORG and SENTRY_PROJECT are set.
  • The browser reads the public settings from a hexlode-config meta tag the root route writes, so both start before hydration. Their libraries load on their own, so a content blocker cannot stop the app.
  • The PostHog project must have cookieless mode enabled and "Discard client IP data" turned on; without the first, PostHog ignores cookieless events. The client must not clear $ip: PostHog hashes it into the daily anonymous ID and drops cookieless events without it.
  • The event catalogue lives in src/features/usage/events.ts, and the privacy page lists it.

Deployment

The app runs as one Docker container that serves the Nitro build. The Docker image workflow publishes ghcr.io/pixelactstudio/hexlode for x86 and ARM: main on every merge to main, which deploys the staging application, and latest on every release, which deploys production. Each deploy calls that application's Dokploy webhook on the maintainer's VPS. Staging sets HEXLODE_ENVIRONMENT=staging, so Sentry files its reports under staging and PostHog's test account filter keeps its events out of production dashboards. Dokploy pulls the image, sets its environment and handles the domain and HTTPS (ADR 0008). /api/health answers the container health check that Dokploy's zero-downtime updates wait for. A future cloud mode adds a Postgres service on the same VPS, reached through DATABASE_URL. DEPLOY.md has the steps.