Skip to content

P7: compile messages to data - #110

Merged
k0d13 merged 13 commits into
mainfrom
compile-messages
Oct 30, 2026
Merged

k0d13 merged 13 commits into
mainfrom
compile-messages

Conversation

@k0d13

@k0d13 k0d13 commented Sep 6, 2026 •

Copy link
Copy Markdown
Owner

Closes #109, closes #81.

The plugin emitted ICU strings and the runtime parsed and formatted them on demand, so the client got messageformat, its parser, both skeleton parsers, and a format cache whose only job was to avoid paying for the same parse twice. Everything about a message is known at build time, so none of that has to ship.

saykit compile now writes one module per locale, and that module is data:

// Generated by saykit. Do not edit.
export default {
  // {count, plural, one {# article} other {# articles}}
  "c4Ef_2": [3,"_count",{"one":[[6]," article"],"other":[[6]," articles"]}],
};

A view compiles a node into a closure the first time that message is called, and memoises it. What ships is a small evaluator over Intl, and nothing parses anything.

What changed

  • @saykit/config compiles ICU to a Message tree instead of to function source. Styles still resolve at build time, into the Intl options bag they stand for, and skeletons and literal patterns are supported as before
  • A node is an array that starts with its Kind, a number exported from saykit (Value, Number, DateTime, Plural, Ordinal, Select, Hash), and any other array is a concatenation. Operands are descriptor keys rather than nested nodes, so {n, number} is [1, "_n"]. The numbers are the format: a kind is only ever appended, never renumbered
  • A plural or a select compiles to one switch node over its cases, so a plural picks its category once per call, and # is its own node rather than a copy of the value. An ordinal is its own kind rather than a plural with a flag
  • The generated module has no imports, no bound locale and no executable code. The message it came from is written above each entry, as before
  • saykit gains Message and compileMessage(message, locale, options). View.Messages is a record of nodes, and view.call() compiles on first use and caches per view
  • Intl objects are built once per message, not looked up per call. A value that cannot be formatted, such as an invalid date, is written as given with a warning instead of throwing
  • bidiIsolation carries over from Add bidiIsolation option to createCatalogue and createView #115: written values are isolated the way MessageFormat 2 does by default, and 'none' turns it off
  • SayProvider is back to store, or locale and messages (plus options), and <SayProvider> on the server takes no props again — the react-server build reads the established view and hands everything across
  • saykit/runtime is no longer a public entry point. Nothing imports it now that generated modules are data, so it is bundled into the main entry
  • The duration argument type is dropped. No macro could author it, and Intl has nothing to write it with, so a hand-written {s, duration} now writes the bare value, as spellout does
  • The style validator and the compiler resolve styles with the same functions, so a style the transform accepts is one the compiler honours

Stale fallbacks in Metro (#81)

Metro's cache served a stale fallback because the Metro transformer assembled fr.json's fallback chain inside Metro, where editing en.json left fr.json's bytes, and so its cache entry, untouched. That transformer is gone. The CLI merges each locale's fallback chain into its own {locale}.js, and saykit extract --watch recompiles every locale when any catalogue changes, so editing en.json rewrites fr.js and Metro reloads it like any other changed module. Without the watcher, saykit compile does the same.

Why an AST, and not one function per message

#95 proposed emitting the function directly:

export const a1b2 = (v) => `Salut ${v._name}`;

That was the first pass of this branch. It is smaller and faster, and it has one problem with no workaround worth having: a function cannot cross the RSC boundary.

With compiled functions a server component cannot hand its messages to a client tree at all. The app has to import the catalogue in a client module and pass only the locale, which means a providers.tsx per app, a catalogue reachable from a module that is not server-only, and every locale imported eagerly on the client so hydration matches. That branch existed, the nextjs example carried all of it, and it is a real tax for something the compiler is supposed to remove.

Data crosses. So the provider goes back to taking locale and messages, <SayProvider> on the server takes no props, and no example needs a providers module.

Two things fall out of it that are worth having on their own:

  • Distance from ICU. Nothing at runtime knows the nodes came from ICU. Another input format compiles to the same nodes, which is what Idea: MF2 (ICU MessageFormat 2) as an opt-in output format #75 would need
  • An inert module. No imports and no bound locale in the file a bundler reads, so the same messages can serve any locale binding

The cost is one indirection per node at format time, against a template literal the engine JITs, after a compile that happens once per message per locale and only for messages that render. A message with no placeholders is emitted as a bare string and skips all of it.

Examples

  • nextjs: <SayProvider> with no props inside the withSay-wrapped layout, getSay() for lang. No providers module
  • tanstack-start: unchanged, locale and messages from the loader is still the right form
  • Every example's committed .d.ts is now Record<string, Message>

@changeset-bot

changeset-bot Bot commented Sep 6, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5f70e0c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
Name Type
saykit Minor
@saykit/config Minor
@saykit/react Minor
babel-plugin-saykit Minor
unplugin-saykit Minor
@saykit/carbon Minor
@saykit/format-json Minor
@saykit/format-po Minor
@saykit/transform-js Minor
@saykit/transform-jsx Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Sep 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
saykit Ready Ready Preview Oct 30, 2026 8:36am UTC

@github-actions github-actions Bot added dependencies Updates or changes related to project dependencies examples Updates or additions to example apps tests Modifications, additions, or fixes related to testing package: core Related to the core saykit package package: react Related to @saykit/react package: config Related to @saykit/config and the CLI package: babel-plugin Related to babel-plugin-saykit package: unplugin Related to unplugin-saykit website Updates to the documentation website labels Sep 6, 2026
@coderabbitai

coderabbitai Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Too many files!

This PR contains 175 files, which is 75 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 50e78692-3d57-4477-9308-dc6f2d86d635
Commits

Reviewing files that changed from the base of the PR and between f583498 and 5f70e0c.

Files ignored due to path filters: 1
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
Files selected for processing: 175
  • .changeset/lucky-moons-shave.md
  • .gitignore
  • README.md
  • examples/babel/README.md
  • examples/babel/package.json
  • examples/babel/src/locales/en.d.po.ts
  • examples/babel/src/locales/en.d.ts
  • examples/babel/src/locales/fr.d.po.ts
  • examples/babel/src/locales/fr.d.ts
  • examples/babel/src/main.ts
  • examples/browser-extension/_locales/de/messages.d.json.ts
  • examples/browser-extension/_locales/de/messages.d.ts
  • examples/browser-extension/_locales/en/messages.d.json.ts
  • examples/browser-extension/_locales/en/messages.d.ts
  • examples/browser-extension/_locales/fr/messages.d.json.ts
  • examples/browser-extension/_locales/fr/messages.d.ts
  • examples/browser-extension/package.json
  • examples/browser-extension/src/i18n.ts
  • examples/carbon/package.json
  • examples/carbon/src/i18n.ts
  • examples/carbon/src/locales/de.d.json.ts
  • examples/carbon/src/locales/de.d.ts
  • examples/carbon/src/locales/en-US.d.json.ts
  • examples/carbon/src/locales/en-US.d.ts
  • examples/carbon/src/locales/fr.d.json.ts
  • examples/carbon/src/locales/fr.d.ts
  • examples/carbon/src/locales/ja.d.json.ts
  • examples/carbon/src/locales/ja.d.ts
  • examples/custom-formatter/package.json
  • examples/custom-formatter/src/i18n.ts
  • examples/custom-formatter/src/locales/de.d.ts
  • examples/custom-formatter/src/locales/de.d.yml.ts
  • examples/custom-formatter/src/locales/en.d.ts
  • examples/custom-formatter/src/locales/en.d.yml.ts
  • examples/custom-formatter/src/locales/fr.d.ts
  • examples/custom-formatter/src/locales/fr.d.yml.ts
  • examples/expo/README.md
  • examples/expo/babel.config.js
  • examples/expo/metro.config.js
  • examples/expo/package.json
  • examples/expo/src/i18n.ts
  • examples/expo/src/locales/en.d.json.ts
  • examples/expo/src/locales/en.d.ts
  • examples/expo/src/locales/fr.d.json.ts
  • examples/expo/src/locales/fr.d.ts
  • examples/expo/src/locales/ja.d.json.ts
  • examples/expo/src/locales/ja.d.ts
  • examples/nextjs/.babelrc
  • examples/nextjs/README.md
  • examples/nextjs/next.config.mjs
  • examples/nextjs/package.json
  • examples/nextjs/src/i18n.ts
  • examples/nextjs/src/locales/en.d.po.ts
  • examples/nextjs/src/locales/en.d.ts
  • examples/nextjs/src/locales/fr.d.po.ts
  • examples/nextjs/src/locales/fr.d.ts
  • examples/nextjs/src/locales/pl.d.po.ts
  • examples/nextjs/src/locales/pl.d.ts
  • examples/react/package.json
  • examples/react/src/i18n.ts
  • examples/react/src/locales/en.d.po.ts
  • examples/react/src/locales/en.d.ts
  • examples/react/src/locales/fr.d.po.ts
  • examples/react/src/locales/fr.d.ts
  • examples/react/src/locales/ja.d.po.ts
  • examples/react/src/locales/ja.d.ts
  • examples/react/src/locales/pl.d.po.ts
  • examples/react/src/locales/pl.d.ts
  • examples/tanstack-start/package.json
  • examples/tanstack-start/src/i18n.ts
  • examples/tanstack-start/src/locales/en-GB.d.json.ts
  • examples/tanstack-start/src/locales/en-GB.d.ts
  • examples/tanstack-start/src/locales/en-NZ.d.json.ts
  • examples/tanstack-start/src/locales/en-NZ.d.ts
  • examples/tanstack-start/src/locales/en.d.json.ts
  • examples/tanstack-start/src/locales/en.d.ts
  • examples/tanstack-start/src/locales/fr.d.json.ts
  • examples/tanstack-start/src/locales/fr.d.ts
  • examples/vanilla/README.md
  • examples/vanilla/package.json
  • examples/vanilla/src/i18n.ts
  • examples/vanilla/src/locales/en.d.po.ts
  • examples/vanilla/src/locales/en.d.ts
  • examples/vanilla/src/locales/fr.d.po.ts
  • examples/vanilla/src/locales/fr.d.ts
  • examples/vanilla/src/locales/ja.d.po.ts
  • examples/vanilla/src/locales/ja.d.ts
  • examples/vanilla/src/locales/pl.d.po.ts
  • examples/vanilla/src/locales/pl.d.ts
  • mise.toml
  • package.json
  • packages/config/README.md
  • packages/config/package.json
  • packages/config/src/commands/compile.ts
  • packages/config/src/commands/index.ts
  • packages/config/src/features/catalogue/emit.test.ts
  • packages/config/src/features/catalogue/emit.ts
  • packages/config/src/features/catalogue/index.ts
  • packages/config/src/features/catalogue/path.ts
  • packages/config/src/features/catalogue/storage.test.ts
  • packages/config/src/features/catalogue/storage.ts
  • packages/config/src/features/messages/compile.test.ts
  • packages/config/src/features/messages/compile.ts
  • packages/config/src/features/messages/format.ts
  • packages/config/src/features/messages/index.ts
  • packages/config/src/features/messages/styles.test.ts
  • packages/config/src/features/messages/styles.ts
  • packages/config/src/features/messages/types.ts
  • packages/config/src/features/workers/extract-worker.test.ts
  • packages/config/src/features/workers/extract-worker.ts
  • packages/integration-react/README.md
  • packages/integration-react/src/runtime/client.server.ts
  • packages/integration-react/src/runtime/client.test.tsx
  • packages/integration-react/src/runtime/client.ts
  • packages/integration-react/src/runtime/index.ts
  • packages/integration-react/src/runtime/server.test.tsx
  • packages/integration-react/src/runtime/server.ts
  • packages/integration/README.md
  • packages/integration/package.json
  • packages/integration/src/catalogue.test.ts
  • packages/integration/src/catalogue.ts
  • packages/integration/src/index.ts
  • packages/integration/src/message.test.ts
  • packages/integration/src/message.ts
  • packages/integration/src/messageformat/convert.ts
  • packages/integration/src/messageformat/functions.ts
  • packages/integration/src/messageformat/index.test.ts
  • packages/integration/src/messageformat/index.ts
  • packages/integration/src/messageformat/options.ts
  • packages/integration/src/messageformat/styles.ts
  • packages/integration/src/messageformat/values.ts
  • packages/integration/src/runtime.test.ts
  • packages/integration/src/runtime.ts
  • packages/integration/src/store.test.ts
  • packages/integration/src/view.test.ts
  • packages/integration/src/view.ts
  • packages/plugin-babel/README.md
  • packages/plugin-babel/package.json
  • packages/plugin-babel/src/catalogue.ts
  • packages/plugin-babel/src/index.test.ts
  • packages/plugin-babel/src/index.ts
  • packages/plugin-babel/src/metro/index.ts
  • packages/plugin-babel/src/metro/transformer.ts
  • packages/plugin-babel/src/next/index.ts
  • packages/plugin-babel/src/next/loader.ts
  • packages/plugin-babel/tsdown.config.ts
  • packages/plugin-unplugin/package.json
  • packages/plugin-unplugin/src/index.test.ts
  • packages/plugin-unplugin/src/index.ts
  • packages/transform-js/src/generator.ts
  • packages/transform-js/src/index.test.ts
  • packages/transform-jsx/src/generator.ts
  • packages/transform-jsx/src/index.test.ts
  • skills/saykit/SKILL.md
  • website/content/core-concepts/architecture.mdx
  • website/content/core-concepts/configuration.mdx
  • website/content/core-concepts/extraction.mdx
  • website/content/core-concepts/formats.mdx
  • website/content/core-concepts/messages.mdx
  • website/content/core-concepts/runtime.mdx
  • website/content/getting-started/installation.mdx
  • website/content/getting-started/introduction.mdx
  • website/content/getting-started/quickstart.mdx
  • website/content/guides/custom-formatter.mdx
  • website/content/guides/dynamic-loading.mdx
  • website/content/guides/typed-messages.mdx
  • website/content/integrations/babel.mdx
  • website/content/integrations/carbon.mdx
  • website/content/integrations/react.mdx
  • website/content/integrations/vite.mdx
  • website/content/reference/api/config.mdx
  • website/content/reference/api/react.mdx
  • website/content/reference/api/saykit.mdx
  • website/content/reference/cli.mdx
  • website/package.json

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added package: transform-js Related to @saykit/transform-js package: transform-jsx Related to @saykit/transform-jsx labels Sep 21, 2026
@k0d13

k0d13 commented Sep 21, 2026

Copy link
Copy Markdown
Owner Author

@greptileai review @coderabbitai review

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

@k0d13 I will review the changes in #110.

⚠️ Action not completed

Review skipped: 156 files exceed the limit of 100.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@k0d13 k0d13 added the preview Publish a preview build and link it from the pull request label Sep 21, 2026
@pkg-pr-new

pkg-pr-new Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@saykit/config

npm i https://pkg.pr.new/@saykit/config@110

@saykit/format-json

npm i https://pkg.pr.new/@saykit/format-json@110

@saykit/format-po

npm i https://pkg.pr.new/@saykit/format-po@110

saykit

npm i https://pkg.pr.new/saykit@110

@saykit/carbon

npm i https://pkg.pr.new/@saykit/carbon@110

@saykit/react

npm i https://pkg.pr.new/@saykit/react@110

babel-plugin-saykit

npm i https://pkg.pr.new/babel-plugin-saykit@110

unplugin-saykit

npm i https://pkg.pr.new/unplugin-saykit@110

@saykit/transform-js

npm i https://pkg.pr.new/@saykit/transform-js@110

@saykit/transform-jsx

npm i https://pkg.pr.new/@saykit/transform-jsx@110

commit: dbbf8f9

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

The preview build of dbbf8f9 is published. Open it in the playground to run this pull request against your own code, straight from the browser.

@github-actions github-actions Bot removed the preview Publish a preview build and link it from the pull request label Sep 21, 2026
@k0d13

k0d13 commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

@greptile review

@k0d13 k0d13 added the preview Publish a preview build and link it from the pull request label Oct 17, 2026
@github-actions github-actions Bot removed the preview Publish a preview build and link it from the pull request label Oct 17, 2026
@k0d13

k0d13 commented Oct 27, 2026

Copy link
Copy Markdown
Owner Author

@kevinatbuape what are your thoughts on this pr, and how it might work with kiai

@kevinatbuape

Copy link
Copy Markdown

@k0d13 Based on the design described here, moving ICU parsing to build time looks like a good fit for Kiai: a smaller dashboard runtime, with the same compiled catalogues potentially reusable for bot responses. Serializable messages also make server-to-client locale handoff cleaner. I’d keep locale/view caches request-scoped on the server so concurrent users never share the wrong locale, and measure real catalogue size plus cold/steady-state formatting before calling it a performance win.

For adoption, I’d trial one dashboard flow and one bot response, checking plural/select behaviour, fallback locales, date/number output and Discord mention/Markdown rendering with bidi isolation. This is an architectural take from the PR description, not a code review or tested integration. Request my review through GitHub if you want the full PR review workflow.

@k0d13 k0d13 added the preview Publish a preview build and link it from the pull request label Oct 30, 2026
@github-actions github-actions Bot removed the preview Publish a preview build and link it from the pull request label Oct 30, 2026
@k0d13 k0d13 added the preview Publish a preview build and link it from the pull request label Oct 30, 2026
@github-actions github-actions Bot removed the preview Publish a preview build and link it from the pull request label Oct 30, 2026
@k0d13

k0d13 commented Oct 30, 2026

Copy link
Copy Markdown
Owner Author

@greptile review

Comment on lines +113 to +120
if (this.bucket.match(event.filename)) {
await this.update(path);
continue;
}

// A translation changed, not a call site: recompile only, since extraction writes just the
// source locale
if (this.bucket.output.match(event.filename)) await this.compile();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Watch mode misses translation edits

If a catalogue also matches the bucket's include glob, such as include: ['src/**/*'] with JSON catalogues under src/locales, this branch calls update() and skips the catalogue check. The transformer finds no messages in the JSON, so nothing recompiles and the app keeps showing old translations. Check catalogue paths before source paths.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/config/src/features/workers/extract-worker.ts
Line: 113-120

Comment:
**Watch mode misses translation edits**

If a catalogue also matches the bucket's `include` glob, such as `include: ['src/**/*']` with JSON catalogues under `src/locales`, this branch calls `update()` and skips the catalogue check. The transformer finds no messages in the JSON, so nothing recompiles and the app keeps showing old translations. Check catalogue paths before source paths.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

a message's values in their own namespace, so a value named `id` cannot displace the message being
looked up. `call` strips exactly one underscore back off, and passes keys written without one
through untouched, so a hand-written or custom-transformer-generated call works either way.
The descriptor names a message by `id`, or carries a compiled `message` itself, as for a lone `say.date(x)`. The other keys are values: `call` strips one leading underscore from each, and passes a key without one through as is. Throws if the id is not in the view.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Reference promises unsupported value keys

The call() reference still says it strips underscores and accepts keys without them. The new runtime passes the descriptor unchanged, and compiled messages read exact keys such as _name. Following this advice with call({ id, name: 'Ada' }) leaves the placeholder empty. Document that callers must use the keys stored in the compiled message.

Suggested change
The descriptor names a message by `id`, or carries a compiled `message` itself, as for a lone `say.date(x)`. The other keys are values: `call` strips one leading underscore from each, and passes a key without one through as is. Throws if the id is not in the view.
The descriptor names a message by `id`, or carries a compiled `message` itself, as for a lone `say.date(x)`. The other keys supply values using the exact keys stored in the compiled message. ICU arguments compile to underscore-prefixed keys, so `{name}` reads `_name`; `call` does not strip underscores. Throws if the id is not in the view.
Prompt To Fix With AI
This is a comment left during a code review.
Path: website/content/reference/api/saykit.mdx
Line: 130

Comment:
**Reference promises unsupported value keys**

The `call()` reference still says it strips underscores and accepts keys without them. The new runtime passes the descriptor unchanged, and compiled messages read exact keys such as `_name`. Following this advice with `call({ id, name: 'Ada' })` leaves the placeholder empty. Document that callers must use the keys stored in the compiled message.

```suggestion
The descriptor names a message by `id`, or carries a compiled `message` itself, as for a lone `say.date(x)`. The other keys supply values using the exact keys stored in the compiled message. ICU arguments compile to underscore-prefixed keys, so `{name}` reads `_name`; `call` does not strip underscores. Throws if the id is not in the view.
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

…clude, and document that call strips no underscores
@k0d13
k0d13 merged commit 1475368 into main Oct 30, 2026
9 checks passed
@k0d13
k0d13 deleted the compile-messages branch October 30, 2026 08:36

This branch was successfully deployed

1 active deployment
Preview — 5f70e0c8 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Updates or changes related to project dependencies examples Updates or additions to example apps package: babel-plugin Related to babel-plugin-saykit package: config Related to @saykit/config and the CLI package: core Related to the core saykit package package: react Related to @saykit/react package: transform-js Related to @saykit/transform-js package: transform-jsx Related to @saykit/transform-jsx package: unplugin Related to unplugin-saykit tests Modifications, additions, or fixes related to testing website Updates to the documentation website

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P7: compile messages to data, not to functions Metro: editing a fallback locale leaves dependent catalogues stale

2 participants