Skip to content

feat(net): publish and consume compressed lite-07 tracks in JavaScript - #6230

Merged
kixelated merged 10 commits into
mainfrom
quest/m1/compress/js
Oct 22, 2026
Merged

kixelated merged 10 commits into
mainfrom
quest/m1/compress/js

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

JavaScript refused compressed lite-07 tracks and could not publish them through the networking model. Rust publishers already use the negotiated format, so JS readers need the same bounded storage and plain-frame contract.

Approach

Add Info.compress, keeping model writes synchronous. Store each group's DEFLATE slices once, forward those slices unchanged on lite-07, and build one exact-size plain copy shared by application readers and older sessions. Datagrams have independent windows; malformed compressed datagrams are refused before delivery.

Charge declared plain sizes against the existing group limit before inflation, stop excessive output inside the decoder, retain committed prefixes on cuts, and measure refresh against stored bytes. Complete the JS quest and remove its references.

Impact

  • Additive public API: optional @moq/net Track.Info.compress, default false. Application frame/datagram payloads remain plain, and writes remain synchronous. Codec classes and wire-ingest hooks remain private.
  • Wire: implements the existing lite-07 codec and inflated-size fields. Lite-06 and other sessions receive plain payloads without compression flags. No new format or published-version change.
  • Current consumers: JS applications opting into compressed data tracks, and JS subscribers receiving existing Rust compressed-track publishers through moq-relay.
  • JS retains bounded stored and declared plain data per group; every plain reader shares the same inflated buffers.

Checks

  • Final just ci passes on current main, including the merged fixture repair. No temporary overlay remains.
  • All 2,486 net tests pass. Regressions cover shared inflation, hostile declarations, empty frames, synchronous admission independent of readers, cut prefixes, stored-byte refresh, immutable route metadata, FETCH, datagrams and lite-06 fallback.
  • just test interop --all passes. Added Rust ↔ JS Node/Bun compressed exchange through moq-relay on lite-07, plus lite-06 plain fallback, with multiple frames per window and an empty frame.
  • just drafts check and quest check pass.
  • Maintained just js bench compression sweeps 1/4/16 publishers × 1/4/16 readers. Each publisher stores 740 bytes and inflates 71,680 bytes once for the fixture, independent of reader count.
  • Consumer-only minified Bun entry: 31,410 gzip bytes versus 19,934 on main. Isolated decoder: 10,969; decoder plus encoder: 13,984, a 3,015-byte static encoder increment.

Alternatives

The maintainer approved the static encoder cost to preserve synchronous writes. Async pending writes and a preparation API were rejected. Both codecs now import one pako module so their helpers are shared.

Follow-ups

No open decisions. Native browser codec work is separately planned. The existing compression migration quest owns consumer migration and removing @moq/flate; this PR keeps that scope separate.

(Written by GPT-6)

kixelated and others added 5 commits October 21, 2026 15:55
Co-Authored-By: GPT-6 <noreply@openai.com>
Prepare the group-scoped codec foundation without changing the model, wire behavior, or advertised capabilities. The decoder enforces exact declared sizes, stops oversized inflation inside pako onData, rejects final DEFLATE blocks, and retains tight frame buffers. The encoder is isolated behind pako browser deflate for eventual lazy loading.

Unresolved: synchronous writes currently require a static compressor, measured at 2870 gzip bytes beyond Inflate alone. Lazy loading requires observable pending-write timing or an explicit preparation API. Planning PR5885 and core PR5928 do not settle that JS choice. No writer timing or public API decision is implemented.

Passed: 5 private codec tests (14 assertions), js/net type checks including examples and benchmarks, Biome, git diff --check. Full CI, drafts, Rust/JS exchange, interop --all, integrated bundles and fanout remain unrun because the feature is not integrated.

Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Keep model writes synchronous, store slices once per group, and share exact-size inflation across plain readers. Serve older sessions plain frames and exercise Rust/JS relay exchange and lite-06 fallback.

Co-Authored-By: GPT-6 <noreply@openai.com>
@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ac39b8b4-f1e0-40ee-b9bd-4c03f187ba38
📥 Commits

Reviewing files that changed from the base of the PR and between b15af4a and 4aecc39.

⛔ Files ignored due to path filters: 1
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing: 36
  • doc/concept/moq-lite.md
  • js/justfile
  • js/net/bench/compression.ts
  • js/net/bench/fixtures/compression-consumer.ts
  • js/net/bench/frames.ts
  • js/net/bench/lite-varint.ts
  • js/net/package.json
  • js/net/src/flate/codec.test.ts
  • js/net/src/flate/decoder.ts
  • js/net/src/flate/encoder.ts
  • js/net/src/flate/group.test.ts
  • js/net/src/flate/group.ts
  • js/net/src/group.ts
  • js/net/src/internal.ts
  • js/net/src/lite/compression.test.ts
  • js/net/src/lite/datagram.test.ts
  • js/net/src/lite/datagram.ts
  • js/net/src/lite/group.test.ts
  • js/net/src/lite/group.ts
  • js/net/src/lite/publisher.ts
  • js/net/src/lite/subscriber.ts
  • js/net/src/resume.test.ts
  • js/net/src/resume.ts
  • js/net/src/track.ts
  • quest/m1/audit-size.md
  • quest/m1/compress/README.md
  • quest/m1/compress/app-range-widen.md
  • quest/m1/compress/js.md
  • quest/m1/compress/migrate.md
  • quest/m1/lite07-finalize.md
  • quest/m2/native-compression-adopt.md
  • test/interop/README.md
  • test/interop/clients/js-native/tail.ts
  • test/interop/clients/rust/tail.rs
  • test/interop/interop.sh
  • test/interop/lite-varint.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T11:02:57.129922Z 4aecc39 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Collaborator Author

Quest outcome: complete, with the maintainer-approved synchronous write contract and static encoder tradeoff. Final just ci, all 2,486 net tests, full just test interop --all, drafts and quest checks pass on the published head. Codex completed its review of 231a78db6e24ece0f021d648c533454e85d91681 with no findings.

The maintained publisher × reader benchmark confirms one shared inflated copy, independent of reader count. The standard sh/size/bundles.ts first-load gzip measurements are net 113,506 bytes, watch 161,891, watch-ui 174,261 and publish 150,744. The separate consumer-only probe and 3,015-byte isolated encoder increment are recorded in the PR body.

No open decisions or additional follow-up quests. Existing consumer migration and native codec plans retain their separate scope. The PR is ready; merge and queue actions remain with the batch coordinator.

(Written by GPT-6)

kixelated and others added 2 commits October 21, 2026 22:58
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main into the branch (2102dba); no force-push.

On the merged head: just check passes (net 2,511 tests, 0 fail), just test interop --tail passes every plain and compression lane on lite-07-wip and lite-06 for js-native-node and js-native-bun, and the rust/js/js-native matrix passes. The Codex review of 231a78d had no findings, and everything since is mechanical, so enabling auto-merge.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 22, 2026
kixelated and others added 2 commits October 22, 2026 02:23
Combine lite-05/06 timestamp dropping (#6150) with compressed frame decoding:
frameDecoder and readFrames take one FrameFormat of scale, timed, and compress,
and the publisher threads both the Stamp and the compress flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged main after the queue dropped this for a conflict with moq-dev/moq#6150 (lite-05/06 timestamp dropping).

  • frameDecoder and readFrames now take one internal FrameFormat { scale, timed, compress } so both the dropped-timestamp and declared-size reads apply; callers in the subscriber, bench, and test/interop/lite-varint.ts updated.
  • The publisher threads main's Stamp alongside compress everywhere timescale used to go; TRACK_INFO keeps liteTimescale(info) plus the compression flag.
  • infoDefaults keeps both the compress type check and main's unchecked validation.
  • quest/m2/native-compression-adopt.md (from moq-dev/moq#6227) required the js.md quest this PR deletes; dropped that finished blocker.

Checks: just check, all 2,519 js/net tests, biome, quest check --base origin/main, and just test interop --all pass.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f87cd23c94

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/lite/publisher.ts
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 22, 2026
Main's interop harness now runs the tails once per pinned version
(#6246). The compress tail lanes pin their own lite versions like the
drop lanes, so they run in the auto pass and are planned as
case=compress with their version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main (4aecc39) after the queue dropped this for a conflict.

  • quest/m1/compress/README.md: both sides removed their own Required entry (js.md here, ietf.md via feat(net): negotiate IETF compressed tracks from draft17 #6228), so both lines are gone.
  • test/interop: test(interop): run the matrix on each even moq-transport draft #6246 moved the tails into a per-version run_tails pass. The compress lanes now pin their own lite versions like the drop lanes (COMPRESS_VERSIONS=(moq-lite-07-wip moq-lite-06)), run only in the auto pass, record as case=compress version=<v>, and are planned in plan_pass. run_tail_pair <pub> <sub> [<mode> <version>] keeps main's pinned-version/join path when no mode is given, and the version check now covers the compress lanes.
  • tail.rs/tail.ts: compress joins main's validated mode list; Rust's new write() takes compress, with a shared sizes() for writer and reader.

Checks: quest check (and --base origin/main), js/net tsc + 2524 tests, biome, interop client tsc, clippy + rustfmt on interop-tail, and interop.sh --tail with rust/node/bun: all 21 cases pass, including the 8 compress lanes.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 7f909af Oct 22, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/compress/js branch October 22, 2026 12:12
kixelated added a commit that referenced this pull request Oct 22, 2026
Main's #6230 makes js/net read and write compressed lite-07 tracks, so the listing carries compression instead of refusing it: TRACKS_ADD answers the same TRACK_INFO as a TRACK stream, compression included, mirroring Rust's TrackInfo::serve.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 22, 2026
Resolves js/net resume.ts and resume.test.ts against #6251, #6230, #6222
and #6209: keeps main's verdict check and identity-based datagram dedup,
and tracks the newest datagram handed out separately so a declared end is
still checked against it now that `#datagram` is the last one. Renames
`advanceAsync` to `advance` for the new manual clock. Drops the quest
links to the finished resume-closed, recovery-verdicts and
resume-datagrams quests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant