feat(assets): read a container block by block over byte ranges - #744
Merged
Merged
Conversation
added 3 commits
September 14, 2026 11:10
Exoridus
enabled auto-merge (squash)
September 14, 2026 09:27
Bundle ReportChanges will decrease total bundle size by 19.84MB (-37.86%) ⬇️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: site-server-esmAssets Changed:
App Routes Affected:
view changes for bundle: exo-iife-min-Exo-iifeAssets Changed:
Files in
view changes for bundle: exo-iife-Exo-iifeAssets Changed:
Files in
view changes for bundle: exo-esm-esmAssets Changed:
Files in
view changes for bundle: exo-full-iife-min-Exo-iifeAssets Changed:
view changes for bundle: exo-full-iife-Exo-iifeAssets Changed:
view changes for bundle: exo-esm-modules-esmAssets Changed:
Files in
Files in
Files in
Files in
Files in
Files in
Files in
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 2 of the
.exoablock container: the range-capable reader and the block store, which is where the format's second and third demands - "only what changed is re-fetched" and "fragments" - actually pay. Stage 1 (format, writer, whole-file reader) shipped in #705. No format version changes.exo serveanswers byte rangesAccept-Ranges: bytesgoes on every file response, not only a ranged one: a client decides whether to range-fetch at all from the answer to its first request. Added with it are206plusContent-Range,416for a range starting past the end, and suffix ranges. A multi-range request is answered whole rather than refused - always allowed, and a dev server has no reason to build a multipart body. The SPA fallback always answers whole, because a range over it would address the fallback document rather than the path the client asked for.The reader
ContainerReader.open(url)costs one request: a speculative range over the first 64 KiB. That does double duty. A server honouring it answers206, and the prefix almost always already holds the whole head, so the reader is ready without a second round trip; a server ignoring it answers200with the file, which is the single-request path. Blocks are then fetched individually, decoded blocks are shared for the reader's lifetime, and two entries needing one block await a single fetch.readEntryRangeserves the fragment demand - for an entry stored uncompressed the region maps straight onto a byte range and no block is touched at all.Degradation is tested, not asserted: a server that ignores ranges, and a response carrying
Content-Encoding(where a range would address the encoded stream and every offset in the head would be wrong), both fall back to the whole file, and the tests pin that no further request follows.parseContainerHead(prefix, totalLength)is split out ofparseContainer, which becomes the one-liner over it. Block extents were always validated against the file's total length rather than against the buffer - that was the only real coupling, and it is now a parameter.The block store
cacheApiBlockStore()keys on the block hash rather than on the pack, so two packs sharing a block share the stored entry. It is entirely optional and never fails loudly: nocaches, a blocked bucket, a full quota - each is a miss, and the block is fetched.What changes for callers
loadContainer(url)is unchanged: one request, whole file, the existing cache and connectivity policy.loadContainer(url, { store })selects the block-wise path.The one consequence worth stating: the block store replaces the whole-file cache entry rather than adding to it. Both hold the same bytes at different granularity, and keeping both would store every pack twice - so the block-wise path deliberately bypasses
_acquireContainer, resolving the URL and the application'sfetchOptionsthrough the decoder but fetching for itself.A selection option (
{ only: [...] }) is deliberately not here. It would change the scope's ownership semantics - one claim per entry is what makes unpacking safe next to a scene that shares an asset - and it belongs with the Stage 3 manifest, which is what decides which packs an application wants in the first place.Validation
pnpm gates typecheck,lint,sync,site;pnpm test(13150 passed);pnpm typecheck:packages. 14 new reader tests and 6 new server tests. Two export-surface snapshots updated; their diffs carry exactly the ten symbols this adds.https://claude.ai/code/session_013jqQeGU6gwsQhYDFrKiN1A