Repository navigation
Honor a default source's expectedLength in TypeScript streams - #8050
Conversation
|
💬 Approved with comments — The approver's follow-ups (underflow enforcement and the draining reader's length after a read) are fixed. The only remaining finding is an INFO about a comment that overstates how closely the C++ implementation matches on underflow. Findings: 1 info Not opened by the correctness review: 1 changed file it was meant to read
The decision is advisory: this comment never approves, blocks or requests changes on the pull request. 📝 Documentation: 💬 1 infoOne comment overstates how closely the C++ implementation matches on underflow. Recommendation: 💬 Optional Nits
View full reviewThe comment at The earlier INFO about the ✅ All other checks passed🔒 Security: ✅ LGTMBody consumption now rejects a body that delivers more or fewer bytes than it declared, and cancels the stream. A partly read stream, or a fork made after a read, reports no length, so a body cannot advertise a Content-Length larger than what is left to send. The internal 📖 AGENTS.md: ✅ LGTMThe AGENTS.md ledger updates match the tests and configuration changes. 📋 Code Quality: ✅ LGTM
⚡ Performance: ✅ LGTMThe default controller reads Reviewed commit: |
|
Since last review: 0 resolved, 0 still open, 0 new. Not re-run: api-compat, tests, docs, compat-flags, design-simplicity, jsg-gc, kj-style, memory-safety (no author changes in their files since the last review) Specialist findings not posted (1)
Reviewed commit: 30509c11 · github run |
b1f9752 to
86a01dc
Compare
86a01dc to
57dd431
Compare
Under typescript_implemented_streams, the default controller ignored the expectedLength its underlying source declared: only byte sources, a TransformStream's readable (through an internal setter) and native streams reported a length. A body built from a default source that declared its length was therefore sent with chunked encoding, where the C++ implementation sends a Content-Length. The default controller now caches the converted source's expectedLength, as the byte controller does, and TransformStream passes its transformer's through the source it builds for its readable, which retires the internal setter. Body consumption already treated a TransformStream readable's declaration as an exact total (readable ledger #21); that now covers any default source that declares one. A source that delivers more than it declared rejects text(), arrayBuffer() and the other consumers with a RangeError and is cancelled with it, where the C++ implementation returns everything. The divergence is intentional: the declaration is the exact-total contract that byte and native sources already enforce. typescript_implemented_streams is still experimental, with no enable date, so the change needs no compatibility flag. Fixes: #8040
A queued TypeScript stream reported the expectedLength its source declared to the C++ bridge, which uses it as a body's Content-Length, even after part of the stream had been read. The declaration is the stream's total, and the controller, which tee branches share, does not count what a stream has already delivered. So a tee branch made after a read, or the copy detach() makes of one (as new Request() does with a body), sent the full declared length as its Content-Length, and the request failed with "Network connection lost." when the body fell short. A partly read stream handed to JS RPC, KV put() or R2 put() announced its full total the same way. A queued stream now reports its declared length only while nothing has been read from it or, for a tee branch or detached copy, before the fork. makeShell() records whether the stream it forks had been read, and getReadableStreamExpectedLength() checks that record along with whether the stream itself has been read. Native streams are unchanged: their forks are fresh streams over sources that count what is left, and KV, R2 and RPC reject a partly read native stream. This leaves three intentional divergences from the C++ implementation, pinned in the streams suites and the cross-implementation RPC test: - A fork made before any read keeps the declared length in TypeScript. C++ tee branches never carry a length. - A partly read stream reports no length in TypeScript: RPC and KV send it chunked, and R2 put() rejects it for want of a known length. C++ still reports the declared total, so the transfer fails. - A FixedLengthStream branch teed after a read goes out chunked in TypeScript. C++ carries what the branch has left. The identity suite gains a SELF echo to observe a FixedLengthStream's Content-Length. typescript_implemented_streams is still experimental, with no enable date, so the change needs no compatibility flag.
A response whose body declares expectedLength and carries Content-Encoding: gzip is compressed on the way out, so its uncompressed declaration is not a valid Content-Length. The C++ bridge already reports no length for a queued TypeScript stream whose body is encoded, as for a FixedLengthStream, but only a C++ unit test checked it. The readable and readable-byte suites now serve such a response through SELF from a default source, a TransformStream readable and a byte source. Under TypeScript it goes out chunked and decodes in full. The C++ implementation keeps the uncompressed length as the Content-Length, so the compressed body falls short of it and the reader fails with "Network connection lost." That is a pre-existing C++ defect, pinned as a divergence (readable ledger #37, readable-byte ledger #46) rather than reproduced. Refs: #8040 (comment)
A default source or TransformStream readable whose declared expectedLength is sent as the Content-Length, but which delivers less than it declared, fails the transfer: as a request body the receiver's read rejects with "Network connection lost.", and as a response body the client's does. The transport enforces the framing it was given, so a short body is never taken for a complete one. The C++ implementation behaves the same, so this is pinned as parity. The readable suite's SELF service serves the short response at /short, and the body helper behind the /gzip response now takes the declared length and the text to deliver. The parity note in the suite's AGENTS.md also now lists ledger #37 among the integration divergences.
sinkReleasedOnceEnded flaked under typescript_implemented_streams (the write-rejects case, on macOS and Linux CI; seen on unrelated branches too): a stream that had ended still kept its underlying sink alive. The write, close and abort algorithms were closures over the sink, and ClearAlgorithms dropped them, but once V8 had JIT-compiled them their context, and the sink in it, could stay reachable. With Maglev tiering up early (--invocation-count-for- maglev=5 --no-turbofan --minimum-invocations-after-ic-update=2) the test fails every run under TypeScript and passes under C++. The controller now holds the sink and its uncurried hooks in fields that ClearAlgorithms clears, and calls them from private methods, so no closure captures the sink. The test passes every run with those flags.
sinkReleasedOnceEnded checks that an ended stream releases its sink, but it ran before V8 had optimized the TypeScript sink algorithms in most runs, so it rarely caught closures over the sink kept alive by optimized code. Both main cells now pass --invocation-count-for-maglev=5 and --minimum-invocations-after-ic-update=2, so the algorithms are optimized by the time gc.js runs. Under them the closure-based algorithms fail the test in 30 of 36 runs across the three variants; the field-based ones pass all 36, and the C++ cell passes too. A test that warms the algorithms inside gc.js does not reproduce the retention, nor does gc.js alone in a cell of its own with the same flags: it takes the state the rest of the suite leaves behind, so the flags go on the full cells.
Body consumption under typescript_implemented_streams treated a default source's or TransformStream readable's declared expectedLength as an exact total only on overflow: a stream closing short of its declaration resolved with what had arrived. It now rejects text(), arrayBuffer() and the other consumers with a RangeError, as byte and native sources already do by erroring at close(), and as the transport does by failing a short body. The stream has already closed, so there is nothing to cancel. Readable ledger #21 now covers both directions; C++ consumption still ignores the declaration. ReadableStreamDrainingReader's expectedLength reported the source's declaration even after the stream had been read, disagreeing with the length the C++ bridge sees. It now asks getReadableStreamExpectedLength(), so a partly read stream, or a fork made after a read, reports none. The C++ sides of readable ledgers #36 and #37 (readable-byte #45 and #46) are now marked as defects: ReadableStreamJsController's tryGetLength() reports the declared total regardless of reads or encoding.
57dd431 to
30509c1
Compare
Under typescript_implemented_streams, the default controller ignored
the expectedLength its underlying source declared: only byte sources,
a TransformStream's readable (through an internal setter) and native
streams reported a length. A body built from a default source that
declared its length was therefore sent with chunked encoding, where
the C++ implementation sends a Content-Length.
The default controller now caches the converted source's
expectedLength, as the byte controller does, and TransformStream
passes its transformer's through the source it builds for its
readable, which retires the internal setter.
Body consumption already treated a TransformStream readable's
declaration as an exact total (readable ledger #21); that now covers
any default source that declares one. A source that delivers more
than it declared rejects text(), arrayBuffer() and the other
consumers with a RangeError and is cancelled with it, where the C++
implementation returns everything. The divergence is intentional:
the declaration is the exact-total contract that byte and native
sources already enforce.
typescript_implemented_streams is still experimental, with no enable
date, so the change needs no compatibility flag.
Fixes: #8040