Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/deepseek-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,9 +235,10 @@ worker's first answer reads as an answer. Three things about the file shaped the
reader (`src/deepseek-transcript.ts`):

- **It is one zstd FRAME per append, not one zstd stream.** `zstd -dc` decodes all
of them, Node's `zlib` zstd decoder stops at the first: a real 56-line
transcript came back as 1 line. The reader walks frame headers itself. On a Node
older than 22.15 (no zstd at all) the mode falls back to the pane, as before.
of them, Node's `zlib` zstd decoder before 24.22 and 26.10 stops at the first:
a real 56-line transcript came back as 1 line. The reader walks frame headers
itself, which stays correct on a Node that decodes every frame
(nodejs/node#65865). On a Node older than 22.15 (no zstd at all) the mode falls back to the pane, as before.
- **Not every `user/message` is the user.** Each turn also records a
plugin-sourced runtime-context snapshot; only `source.kind === 'user'` is a
prompt.
Expand Down
9 changes: 5 additions & 4 deletions src/deepseek-transcript.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,11 @@
* **1. One zstd FRAME per append, not one zstd stream.** The file is
* `session.jsonl.zstd`, and dsh appends by compressing each batch of lines into
* its own frame and writing it at the end. `zstd -dc` handles that (frames
* concatenate by definition), but Node's `zlib.zstdDecompress()` and
* `createZstdDecompress()` both stop at the first frame end: measured on a real
* 56-line transcript, Node returned 158 bytes / 1 line where the CLI returned
* 43,747 bytes / 56 lines. That is a silent truncation to the session header —
* concatenate by definition), but before Node 24.22 and 26.10
* (nodejs/node#65865) `zlib.zstdDecompress()` and `createZstdDecompress()` both
* stop at the first frame end, and the supported range still includes those
* versions: measured on a real 56-line transcript, Node returned 158 bytes /
* 1 line where the CLI returned 43,747 bytes / 56 lines. That is a silent truncation to the session header —
* every call would have reported "no answer yet" forever. `decodeZstdFrames()`
* below walks the frame headers itself and decompresses each frame, and
* `test/deepseek-transcript.test.ts` pins it against multi-frame fixtures.
Expand Down
13 changes: 5 additions & 8 deletions test/deepseek-transcript.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@
* Two of these assertions exist because the obvious implementation was measured
* to be wrong against real files:
*
* - dsh appends ONE ZSTD FRAME PER WRITE, and Node's `zlib` zstd decoder stops
* at the first frame end. A 56-line transcript decoded as 1 line / 158 bytes,
* which reads as "the worker never answered" rather than as an error. The
* multi-frame fixtures below are the guard.
* - dsh appends ONE ZSTD FRAME PER WRITE, and Node's `zlib` zstd decoder
* before 24.22 and 26.10 stops at the first frame end. A 56-line transcript
* decoded as 1 line / 158 bytes, which reads as "the worker never answered"
* rather than as an error. Newer Node decodes every frame (nodejs/node#65865),
* so the multi-frame fixtures below pin the walker itself, not Node's decoder.
* - a fresh worker in a case directory that had been used before answered its
* first `last-response` with the PREVIOUS session's reply. Session-to-
* transcript pairing is therefore its own describe block.
Expand Down Expand Up @@ -67,10 +68,6 @@ describe.skipIf(!zstdSupported())('zstd frame walking', () => {
const lines = Array.from({ length: 40 }, (_, i) => JSON.stringify({ type: 'noise', seq: i }));
const buf = framed(lines);

// The one-shot decoder is what this module had to replace.
const oneShot = (zlib as unknown as { zstdDecompressSync?: (b: Buffer) => Buffer }).zstdDecompressSync!(buf);
expect(oneShot.toString('utf8').trim().split('\n')).toHaveLength(1);

expect(zstdFrameRanges(buf)).toHaveLength(40);
expect(decodeZstdFrames(buf).trim().split('\n')).toHaveLength(40);
});
Expand Down
Loading