diff --git a/docs/guides/agent-first-setup.md b/docs/guides/agent-first-setup.md index 321cf259b..8573af4ef 100644 --- a/docs/guides/agent-first-setup.md +++ b/docs/guides/agent-first-setup.md @@ -89,8 +89,10 @@ so a prefix cannot masquerade as a complete result. The log tools but reaching it is a cutoff: the result is `ok: true` with `outputTruncated: true`, `data.cutoff` set to the `bytes` bound, and the events parsed before the cut. See [MCP logs contract](../reference/mcp-logs.md). Run the equivalent CLI command when -full output is needed. The budget bounds captured bytes, not total process memory -or JSON response size. +full output is needed. The budget bounds captured bytes, not total process memory. +A separate limit keeps each serialized result under 9.5 MiB, below the 10 MiB at +which SDK clients close the connection. See +[Result size limit](../reference/mcp-tools.md#result-size-limit). The [MCP tool contract](../reference/mcp-tools.md) lists every tool with its annotations, inputs, the CLI command it runs and its result shape. diff --git a/docs/reference/mcp-logs.md b/docs/reference/mcp-logs.md index f6cab90d3..144feff34 100644 --- a/docs/reference/mcp-logs.md +++ b/docs/reference/mcp-logs.md @@ -32,6 +32,28 @@ non-zero. Use `ok` and `data.cutoff` to read the result, not `exitCode`. `data.stop_reason` (tail only) keeps its earlier values: `eof`, `max_events`, `timeout`, `output_limit`, `cancelled` or `interrupted`. +The whole result is also limited to 9.5 MiB, because SDK clients drop the +connection above 10 MiB. See [Result size limit](mcp-tools.md#result-size-limit). +Over that limit, `content` becomes a smaller view (below), and `stdout` in +`structuredContent` is cut before any event. Events go next, from the end, always +as whole events. `outputTruncated` is `true` whenever `structuredContent` was +cut. When events were removed from a successful collection: + +- `data.cutoff` is `{ "bound": "bytes", "limit": 9961472 }`, replacing a `null` + or an earlier cutoff. A collection that completed is reported as cut, because + the result no longer holds all of it. +- `data.count` is the number of events kept. +- `data.stop_reason` still says why collection stopped, so it can be `eof` beside + a byte cutoff. + +A backend failure stays a failure: `ok` is `false`, `error` is unchanged and +`data.cutoff` stays `null`, even if events were removed. + +The `content` text is JSON with the same shape as `structuredContent`, in a +256 KiB view. It keeps fewer events, and when it removes events its own +`data.cutoff.limit` is `262144`. A client should read `structuredContent` when it +can. + A backend failure is not a truncated success. It is reported when the stream contains an `error` event or an `end` event with a reason other than `eof` or `closed`, or when the child exits non-zero at the end of the stream. The error diff --git a/docs/reference/mcp-tools.md b/docs/reference/mcp-tools.md index ff4b76830..5959726fd 100644 --- a/docs/reference/mcp-tools.md +++ b/docs/reference/mcp-tools.md @@ -94,7 +94,7 @@ Every result has the same structured content: | `exitCode` | The child's exit code (1 for a refusal) | | `stdout`, `stderr` | Captured child output (withheld for the env tools; `stdout` withheld for instances) | | `data` | Parsed or projected result, when the tool defines one | -| `outputTruncated` | `true` when output reached the 8 MiB capture limit | +| `outputTruncated` | `true` when output reached the 8 MiB capture limit, or this payload was cut to the result size limit | | `spawned` | `false` only on a refusal: no child, no audit line | | `error` | Closed refusal or backend error: `code`, `reason`, `message` | @@ -103,6 +103,29 @@ When `ok` is `false`, the result also has `isError: true`. Output above the never presented as a complete result. The log tools are the exception, as their contract describes. +### Result size limit + +Every tool's serialized result is limited to 9.5 MiB of UTF-8, measured after +JSON escaping. MCP SDK clients on stdio close the connection when a message passes +10 MiB. The rest of that margin covers the JSON-RPC envelope. The 8 MiB capture +limit cannot do this job, because it counts raw output, and a result repeats that +output and escapes it. + +A result within the limit is unchanged. Above it, the server removes data in this +order: + +1. `content` text. It becomes JSON of at most 256 KiB with the same shape as + `structuredContent`, carrying less output and its own `outputTruncated: true`. + If `structuredContent` now fits, it is complete and has no such flag. +2. For the log tools, `stdout`, then whole events from the end. See the + [logs contract](mcp-logs.md). For other tools, `data`, then the end of `stdout`. +3. The end of `stderr`, past its first 16 KiB, which step 2 cannot take. + +A payload that was cut has `outputTruncated: true`, and its `stderr` ends with a +fixed line saying to use the CLI for full output. A cut never splits a code point. +`ok` and `exitCode` still describe the command, so a command that succeeded stays +`ok` with `data` omitted. Run the equivalent CLI command when full output is needed. + ### Refusals A refusal happens before any child process exists. Its `error.reason` is one of diff --git a/src/mcp/logs-contract.ts b/src/mcp/logs-contract.ts index e2145cd34..916e0bbef 100644 --- a/src/mcp/logs-contract.ts +++ b/src/mcp/logs-contract.ts @@ -5,7 +5,9 @@ * A collection ends in exactly one outcome: * - `complete`: the backend reached end of stream and exited 0. * - `cutoff`: the MCP server stopped collection at one of its own bounds. This - * is a success; `data.cutoff` names the bound and its effective limit. + * is a success; `data.cutoff` names the bound and its effective limit. The + * result can also gain or replace a byte cutoff after classification, when + * `boundMcpToolResult` removes events to fit the message limit. * - `backend_error`: the log backend reported failure (a stream `error` event, * a failing `end` event, or a non-zero exit the server did not cause). This * is a closed error, never a truncated success. @@ -32,7 +34,9 @@ export type McpLogStopReason = /** The bound that ended a successful, incomplete collection. */ export type McpLogCutoff = { /** `events` counts `type: "log"` events only; `bytes` is the shared stdout - * plus stderr capture budget; `time` is the collection window. */ + * plus stderr capture budget, or the serialized result limit when + * `boundMcpToolResult` removed events after collection; `time` is the + * collection window. */ readonly bound: "events" | "bytes" | "time"; readonly limit: number; }; diff --git a/src/mcp/result-budget.ts b/src/mcp/result-budget.ts new file mode 100644 index 000000000..e9a409b10 --- /dev/null +++ b/src/mcp/result-budget.ts @@ -0,0 +1,334 @@ +import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; + +const MIB = 1024 * 1024; + +/** + * Most UTF-8 bytes one serialized `tools/call` result may occupy. + * + * MCP SDK clients on stdio close the transport when their read buffer passes + * 10 MiB. The 8 MiB capture budget in `output-budget.ts` cannot prevent that: it + * bounds raw child output, while the message repeats that output in several + * fields and JSON escaping can grow each copy sixfold. The 512 KiB left under + * 10 MiB covers the JSON-RPC envelope and the start of a following message that + * can share the client's last read. + */ +export const MCP_TOOL_RESULT_LIMIT_BYTES = 9.5 * MIB; + +/** Most bytes the `content` text may occupy once a result has been cut. */ +export const MCP_TOOL_RESULT_TEXT_LIMIT_BYTES = 256 * 1024; + +/** Fixed, value-free line appended to `stderr` in any payload that was cut. */ +export const MCP_TOOL_RESULT_CUT_NOTE = + "MCP tool result exceeded its size limit; use the CLI for full output."; + +/** Stderr kept even when the primary output could fill the whole budget. */ +const STDERR_FLOOR_BYTES = 16 * 1024; + +/** Covers the small growth of `count`, `cutoff` and the note's line break. */ +const SKELETON_SLACK_BYTES = 64; + +/** Serialized size of a value in the channel being filled. */ +type Measure = (value: unknown) => number; + +type ToolPayload = Record & { + readonly ok: boolean; + readonly command: string; + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; +}; + +type LogData = Record & { + readonly events: readonly unknown[]; +}; + +const structuredBytes: Measure = (value) => + Buffer.byteLength(JSON.stringify(value), "utf8"); + +/** Size of a value's JSON once it is itself embedded in a JSON string. */ +const embeddedTextBytes: Measure = (value) => + Buffer.byteLength(JSON.stringify(JSON.stringify(value)), "utf8") - 2; + +/** + * Bound one `tools/call` result. A result within the limit is returned as is. + * + * The dispatcher applies this to every tool, so no builder can bypass it. Over + * the limit, data goes in this order, cheapest loss first: + * + * 1. The `content` text. It repeats `structuredContent` with extra escaping (a + * quote costs four bytes there), so it buys the most and loses the least. It + * stays parseable JSON of the same shape, a smaller view with its own + * `outputTruncated` and cutoff, because some clients read only this channel. + * 2. For the log tools, `stdout`, which repeats `data.events`, then whole + * events from the end. For other tools, `data`, which is omitted as it is + * after a capture cut so that a prefix is never presented as complete, then + * the end of `stdout`. + * 3. The end of `stderr`, which only the 16 KiB floor protects from step 2. + * + * Cuts never split a code point. `ok` and `exitCode` still describe the command: + * a command that succeeded stays `ok`, and `outputTruncated` reports the cut. + */ +export function boundMcpToolResult( + result: CallToolResult, + limits: { + readonly limitBytes?: number; + readonly textLimitBytes?: number; + } = {} +): CallToolResult { + const limitBytes = limits.limitBytes ?? MCP_TOOL_RESULT_LIMIT_BYTES; + const textLimitBytes = + limits.textLimitBytes ?? MCP_TOOL_RESULT_TEXT_LIMIT_BYTES; + if (structuredBytes(result) <= limitBytes) { + return result; + } + const payload = toToolPayload(result.structuredContent); + if (!payload) { + return boundTextOnlyResult({ result, limitBytes }); + } + + const content: CallToolResult["content"] = [ + { + type: "text", + text: JSON.stringify( + cutPayload({ + payload, + budget: textLimitBytes, + reportedLimit: textLimitBytes, + measure: embeddedTextBytes, + }) + ), + }, + ]; + const budget = + limitBytes - structuredBytes({ ...result, content, structuredContent: {} }); + const structuredContent = + structuredBytes(payload) <= budget + ? payload + : cutPayload({ + payload, + budget, + reportedLimit: limitBytes, + measure: structuredBytes, + }); + const bounded = { ...result, content, structuredContent }; + return structuredBytes(bounded) <= limitBytes + ? bounded + : buildLastResortResult({ result, payload }); +} + +/** + * Fit a payload in `budget` serialized bytes. `reportedLimit` is the limit a log + * caller sees in `data.cutoff` when events were removed. A failed collection + * keeps its null cutoff, because a failure is never a truncated success. + */ +function cutPayload(opts: { + readonly payload: ToolPayload; + readonly budget: number; + readonly reportedLimit: number; + readonly measure: Measure; +}): ToolPayload { + const { payload, measure } = opts; + const logData = toLogData(payload.data); + const { data: _dropped, ...rest } = payload; + const command = prefixWithin({ + text: payload.command, + maxBytes: Math.floor(opts.budget / 8), + measure, + }); + const assemble = (parts: { + readonly stdout: string; + readonly stderr: string; + readonly events: readonly unknown[]; + }): ToolPayload => ({ + ...rest, + command, + stdout: parts.stdout, + stderr: appendCutNote(parts.stderr), + outputTruncated: true, + ...(logData + ? { + data: { + ...logData, + events: parts.events, + count: parts.events.length, + cutoff: + payload.ok && parts.events.length < logData.events.length + ? { bound: "bytes", limit: opts.reportedLimit } + : logData.cutoff, + }, + } + : {}), + }); + + let left = + opts.budget - + measure(assemble({ stdout: "", stderr: "", events: [] })) - + SKELETON_SLACK_BYTES; + const cost = (text: string): number => measure(text) - measure(""); + const stderrFloor = Math.max( + 0, + Math.min(STDERR_FLOOR_BYTES, left, cost(payload.stderr)) + ); + left -= stderrFloor; + + const keepPrefix = (text: string): string => { + const kept = prefixWithin({ text, maxBytes: left, measure }); + left -= cost(kept); + return kept; + }; + let stdout = ""; + let events: readonly unknown[] = []; + if (logData) { + const kept = eventPrefixWithin({ + events: logData.events, + maxBytes: left, + measure, + }); + events = kept.events; + left -= kept.bytes; + } else { + stdout = keepPrefix(payload.stdout); + } + left += stderrFloor; + const stderr = keepPrefix(payload.stderr); + if (logData) { + stdout = keepPrefix(payload.stdout); + } + return assemble({ stdout, stderr, events }); +} + +function appendCutNote(stderr: string): string { + const separator = stderr === "" || stderr.endsWith("\n") ? "" : "\n"; + return `${stderr}${separator}${MCP_TOOL_RESULT_CUT_NOTE}\n`; +} + +/** The longest run of whole events from the start that fits. */ +function eventPrefixWithin(opts: { + readonly events: readonly unknown[]; + readonly maxBytes: number; + readonly measure: Measure; +}): { readonly events: readonly unknown[]; readonly bytes: number } { + let bytes = 0; + let kept = 0; + for (const event of opts.events) { + const size = opts.measure(event) + 1; + if (bytes + size > opts.maxBytes) { + break; + } + bytes += size; + kept += 1; + } + return { events: opts.events.slice(0, kept), bytes }; +} + +/** + * The longest prefix whose serialized size fits, ending on a code point + * boundary. It measures with the real serializer instead of a copy of JSON's + * escaping rules, which is fast enough for a path only oversized results reach. + */ +function prefixWithin(opts: { + readonly text: string; + readonly maxBytes: number; + readonly measure: Measure; +}): string { + const { text, measure } = opts; + const overhead = measure(""); + const fits = (length: number): boolean => + measure(text.slice(0, length)) - overhead <= opts.maxBytes; + // Every code unit serializes to at least one byte. + let high = Math.min(text.length, Math.max(0, opts.maxBytes)); + if (high === text.length && fits(high)) { + return text; + } + let low = 0; + while (low < high) { + const middle = Math.ceil((low + high) / 2); + if (fits(middle)) { + low = middle; + } else { + high = middle - 1; + } + } + return text.slice(0, splitsSurrogatePair(text, low) ? low - 1 : low); +} + +function splitsSurrogatePair(text: string, index: number): boolean { + if (index <= 0 || index >= text.length) { + return false; + } + const before = text.charCodeAt(index - 1); + const after = text.charCodeAt(index); + return ( + before >= 0xd8_00 && + before <= 0xdb_ff && + after >= 0xdc_00 && + after <= 0xdf_ff + ); +} + +/** Results without a typed payload, such as a dispatcher error message. */ +function boundTextOnlyResult(opts: { + readonly result: CallToolResult; + readonly limitBytes: number; +}): CallToolResult { + const text = opts.result.content + .map((part) => (part.type === "text" ? part.text : "")) + .join("\n"); + const kept = prefixWithin({ + text, + maxBytes: + opts.limitBytes - + structuredBytes({ + content: [{ type: "text", text: `\n${MCP_TOOL_RESULT_CUT_NOTE}` }], + isError: true, + }), + measure: structuredBytes, + }); + return { + content: [{ type: "text", text: `${kept}\n${MCP_TOOL_RESULT_CUT_NOTE}` }], + ...(opts.result.isError ? { isError: true } : {}), + }; +} + +/** Keeps the size guarantee if an unexpected field defeats the normal cut. */ +function buildLastResortResult(opts: { + readonly result: CallToolResult; + readonly payload: ToolPayload; +}): CallToolResult { + const structuredContent = { + ok: opts.payload.ok, + command: "", + exitCode: opts.payload.exitCode, + stdout: "", + stderr: `${MCP_TOOL_RESULT_CUT_NOTE}\n`, + outputTruncated: true, + }; + return { + content: [{ type: "text", text: JSON.stringify(structuredContent) }], + structuredContent, + ...(opts.result.isError ? { isError: true } : {}), + }; +} + +function toToolPayload(value: unknown): ToolPayload | null { + if (typeof value !== "object" || value === null) { + return null; + } + const payload = value as Record; + return typeof payload.ok === "boolean" && + typeof payload.command === "string" && + typeof payload.exitCode === "number" && + typeof payload.stdout === "string" && + typeof payload.stderr === "string" + ? (payload as ToolPayload) + : null; +} + +function toLogData(value: unknown): LogData | null { + if (typeof value !== "object" || value === null) { + return null; + } + return Array.isArray((value as Record).events) + ? (value as LogData) + : null; +} diff --git a/src/mcp/server.ts b/src/mcp/server.ts index a4f40d5d3..21e3a77e1 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -68,6 +68,7 @@ import { createMcpOutputBudget, MCP_OUTPUT_LIMIT_BYTES, } from "./output-budget.ts"; +import { boundMcpToolResult } from "./result-budget.ts"; import { captureMcpSessionContext, type McpSessionContext, @@ -1433,28 +1434,32 @@ function installTypedToolDispatch(opts: { opts.server.server.setRequestHandler( CallToolRequestSchema, async (request, extra): Promise => { - const entry = opts.tools.get(request.params.name); - if (!entry) { - return buildUntypedToolError(MCP_UNKNOWN_TOOL_MESSAGE); - } - try { - const result = await entry.call(request.params.arguments, extra); - if ( - !( - result.isError || - outputSchema.safeParse(result.structuredContent).success - ) - ) { + const dispatch = async (): Promise => { + const entry = opts.tools.get(request.params.name); + if (!entry) { + return buildUntypedToolError(MCP_UNKNOWN_TOOL_MESSAGE); + } + try { + const result = await entry.call(request.params.arguments, extra); + if ( + !( + result.isError || + outputSchema.safeParse(result.structuredContent).success + ) + ) { + return buildUntypedToolError( + `Output validation error: Invalid structured content for tool ${request.params.name}` + ); + } + return result; + } catch (error) { return buildUntypedToolError( - `Output validation error: Invalid structured content for tool ${request.params.name}` + error instanceof Error ? error.message : String(error) ); } - return result; - } catch (error) { - return buildUntypedToolError( - error instanceof Error ? error.message : String(error) - ); - } + }; + // The one exit for every tool result: see boundMcpToolResult. + return boundMcpToolResult(await dispatch()); } ); } diff --git a/tests/e2e/README.md b/tests/e2e/README.md index 456d8a31f..9197bc203 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -372,7 +372,9 @@ The scenario requires these results: payload may contain U+FFFD, so a split code point fails the run. Where the cut falls varies from run to run, so a single pass does not prove that this run cut inside a code point; `tests/mcp-logs-contract.test.ts` covers that - case deterministically. + case deterministically. The scenario's SDK client keeps its default read limit, + so a result over that limit closes the session. `tests/mcp-tool-result-size.test.ts` + covers the result limit without Docker. - With `DOCKER_HOST` pointing at a missing socket, a snapshot and a tail both return the closed `backend_error`. The tail ends at end of stream, not at `maxMs`. diff --git a/tests/mcp-tool-result-size.test.ts b/tests/mcp-tool-result-size.test.ts new file mode 100644 index 000000000..c2b2488d3 --- /dev/null +++ b/tests/mcp-tool-result-size.test.ts @@ -0,0 +1,563 @@ +import { afterEach, expect, test } from "bun:test"; +import { + chmod, + mkdir, + mkdtemp, + readFile, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; +import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; +import { MCP_OUTPUT_LIMIT_BYTES } from "../src/mcp/output-budget.ts"; +import { + boundMcpToolResult, + MCP_TOOL_RESULT_CUT_NOTE, + MCP_TOOL_RESULT_LIMIT_BYTES, + MCP_TOOL_RESULT_TEXT_LIMIT_BYTES, +} from "../src/mcp/result-budget.ts"; + +/** `STDIO_DEFAULT_MAX_BUFFER_SIZE` in MCP SDK 1.32. Older SDKs have no limit, so + * these tests also measure the bytes on the wire. */ +const SDK_CLIENT_READ_LIMIT_BYTES = 10 * 1024 * 1024; +/** Everything JSON-RPC adds around `result`: `{"jsonrpc":"2.0","id":N,"result":}`. */ +const ENVELOPE_ALLOWANCE_BYTES = 64; +const REPLACEMENT_CHAR = "�"; +const CALL_TIMEOUT_MS = 60_000; +const TEST_TIMEOUT_MS = 120_000; + +/** Emoji cost four bytes as UTF-8; the rest expand when JSON escapes them again. */ +const LOG_BODY = `${"😀".repeat(1024)}${'"\\\n\u0001'.repeat(256)}`; +const BULK_LINES = 1500; + +/** + * Stands in for `hack`. The scenario picks the output, and each one is sized so + * the unbounded message passes the client's 10 MiB limit. + */ +const STUB_SOURCE = ` +const body = ${JSON.stringify(LOG_BODY)}; +const write = (text) => Bun.write(Bun.stdout, text); +const logEvent = (n) => JSON.stringify({ type: "log", n, entry: { raw: "BULK " + n + " " + body } }) + "\\n"; +switch (process.env.HACK_TEST_RESULT_SCENARIO) { + case "logs-over-capture": + for (let n = 1; n <= ${BULK_LINES}; n += 1) await write(logEvent(n)); + break; + case "logs-expanding": { + // Canonical JSON cannot grow when reparsed, except for spellings like 1e20, + // which is what forces events, not just their duplicates, over the limit. + const numbers = Array.from({ length: 4096 }, () => "1e20").join(","); + for (let n = 1; n <= 200; n += 1) { + await write('{"type":"log","n":' + n + ',"values":[' + numbers + "]}\\n"); + } + break; + } + case "plain-over-capture": + await write("😀\\u0001".repeat(2 * 1024 * 1024)); + break; + case "plain-escaped": + await write("\\u0001".repeat(5 * 1024 * 1024)); + break; + case "json-structured-fits": + await write(JSON.stringify({ items: "😀".repeat(1024 * 1024) })); + break; + case "small": + await write(JSON.stringify({ items: ["one"] })); + break; + default: + process.exit(2); +} +`; + +type Payload = { + readonly ok: boolean; + readonly command: string; + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; + readonly outputTruncated?: boolean; + readonly data?: { + readonly events?: readonly { + readonly n?: number; + readonly entry?: { readonly raw?: string }; + }[]; + readonly count?: number; + readonly cutoff?: { bound: string; limit: number } | null; + readonly stop_reason?: string; + readonly items?: unknown; + }; +}; + +const cleanups: Array<() => Promise> = []; + +afterEach(async () => { + for (const cleanup of cleanups.splice(0)) { + await cleanup(); + } +}); + +/** + * Drive the real server over stdio with a real SDK client on default limits. A + * `tee` records the exact bytes the client reads, which keeps the size assertion + * meaningful on an SDK version without a read limit. + */ +async function startSession(scenario: string) { + const root = await mkdtemp(join(tmpdir(), "hack-mcp-result-size-")); + const project = join(root, "repo"); + const projectDir = join(project, ".hack"); + const registryDir = join(root, "registry"); + await mkdir(projectDir, { recursive: true }); + await mkdir(registryDir); + await writeFile(join(projectDir, "docker-compose.yml"), "services: {}\n"); + await writeFile( + join(projectDir, "hack.config.json"), + JSON.stringify({ name: "demo" }) + ); + await writeFile( + join(registryDir, "projects.json"), + JSON.stringify({ + version: 1, + projects: [ + { + id: "demo", + createdAt: "2026-01-01T00:00:00Z", + name: "demo", + repoRoot: project, + projectDirName: ".hack", + projectDir, + }, + ], + }) + ); + const stub = join(root, "hack-stub"); + await writeFile(stub, `#!${process.execPath}\n${STUB_SOURCE}`); + await chmod(stub, 0o755); + const wire = join(root, "wire.jsonl"); + + const transport = new StdioClientTransport({ + command: "/bin/sh", + args: ["-c", '"$TEST_BUN" "$TEST_ENTRY" mcp serve | tee "$TEST_WIRE"'], + cwd: project, + env: { + PATH: process.env.PATH ?? "", + HOME: root, + HACK_HOME: join(root, "state"), + HACK_GLOBAL_CONFIG_PATH: join(registryDir, "config.json"), + HACK_DAEMON_AUTOSTART_DISABLE: "1", + HACK_MCP_COMMAND: stub, + HACK_TEST_RESULT_SCENARIO: scenario, + TEST_BUN: process.execPath, + TEST_ENTRY: resolve(import.meta.dir, "..", "index.ts"), + TEST_WIRE: wire, + }, + stderr: "ignore", + }); + let closed = false; + const client = new Client({ name: "hack-mcp-result-size", version: "1" }); + await client.connect(transport); + const onclose = transport.onclose; + transport.onclose = () => { + closed = true; + onclose?.(); + }; + cleanups.push(async () => { + await client.close().catch(() => undefined); + await rm(root, { recursive: true, force: true }); + }); + // Listing tools makes the SDK validate each result against its outputSchema. + await client.listTools(); + + const call = async ( + name: string, + args: Record + ): Promise<{ + readonly result: CallToolResult; + readonly payload: Payload; + readonly text: Payload; + readonly wireBytes: number; + }> => { + const result = (await client.callTool( + { name, arguments: args }, + undefined, + { timeout: CALL_TIMEOUT_MS } + )) as CallToolResult; + const block = result.content[0]; + return { + result, + payload: result.structuredContent as Payload, + text: JSON.parse(block?.type === "text" ? block.text : "null") as Payload, + wireBytes: await largestWireLine(wire), + }; + }; + /** The session must survive the large result. */ + const expectAlive = async (): Promise => { + await client.listTools(); + expect(closed).toBe(false); + }; + return { call, expectAlive }; +} + +/** Bytes of the largest message sent. Each line must also be whole JSON. */ +async function largestWireLine(path: string): Promise { + const bytes = await readFile(path); + let largest = 0; + let start = 0; + while (start < bytes.length) { + const end = bytes.indexOf(0x0a, start); + expect(end).not.toBe(-1); + largest = Math.max(largest, end - start); + JSON.parse(bytes.toString("utf8", start, end)); + start = end + 1; + } + return largest; +} + +function expectWithinLimits(wireBytes: number): void { + expect(wireBytes).toBeLessThanOrEqual( + MCP_TOOL_RESULT_LIMIT_BYTES + ENVELOPE_ALLOWANCE_BYTES + ); + expect(wireBytes).toBeLessThan(SDK_CLIENT_READ_LIMIT_BYTES); +} + +function expectNoSplitCodePoint(payload: Payload): void { + expect(JSON.stringify(payload)).not.toContain(REPLACEMENT_CHAR); + expect(JSON.stringify(payload)).not.toMatch(/\\ud[89ab]/i); +} + +/** A complete, ordered prefix of whole events, starting at the first. */ +function expectEventPrefix(payload: Payload): number { + const events = payload.data?.events ?? []; + expect(payload.data?.count).toBe(events.length); + for (const [index, event] of events.entries()) { + expect(event.n).toBe(index + 1); + } + return events.length; +} + +function expectWholeBulkEvents(payload: Payload): void { + for (const event of payload.data?.events ?? []) { + expect(event.entry?.raw).toBe(`BULK ${event.n} ${LOG_BODY}`); + } +} + +/** The shortened `content` view: valid JSON that admits it was cut. */ +function expectTruthfulTextView(opts: { + readonly text: Payload; + readonly payload: Payload; +}): void { + const { text, payload } = opts; + expect(text.ok).toBe(payload.ok); + expect(text.exitCode).toBe(payload.exitCode); + expect(text.outputTruncated).toBe(true); + expect(text.stderr).toContain(MCP_TOOL_RESULT_CUT_NOTE); + expectNoSplitCodePoint(text); +} + +for (const tool of [ + "hack.project.logs.snapshot", + "hack.project.logs.tail", +] as const) { + test( + `${tool} over the capture limit keeps whole events within the client limit`, + async () => { + const session = await startSession("logs-over-capture"); + const { result, payload, text, wireBytes } = await session.call(tool, { + projectName: "demo", + tail: 10_000, + ...(tool === "hack.project.logs.tail" + ? { maxEvents: 10_000, maxMs: 60_000 } + : {}), + }); + expectWithinLimits(wireBytes); + await session.expectAlive(); + + expect(result.isError).toBeUndefined(); + expect(payload.ok).toBe(true); + expect(payload.outputTruncated).toBe(true); + // The capture limit cut these events, and it is the limit that is reported. + expect(payload.data?.cutoff).toEqual({ + bound: "bytes", + limit: MCP_OUTPUT_LIMIT_BYTES, + }); + const kept = expectEventPrefix(payload); + expect(kept).toBeGreaterThan(0); + expect(kept).toBeLessThan(BULK_LINES); + expectWholeBulkEvents(payload); + expectNoSplitCodePoint(payload); + if (tool === "hack.project.logs.tail") { + expect(payload.data?.stop_reason).toBe("output_limit"); + } + + expectTruthfulTextView({ text, payload }); + const preview = expectEventPrefix(text); + expect(preview).toBeGreaterThan(0); + expect(preview).toBeLessThan(kept); + expectWholeBulkEvents(text); + expect(text.data?.cutoff).toEqual({ + bound: "bytes", + limit: MCP_TOOL_RESULT_TEXT_LIMIT_BYTES, + }); + }, + TEST_TIMEOUT_MS + ); +} + +test( + "a complete log stream over the message limit reports the result byte limit", + async () => { + const session = await startSession("logs-expanding"); + const { result, payload, text, wireBytes } = await session.call( + "hack.project.logs.tail", + { projectName: "demo", maxEvents: 10_000, maxMs: 60_000 } + ); + expectWithinLimits(wireBytes); + await session.expectAlive(); + + expect(result.isError).toBeUndefined(); + expect(payload.ok).toBe(true); + expect(payload.exitCode).toBe(0); + expect(payload.outputTruncated).toBe(true); + expect(payload.stderr).toContain(MCP_TOOL_RESULT_CUT_NOTE); + // Collection reached end of stream; the result is what was cut. + expect(payload.data?.stop_reason).toBe("eof"); + expect(payload.data?.cutoff).toEqual({ + bound: "bytes", + limit: MCP_TOOL_RESULT_LIMIT_BYTES, + }); + const kept = expectEventPrefix(payload); + expect(kept).toBeGreaterThan(0); + expect(kept).toBeLessThan(200); + expectTruthfulTextView({ text, payload }); + }, + TEST_TIMEOUT_MS +); + +test( + "a non-log tool over the capture limit stays an error within the client limit", + async () => { + const session = await startSession("plain-over-capture"); + const { result, payload, text, wireBytes } = await session.call( + "hack.projects.list", + {} + ); + expectWithinLimits(wireBytes); + await session.expectAlive(); + + expect(result.isError).toBe(true); + expect(payload.ok).toBe(false); + expect(payload.exitCode).toBe(1); + expect(payload.outputTruncated).toBe(true); + expect(payload.data).toBeUndefined(); + expect(payload.stdout.length).toBeGreaterThan(0); + expect(payload.stderr).toContain(MCP_TOOL_RESULT_CUT_NOTE); + expectNoSplitCodePoint(payload); + expectTruthfulTextView({ text, payload }); + }, + TEST_TIMEOUT_MS +); + +test( + "a successful non-log command under the capture limit keeps its status when cut", + async () => { + const session = await startSession("plain-escaped"); + const { result, payload, text, wireBytes } = await session.call( + "hack.projects.list", + {} + ); + expectWithinLimits(wireBytes); + await session.expectAlive(); + + // 5 MiB of control characters is under the capture limit, but 30 MiB once + // JSON-escaped. Nothing went wrong with the command. + expect(result.isError).toBeUndefined(); + expect(payload.ok).toBe(true); + expect(payload.exitCode).toBe(0); + expect(payload.outputTruncated).toBe(true); + expect(payload.stdout.length).toBeGreaterThan(1024 * 1024); + expect(payload.stdout).toBe("\u0001".repeat(payload.stdout.length)); + expect(payload.stderr).toBe(`${MCP_TOOL_RESULT_CUT_NOTE}\n`); + expectTruthfulTextView({ text, payload }); + }, + TEST_TIMEOUT_MS +); + +test( + "shedding only the text keeps structured content complete and unflagged", + async () => { + const session = await startSession("json-structured-fits"); + const { result, payload, text, wireBytes } = await session.call( + "hack.projects.list", + {} + ); + expectWithinLimits(wireBytes); + await session.expectAlive(); + + expect(result.isError).toBeUndefined(); + expect(payload.outputTruncated).toBeUndefined(); + expect(payload.stderr).toBe(""); + expect(payload.data?.items).toBe("😀".repeat(1024 * 1024)); + expect(JSON.parse(payload.stdout)).toEqual(payload.data); + expectTruthfulTextView({ text, payload }); + expect(text.data).toBeUndefined(); + }, + TEST_TIMEOUT_MS +); + +test( + "a result within the limit is unchanged", + async () => { + const session = await startSession("small"); + const { result, payload } = await session.call("hack.projects.list", {}); + await session.expectAlive(); + expect(payload.outputTruncated).toBeUndefined(); + expect(payload.data).toEqual({ items: ["one"] }); + expect(result.content).toEqual([ + { type: "text", text: JSON.stringify(payload, null, 2) }, + ]); + }, + TEST_TIMEOUT_MS +); + +function toolResult(payload: Record): CallToolResult { + return { + content: [{ type: "text", text: JSON.stringify(payload, null, 2) }], + structuredContent: payload, + }; +} + +const basePayload = { ok: true, command: "hack", exitCode: 0, stderr: "" }; + +function serializedBytes(value: unknown): number { + return Buffer.byteLength(JSON.stringify(value), "utf8"); +} + +test("a cut never splits a surrogate pair, at any budget", () => { + const stdout = "😀".repeat(4096); + for (let limitBytes = 3000; limitBytes < 3040; limitBytes += 1) { + const bounded = boundMcpToolResult(toolResult({ ...basePayload, stdout }), { + limitBytes, + textLimitBytes: 1024, + }); + const payload = bounded.structuredContent as Payload; + expect(serializedBytes(bounded)).toBeLessThanOrEqual(limitBytes); + expect(payload.stdout.length).toBeGreaterThan(0); + expect(payload.stdout).toBe("😀".repeat(payload.stdout.length / 2)); + } +}); + +test("a first event too large to fit leaves an empty, flagged event list", () => { + const bounded = boundMcpToolResult( + toolResult({ + ...basePayload, + stdout: "", + data: { + events: [{ n: 1, raw: "x".repeat(64 * 1024) }], + count: 1, + cutoff: null, + }, + }), + { limitBytes: 16 * 1024, textLimitBytes: 4096 } + ); + const payload = bounded.structuredContent as Payload; + expect(serializedBytes(bounded)).toBeLessThanOrEqual(16 * 1024); + expect(payload.ok).toBe(true); + expect(payload.outputTruncated).toBe(true); + expect(payload.data?.events).toEqual([]); + expect(payload.data?.count).toBe(0); + expect(payload.data?.cutoff).toEqual({ bound: "bytes", limit: 16 * 1024 }); +}); + +test("a backend failure stays a failure with a null cutoff when events are cut", () => { + const error = { code: "E_UNEXPECTED", reason: "backend_error", message: "x" }; + const bounded = boundMcpToolResult( + { + ...toolResult({ + ...basePayload, + ok: false, + exitCode: 1, + stdout: "", + error, + data: { + events: Array.from({ length: 64 }, (_, index) => ({ + n: index + 1, + raw: "x".repeat(1024), + })), + count: 64, + cutoff: null, + }, + }), + isError: true, + }, + { limitBytes: 16 * 1024, textLimitBytes: 4096 } + ); + const payload = bounded.structuredContent as Payload & { error: unknown }; + expect(bounded.isError).toBe(true); + expect(payload.ok).toBe(false); + expect(payload.error).toEqual(error); + expect(payload.outputTruncated).toBe(true); + expect(payload.data?.cutoff).toBeNull(); + expect(expectEventPrefix(payload)).toBeLessThan(64); +}); + +test("stderr keeps a floor beside oversized stdout and the rest when stdout is small", () => { + const limits = { limitBytes: 256 * 1024, textLimitBytes: 4096 }; + const crowded = boundMcpToolResult( + toolResult({ + ...basePayload, + stdout: "o".repeat(1024 * 1024), + stderr: "e".repeat(1024 * 1024), + }), + limits + ).structuredContent as Payload; + expect(crowded.stderr.length).toBeGreaterThanOrEqual(16 * 1024); + expect(crowded.stdout.length).toBeGreaterThan(200 * 1024); + + const stderrOnly = boundMcpToolResult( + toolResult({ ...basePayload, stdout: "", stderr: "e".repeat(1024 * 1024) }), + limits + ).structuredContent as Payload; + expect(stderrOnly.stderr.length).toBeGreaterThan(200 * 1024); +}); + +test("a result without a typed payload is cut to bounded text", () => { + const bounded = boundMcpToolResult( + { + content: [{ type: "text", text: `${"😀".repeat(4096)}\u0001`.repeat(8) }], + isError: true, + }, + { limitBytes: 16 * 1024 } + ); + expect(serializedBytes(bounded)).toBeLessThanOrEqual(16 * 1024); + expect(bounded.isError).toBe(true); + expect(JSON.stringify(bounded)).not.toMatch(/\\ud[89ab]/i); + expect(JSON.stringify(bounded)).toContain(MCP_TOOL_RESULT_CUT_NOTE); +}); + +test("an unexpected oversized field still cannot pass the limit", () => { + const bounded = boundMcpToolResult( + { + ...toolResult({ + ...basePayload, + ok: false, + exitCode: 3, + stdout: "", + data: { + events: [], + count: 0, + cutoff: null, + unexpected: "x".repeat(64 * 1024), + }, + }), + isError: true, + }, + { limitBytes: 16 * 1024, textLimitBytes: 4096 } + ); + const payload = bounded.structuredContent as Payload; + expect(serializedBytes(bounded)).toBeLessThanOrEqual(16 * 1024); + expect(bounded.isError).toBe(true); + expect(payload.ok).toBe(false); + expect(payload.exitCode).toBe(3); + expect(payload.outputTruncated).toBe(true); +});