Skip to content

Route browser fs and logs endpoints directly to the VM - #178

Open
tnsardesai wants to merge 5 commits into
mainfrom
hypeship/direct-vm-fs-logs
Open

Route browser fs and logs endpoints directly to the VM#178
tnsardesai wants to merge 5 commits into
mainfrom
hypeship/direct-vm-fs-logs

Conversation

@tnsardesai

@tnsardesai tnsardesai commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds fs and logs/stream to the default KERNEL_BROWSER_ROUTING_SUBRESOURCES prefixes, so every /browsers/{id}/fs/* operation (JSON, binary read/write, multipart upload, watch SSE) and /browsers/{id}/logs/stream is rewritten by the routing fetch to the browser VM's cached base_url with ?jwt= and no API-key Authorization.

Browser lifecycle/metadata, extensions, replays, telemetry/events, and anything else under logs/ stay on the control plane. KERNEL_BROWSER_ROUTING_SUBRESOURCES still overrides the list, and an empty value still disables routing.

Fixes one latent bug the new routing surfaced: createRoutingFetch builds a probe Request from the incoming body for cache sniffing, and that construction derives a multipart/form-data content-type carrying its own boundary. Copying that header onto the routed request while re-encoding the FormData body produced a boundary mismatch, and the VM rejected direct fs.upload calls with 400 failed to read form part. The routed init now drops an inherited content-type when the body derives its own (FormData, URLSearchParams, Blob) and the caller did not set one explicitly. fs.upload is the first routed multipart endpoint, so nothing previously routed was affected.

Query preservation, header stripping, signal propagation, stale-JWT eviction and the fallback body construction are otherwise unchanged; the added tests pin the fallback's existing body replay behavior for the body shapes the SDK produces.

Tests

tests/lib/browser-routing.test.ts:

  • default prefix list and segment-boundary matching include fs and logs/stream, and still exclude telemetry/events, fsx/..., logs, logs/history, logstream, extensions, replays
  • a routing-fetch level test proves logs/stream routes to the VM while logs, logs/history and logstream stay on the API origin
  • fs JSON (listFiles), binary readFile, binary writeFile, multipart upload (indexed files[0][…] field names), fs/watch/{id}/events SSE and logs/stream SSE route to the VM with the query preserved, ?jwt= appended and no authorization header
  • a routed multipart request parses with request.formData(), which fails on the boundary mismatch described above (verified: the test fails without the fix)
  • logs.stream aborts stay wired to the routed request's signal
  • stale-JWT fallback replays the writeFile binary body byte for byte and the upload multipart body on the control plane with bearer auth, and evicts the route
  • env override keeps fs/logs on the control plane; empty env disables routing
  • control-plane subresources (telemetry/events, replays, extensions) keep the API origin and bearer auth

Ran locally: jest (421 passed, 227 skipped), prettier --check ., eslint ., tsc, ./scripts/build.

Live validation

Ran against staging with real headless browsers: fs.writeFile, fs.readFile, fs.listFiles, fs.upload (two entries), fs.watch.start/events/stop and logs.stream all hit https://<browser-host>/browser/kernel/...?jwt=... and returned the expected data, with uploaded files reading back with the correct per-entry contents. telemetry/events and the browser delete stayed on the control plane. The fs.upload failure above was found this way and re-verified as fixed; re-validated after switching the default to logs/stream.


Note

Medium Risk
Changes default request paths for fs and streaming logs (data and uploads) and touches request header/body handling for routed multipart; mitigated by control-plane fallback, env allowlist, and broad test coverage.

Overview
Default direct-to-VM routing now includes fs and logs/stream, so browser file operations (list/read/write/upload, watch SSE) and live log SSE go to the session VM with JWT query auth instead of the API control plane. Other paths under logs/, plus telemetry/events, extensions, and replays, stay on the API origin.

Multipart fix: When rebuilding the routed fetch body, inherited content-type from an intermediate Request could disagree with a re-encoded FormData boundary and break fs.upload. Routed requests now drop content-type when the body is FormData/URLSearchParams/Blob and the caller did not set it explicitly, so the runtime derives a matching boundary.

Tests cover prefix matching, VM routing for binary and multipart bodies, SSE abort wiring, stale-JWT fallback with body replay, and KERNEL_BROWSER_ROUTING_SUBRESOURCES overrides.

Reviewed by Cursor Bugbot for commit dd066e9. Bugbot is set up for automated code reviews on this repo. Configure here.

Add fs and logs to the default direct-to-VM subresource prefixes so
filesystem operations and log streaming use the cached browser base_url
and JWT instead of the control plane. Extensions, replays and
telemetry/events stay on the control plane.
The probe Request built for cache sniffing derives a multipart content-type
that carries its own boundary. Copying that header onto the routed request
while re-encoding the body produced a boundary mismatch, so the browser VM
rejected direct fs.upload calls with "failed to read form part". Let the
routed fetch derive the header when the body derives its own content-type.
Narrows the default prefix so a future logs read served by the control
plane is not swept onto the browser VM.
@tnsardesai
tnsardesai marked this pull request as ready for review September 2, 2026 20:57
@tnsardesai
tnsardesai requested review from Sayan- and rgarcia September 2, 2026 21:19

@rgarcia rgarcia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed alongside the go (#174) and python (#164) siblings, including a control-plane vs kernel-images surface comparison. all 16 fs/* and logs/stream endpoints match on params, body fields, 2xx codes and content types, and metro-api's /browser/kernel/* handler already wakes standby VMs and records session activity for direct requests, so the routing change itself looks safe.

the multipart content-type fix is correct and minimal: constructing the Request serialized the FormData once with boundary A, and the routed fetch re-serialized with boundary B while still sending header A. no previously routed endpoint had a multipart body, so this only surfaces with fs/upload / fs/upload_zip. the request.formData() test is the one that would have caught it. files[0][dest_path] naming is what the VM parser expects and matches what the control plane already forwarded.

Nits

  • src/lib/browser-routing.ts:314-319 — the headers.delete relies on routedInit.headers sharing the same Headers instance; consider deleting before building routedInit or a one-line note
  • tests/lib/browser-routing.test.ts — fallback mocks return 204 for fs/upload / fs/write_file; production returns 201

@Sayan- Sayan- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Verified against a mock control plane and VM over real HTTP: fs/* and logs/stream route to the VM with ?jwt= and no Authorization, telemetry/events and replays stay on the control plane with bearer auth, query strings and binary bodies survive intact, the env override pushes everything back, and a stale-JWT fallback replays a buffered body byte for byte and evicts the route.

  • p2: the stale-JWT replay tests inject fetch, which cannot distinguish a safe replay from an unsafe one. Constructing a Request from a drained stream inside an injected fetch yields an empty body and no error, so a body shape that fails at runtime still satisfies those assertions. Real undici throws instead. The routed-multipart boundary test does exercise the real encoder, so this applies only to the fallback-replay assertions.

  • p2: a non-replayable body on the stale-JWT path surfaces APIConnectionError: Connection error. against a healthy server. Python surfaces the original 401 and Go the failed request. Measured with fs.createReadStream and an async generator. The route is evicted either way, so the next call recovers.

  • p2, pre-existing: the writeFile JSDoc example passes fs.createReadStream('path/to/file'), which is not assignable to the declared contents type and is the shape that hits the above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants