diff --git a/CHANGELOG.md b/CHANGELOG.md index e857462234..42d1697f49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,26 @@ file is the human-readable summary of what changed and why. ### Sandbox +- **CoDev tells you when a new sandbox version is out, and keeps your work when + you take it** — a sandbox ran the version it was created with, for good: there + was no sign a newer one existed, and the only way to get it was **Reset**, + which throws away everything inside the sandbox on the way. Work now shows a + line when a new version is available, Settings → Servers marks the sandbox and + offers **Update sandbox**, and updating carries your sessions and the files in + the sandbox's home folder across to the new version. Software you installed + inside the sandbox yourself is replaced by what the new version ships, and the + folders you mounted from this computer are never touched. **Reset** is + unchanged and still means a clean sandbox. CoDev checks shortly after it + starts and every few hours after that; it never downloads a version on its own, + because that is over a gigabyte and the sandbox restarts to take it. On a Mac, + keeping your work needs a sandbox version new enough to know how — the update + says so plainly when it is not, rather than finding out afterwards. Updating + also clears the version it replaced off the disk, and deleting a sandbox now + clears the version it was running — a sandbox version is over a gigabyte, and + until now nothing ever removed one, so every version a computer had ever + downloaded stayed on it for good. A version another sandbox is still on is + kept. + - **CoDev installs WSL for you when a sandbox needs it** — a sandbox on Windows runs inside WSL 2, and a computer without it got a dead end: the create-sandbox dialog said WSL was not installed, offered nothing but **Cancel**, and told you diff --git a/DIVERGENCES.md b/DIVERGENCES.md index 8ad19730a3..3fe49786db 100644 --- a/DIVERGENCES.md +++ b/DIVERGENCES.md @@ -276,6 +276,7 @@ from the squash message first, or use a plain merge. | `turbo.json` | Added task `"@codev/sandbox#test"` (no deps, no outputs) | CI's `bun turbo test` only runs test tasks registered here; the fork-only sandbox package's tests must gate PRs. | Keep our task entry; take upstream's other tasks. | | Sandbox project label fallback: `packages/app/src/context/global.tsx` | `enrich()` (the per-server project-list decorator) gains one fork line: when the merged project record has no `name` yet, it fills it from the fork-only `sandboxProjectHostName(conn, worktree, primary)` (`packages/app/src/sandbox/projects.ts`), which maps a sandbox connection's guest paths back to host folder basenames — `conn.projectDir` for the primary (matched by the synced primary directory, or the `/workspace` literal before sync loads) and `conn.mounts[].path` for extras. Non-sandbox connections and named projects are untouched (the helper returns `undefined` / the branch is skipped). | A fresh sandbox project renders `displayName()` = name-or-worktree-basename, and inside the guest the primary worktree is `/workspace`, so every new sandbox flashed the label "workspace" until `adoptFolderName`'s server-side rename round-tripped (guest reports the project → `project.update` → `project.updated` event). The fallback is display-only; the durable rename through the server is unchanged and takes precedence the moment it lands. | Keep the one `if (!base.name)` line after upstream's `base` construction and take upstream's surrounding `enrich` changes. If upstream renames or restructures `enrich`, re-apply the fallback wherever project records are decorated for the sidebar list. The helper stays in the fork-only sandbox module — do not inline it. | | Sandbox/WSL delete cleanup + confirmation: `packages/app/src/context/server.tsx`, `packages/app/src/components/dialog-select-server.tsx`, `packages/app/src/i18n/{en,es}.ts` | `server.tsx` gains `forget(key)` beside `remove(key)`: it deletes the persisted per-server `projects`/`lastProject`/`recentlyClosed` map entries for one key. `dialog-select-server.tsx` `handleRemove` (already diverged, see the renderer-slice row) additionally: for `sandbox:` and `wsl:` keys, awaits a confirmation through the fork-only `packages/app/src/sandbox/dialog-delete.tsx` / `packages/app/src/wsl/dialog-delete.tsx` (lazy-imported, pushed with `dialog.push` so it stacks on whatever dialog opened the menu) and returns without touching anything when declined; and calls `server.forget(key)` right after `server.remove(key)` for both key kinds. The controller gains a `useDialog()` binding for this. i18n gains the `sandbox.delete.*` and `wsl.delete.*` families. The two dialogs state different consequences on purpose: deleting a sandbox erases its disk (sessions, terminals, guest files — permanently; mounted host folders survive), while removing a WSL server destroys nothing — the app never unregisters distributions — it only forgets the app-side project state. | `remove()` only edits the stored HTTP server list; the project maps are keyed by server key and outlive the server, which is right for an HTTP server that still exists remotely but wrong for a deleted sandbox or removed distro — a recreated sandbox with the same name inherited the old projects (`server.projects["sandbox:"]`) and, via the leftover `recentlyClosed`, silently refused to seed folders closed in the previous sandbox's life; a re-added distro likewise inherited its previous project list. And deleting a sandbox erases guest data permanently, so it must not be a one-click menu action; the confirmations live in `handleRemove` because the settings rows and the generic server list all funnel through it. | Keep the confirm-and-forget block at the top of `handleRemove` and the `forget` calls between `server.remove(key)` and the awaited platform removals; take upstream's surrounding changes. `forget` is fork-only API — if upstream reworks the persisted `server` record shape, re-point the three deletes at the new maps. Do not gate `forget` behind the platform removal succeeding: the tab/workspace cleanup ahead of it already treats the server as gone. HTTP removals deliberately keep their maps — the server still exists remotely and re-adding it should restore its projects. If the app ever gains real distro unregistration, the WSL dialog's "nothing is deleted" wording must change with it. | +| Sandbox image updates: `packages/app/src/components/app-sidebar.tsx`, `packages/app/src/i18n/{en,es}.ts`, `packages/desktop/src/preload/index.ts` | `app-sidebar.tsx` renders the fork-only `` directly under ``: one import, one element, and the component itself returns nothing unless the showing side is Work and some sandbox has a newer published image. `preload/index.ts` exposes `sandbox-update` (rebuild a sandbox on the current image, keeping the guest's home) and `sandbox-check-updates` (re-ask now, beside the main process's own slow sweep). i18n gains the `sandbox.update.*` family plus `sandbox.server.update` and `sandbox.job.update`. Everything else is fork-only: the verdict comes from `SandboxProvider.imageStatus` and the controller's `checkImageUpdates`, and the offer is rendered by `src/sandbox/{update-notice,dialog-update}.tsx` and the sandbox settings rows. | Reset already re-provisioned from the current manifest, so the upgrade existed but nothing ever said it was available — guests stayed on the image they were created with until someone happened to reset, which also wiped everything inside them. Telling the user where they actually use sandboxes is the point of the sidebar notice; Settings → Servers is where a sandbox is managed, not where anyone looks. | Keep the one element and its import, and take upstream's other sidebar changes. It must stay above the projects header — it is a notice about the side, not about a project. Do not merge it into the empty-state branches: it has to show on a Work side that is working perfectly well. If upstream restructures the preload surface, keep both channels; the renderer must never mint a sandbox id, so `update` takes the id it was given in the state and nothing else. | ## Known rebrand gaps diff --git a/packages/app/src/components/app-sidebar.tsx b/packages/app/src/components/app-sidebar.tsx index d4a7427e88..a5a7071b80 100644 --- a/packages/app/src/components/app-sidebar.tsx +++ b/packages/app/src/components/app-sidebar.tsx @@ -37,6 +37,7 @@ import { HomeUtilityNav, } from "@/pages/home" import { SandboxMountPrompt } from "@/sandbox/mount-prompt" +import { SandboxUpdateNotice } from "@/sandbox/update-notice" import { useSandboxMountDialog } from "@/sandbox/mount-dialog" import { isPlaceholderSandbox } from "@/sandbox/projects" import { SidebarSection } from "@/components/sidebar-section" @@ -98,6 +99,10 @@ export function AppSidebar(props: { class?: string; debugTools?: { visible: bool {/* Fork: the Code/Work split. Everything below is unchanged — it just sees the servers of the selected side. */} + {/* Fork: a newer guest image, offered where sandboxes are actually + used. Renders nothing on Code, and nothing on Work until there is + one. */} +
{language.t("home.projects")}
void +}) { + const language = useLanguage() + const dialog = useDialog() + + return ( + + + {language.t("sandbox.update.title")} + + + +

+ {props.version + ? language.t("sandbox.update.confirmVersion", { name: props.name, version: props.version }) + : language.t("sandbox.update.confirm", { name: props.name })} +

+
    + {language.t("sandbox.update.homeLost")}}> +
  • {language.t("sandbox.update.homeKept")}
    • +     +
  • {language.t("sandbox.update.software")}
    • +
  • {language.t("sandbox.update.mounts")}
    • +
  • {language.t("sandbox.update.download")}
    • +
+ + + dialog.close()}> + {language.t("common.cancel")} + + {/* Danger only when the guest's contents cannot come along: that is the + case where agreeing throws something away. */} + { + props.onConfirm() + dialog.close() + }} + > + {language.t("sandbox.update.button")} + + + + ) +} diff --git a/packages/app/src/sandbox/image-update.ts b/packages/app/src/sandbox/image-update.ts new file mode 100644 index 0000000000..08e6c35c78 --- /dev/null +++ b/packages/app/src/sandbox/image-update.ts @@ -0,0 +1,11 @@ +import type { SandboxImageStatus } from "./types" + +// The only image verdict worth acting on. Everything else — up to date, a +// custom image the manifest has no say over, or a sandbox nothing has answered +// for yet — offers nothing to press. +// +// Its own module so the sidebar's update notice does not have to import the +// settings rows to ask the question. +export function sandboxImageUpdate(image: SandboxImageStatus | undefined) { + return image?.kind === "update" ? image : null +} diff --git a/packages/app/src/sandbox/settings.tsx b/packages/app/src/sandbox/settings.tsx index 7a95ae32eb..4054303da8 100644 --- a/packages/app/src/sandbox/settings.tsx +++ b/packages/app/src/sandbox/settings.tsx @@ -5,7 +5,7 @@ import { IconButtonV2 } from "@opencode-ai/ui/v2/icon-button-v2" import { MenuV2 } from "@opencode-ai/ui/v2/menu-v2" import { useMutation } from "@tanstack/solid-query" import fuzzysort from "fuzzysort" -import { type Accessor, For, Show, createMemo } from "solid-js" +import { type Accessor, For, Show, createMemo, onMount } from "solid-js" import type { useServerManagementController } from "@/components/dialog-select-server" import { ServerHealthIndicator } from "@/components/server/server-row" import { useLanguage } from "@/context/language" @@ -13,7 +13,9 @@ import { usePlatform } from "@/context/platform" import { ServerConnection } from "@/context/server" import { showToast } from "@/utils/toast" import { useSandboxes } from "./context" +import { sandboxImageUpdate } from "./image-update" import { DialogSandboxMounts } from "./dialog-mounts" +import { DialogSandboxUpdate } from "./dialog-update" import type { SandboxRuntime } from "./types" type Controller = ReturnType @@ -54,6 +56,10 @@ export function SandboxServerSettings(props: { const sandboxes = useSandboxes() const api = platform.sandboxes + // The main process sweeps on a slow heartbeat; this is the screen where the + // answer is on display, so it asks once on the way in. + onMount(() => void api?.checkUpdates()) + const request = useMutation(() => ({ mutationFn: (action: () => Promise) => action(), onError: (error) => @@ -73,6 +79,7 @@ export function SandboxServerSettings(props: { // sandbox's; either way its lifecycle actions must wait. const job = () => sandboxes.state.data?.jobs.find((entry) => entry.id === item.config.id) const busy = () => !!job() + const update = () => sandboxImageUpdate(item.image) const meta = () => { const current = job() if (current) @@ -82,6 +89,18 @@ export function SandboxServerSettings(props: { if (item.runtime.kind === "failed") return item.runtime.message return `${item.config.backend} · ${language.t(`sandbox.status.${item.runtime.kind}`)}` } + const confirmUpdate = () => { + const available = update() + if (!available || !api) return + dialog.push(() => ( + request.mutate(() => api.update(key))} + /> + )) + } return (
@@ -92,6 +111,11 @@ export function SandboxServerSettings(props: { {language.t("sandbox.server.label")} + + + {language.t("sandbox.update.available")} + + {meta()}
@@ -122,6 +146,11 @@ export function SandboxServerSettings(props: { {language.t("sandbox.server.stop")} + + + {language.t("sandbox.server.update")} + + api && request.mutate(() => api.reset(key))}> {language.t("sandbox.server.reset")} diff --git a/packages/app/src/sandbox/types.ts b/packages/app/src/sandbox/types.ts index f37b5c3391..ccd5411bfc 100644 --- a/packages/app/src/sandbox/types.ts +++ b/packages/app/src/sandbox/types.ts @@ -50,15 +50,31 @@ export type SandboxRuntime = | { kind: "failed"; message: string } | { kind: "stopped" } +// How a sandbox's guest image compares with the one published now. "custom" is +// a sandbox built from a file the user supplied, which the manifest has no say +// over; "unknown" is one with no readable marker. +export type SandboxImageStatus = + | { kind: "current" } + // preservesHome is what the update dialog promises: on Windows and Linux the + // home is saved host-side and always survives, while on macOS it lives in + // the disk being replaced, so only an image that knows how to read the + // outgoing one can carry it across. + | { kind: "update"; version?: string; preservesHome: boolean } + | { kind: "custom" } + | { kind: "unknown" } + export type SandboxItem = { config: SandboxConfig runtime: SandboxRuntime + // Absent until an update check has answered for this sandbox; a check that + // could not reach the manifest leaves the previous answer alone. + image?: SandboxImageStatus } export type SandboxProgress = { bytes: number; totalBytes: number } export type SandboxJob = { - kind: "create" | "destroy" | "reset" + kind: "create" | "destroy" | "reset" | "update" id: string // When the job was admitted to the queue, which is not when it started // running — status says which of those has happened. @@ -141,6 +157,13 @@ export type SandboxesPlatform = { cancelDefault(): Promise remove(id: string): Promise reset(id: string): Promise + // Rebuilds the sandbox from the image published now, keeping the guest's + // home. Same lifecycle as reset: the sandbox goes down and comes back. + update(id: string): Promise + // Re-asks whether each sandbox's image is still the published one. The main + // process also sweeps on its own; this is for the moment the user is looking + // at the answer. + checkUpdates(): Promise start(id: string): Promise // Shuts the sandbox down until the user starts it again, across relaunches. stop(id: string): Promise diff --git a/packages/app/src/sandbox/update-notice.tsx b/packages/app/src/sandbox/update-notice.tsx new file mode 100644 index 0000000000..47a344f9e5 --- /dev/null +++ b/packages/app/src/sandbox/update-notice.tsx @@ -0,0 +1,77 @@ +import { useDialog } from "@opencode-ai/ui/context/dialog" +import { ButtonV2 } from "@opencode-ai/ui/v2/button-v2" +import { useMutation } from "@tanstack/solid-query" +import { Show, createMemo } from "solid-js" +import { useLanguage } from "@/context/language" +import { usePlatform } from "@/context/platform" +import { showToast } from "@/utils/toast" +import type { WorkspaceSide } from "@/workspace/side" +import { useSandboxes } from "./context" +import { DialogSandboxUpdate } from "./dialog-update" +import { sandboxImageUpdate } from "./image-update" + +// Says on Work that a newer guest image exists. Settings → Servers carries the +// same offer, but a sandbox is something people use from Work and rarely go +// looking for in settings — an update nobody is told about is one nobody takes, +// which is how guests went stale before this existed. +// +// Renders nothing at all unless there is something to act on, so the sidebar +// keeps its layout on every other day. +export function SandboxUpdateNotice(props: { side: WorkspaceSide }) { + const language = useLanguage() + const dialog = useDialog() + const platform = usePlatform() + const sandboxes = useSandboxes() + const api = platform.sandboxes + + // The first sandbox with an update, not a count: one line about one sandbox + // is the whole point, and a machine with several updatable sandboxes gets + // this again once it has dealt with the first. + const pending = createMemo(() => { + if (props.side !== "work") return null + for (const item of sandboxes.state.data?.sandboxes ?? []) { + // A sandbox already busy is either taking this update or being rebuilt + // for another reason; either way there is nothing to offer. + if (sandboxes.state.data?.jobs.some((job) => job.id === item.config.id)) continue + const update = sandboxImageUpdate(item.image) + if (update) return { name: item.config.name, id: item.config.id, update } + } + return null + }) + + const request = useMutation(() => ({ + mutationFn: (action: () => Promise) => action(), + onError: (error) => + showToast({ + variant: "error", + title: language.t("common.requestFailed"), + description: error instanceof Error ? error.message : String(error), + }), + })) + + return ( + + {(item) => ( +
+ {language.t("sandbox.update.notice")} + + dialog.push(() => ( + request.mutate(() => api!.update(item().id))} + /> + )) + } + > + {language.t("sandbox.server.update")} + +
+ )} + + ) +} diff --git a/packages/codev-sandbox/README.md b/packages/codev-sandbox/README.md index 830417cce5..80248b7f0b 100644 --- a/packages/codev-sandbox/README.md +++ b/packages/codev-sandbox/README.md @@ -11,8 +11,11 @@ Electron main process. ## Platform core - `provider.ts` — the `SandboxProvider` contract every backend implements: - `availability` / `create` / `start` / `destroy` / `reset`, plus the optional - `applyMounts` hot path. Also the persisted `SandboxConfig` shape (project + `availability` / `create` / `start` / `destroy` / `reset` / `update` / + `imageStatus`, plus the optional `applyMounts` hot path. `update` and `reset` + both re-provision from the published image and differ in one thing: an update + carries the guest's home across, a reset is a wipe. What that takes is + entirely per-backend, which is why it is a method rather than a flag. Also the persisted `SandboxConfig` shape (project dir, extra mounts, per-sandbox cpu/memory limits) and its validation. - `controller.ts` — lifecycle state machine: runtime states, the job queue, stale-start guards. Create, reset and destroy run **one at a time across all @@ -29,7 +32,15 @@ Electron main process. - `manifest.ts` / `image.ts` — pinned manifest parsing and content-addressed, SHA-256-verified staging of guest images and the mirrored WSL installer. Downloads resume across retries and across calls (images are hundreds of MB - and storage hosts cut long transfers), and report progress for the UI. + and storage hosts cut long transfers), and report progress for the UI. Each + provisioned sandbox records the asset it was built from in an `image.json` + marker, which is what `imageUpdateStatus` compares against the published one + — digests, not versions, so a republished version is still detected. Staged + images are pruned whenever a marker changes hands: after a successful + provision, and after a destroy, dropping anything no sandbox's marker names + any more. Nothing else ever deletes them, so without that an update would + leave its predecessor on disk forever and a deleted sandbox would leave its + whole image behind. - `broker.ts` — the sandbox's only egress path: a host-side proxy that forwards exclusively to configured upstreams (default: the LLM gateway), authenticated with a per-start shared secret. Guests have no network device; there is no @@ -71,7 +82,15 @@ Electron main process. mounts, shared read-only via the `config` virtio-fs tag), waits for the helper's `running` event, then reuses the TCP→unix forwarder and health gate; `reset` re-clones the pristine disk, re-provisioning first when the manifest - has moved on — which makes reset the image-upgrade path. + has moved on. `update` does the same but sets the outgoing `disk.img` aside as + `disk.migrate.img`: the guest's home lives in the disk being replaced, so the + new VM boots once with it attached read-only (`extraDisks` in the helper + protocol, `/dev/vdb` in the guest) and `image/guest/codev-init` copies + `/home/codev` across before starting the server. `start` drops that disk once + the server answers, since a healthy guest is proof the copy finished. Only an + image declaring `home-migration` gets one — the code that reads it lives in + the image being updated **to**, so an older one would leave it there for + nothing, and the app says so before the user agrees. ### Windows — `wsl/` @@ -96,6 +115,10 @@ Electron main process. ### Linux — `bwrap/` +- `bwrap/provider.ts`'s `update` is the whole of that backend's story: the home + is a host directory, so carrying it across is a matter of not deleting it. + Only the rootfs is replaced, and wholesale rather than unpacked over the old + one, so files the new image dropped do not linger. - `bwrap/args.ts` — the bubblewrap invocation: unshared user/pid/net/uts/ipc namespaces (loopback-only networking), read-only rootfs binds with merged-usr symlinks preserved, workspace rw at `/workspace`, server bundle ro at @@ -142,7 +165,9 @@ containerized (any machine with Docker): - `image/kernel/` — a minimal pinned kernel (6.6 LTS) with virtio blk/fs/vsock/console + ext4 compiled in: no initrd, no modules, no network drivers. Cached by Docker layers after the first build. -- `script/build-image.ts` — orchestrates buildx → rootfs tar → ext4 disk +- `script/build-image.ts` — records the guest's capabilities (`features`) on + every asset it describes, so the host knows what an image can do before it + downloads it; orchestrates buildx → rootfs tar → ext4 disk (`mkfs.ext4 -d`, no root needed) → kernel → `vz-.tar.gz` / `bwrap-.tar.gz` → `manifest.json` (validated against `src/manifest.ts` before writing). `--office-bundle` stages the arch-independent parts of @@ -167,12 +192,16 @@ vsock (e.g. via socat) for the broker. This package is platform code only; the desktop app owns everything user-facing. - `packages/desktop/src/main/sandbox/` — `controller.ts` (picks the platform - provider, persists configs), `ipc.ts` (the `sandbox-*` channels), `egress.ts` + provider, persists configs, and sweeps for image updates: once shortly after + launch, then every few hours, sharing one memoized manifest fetch across the + sandboxes it asks about), `ipc.ts` (the `sandbox-*` channels), `egress.ts` (broker routes, gateway credentials, the injected guest config — including the gateway's model catalogue, cached per credential set in `gateway-models.ts`), `bundle.ts` (the mounted server bundle). - `packages/app/src/sandbox/` — the UI: add dialog, mounts dialog, settings - rows, and the per-mount project seeding. + rows, the per-mount project seeding, and the update offer (`update-notice.tsx` + on the Work sidebar, `dialog-update.tsx` for the confirmation, which is where + the promise about the guest's home is made). - `packages/desktop/src/renderer/sandbox/` — ready sandboxes as server connections. @@ -185,6 +214,7 @@ This package is platform code only; the desktop app owns everything user-facing. lost. A graceful path — the helper asking the guest to power down, the guest syncing and powering off — needs a listener in the image and is worth doing the next time the image is rebuilt. -- **Image upgrades are manual.** Reset re-provisions from the current manifest, - but nothing surfaces that a newer image exists, so guests stay stale until - the user happens to reset. +- **An update is still a rebuild.** The guest's home comes across, but anything + the user installed inside the sandbox does not — that is what the new image + replaces. Only the home is carried, because everything else in the guest is + the image. diff --git a/packages/codev-sandbox/image/guest/codev-init b/packages/codev-sandbox/image/guest/codev-init index 68c3fa4bf6..a2ef28277a 100644 --- a/packages/codev-sandbox/image/guest/codev-init +++ b/packages/codev-sandbox/image/guest/codev-init @@ -44,6 +44,29 @@ for (const [key, value] of Object.entries(boot.env)) console.log("export " + key ')" export HOME=/home/codev + +# Update path: the sandbox's outgoing rootfs is attached read-only as a second +# block device and boot.json names it, so the home the user built up survives +# an image change. Runs before the server starts, since the server's own +# database is part of what is being carried. The marker is written in the new +# home because the host keeps the old disk until this boot comes up healthy: a +# boot interrupted in between must not copy a second time over a home that has +# since moved on. +MIGRATE=$(node -e 'process.stdout.write(require("/codev-config/boot.json").migrate ?? "")') +if [ -n "$MIGRATE" ] && [ ! -e "$HOME/.codev-home-migrated" ]; then + mkdir -p /mnt/codev-previous + if mount -o ro "$MIGRATE" /mnt/codev-previous 2>/dev/null; then + if [ -d /mnt/codev-previous/home/codev ]; then + # The old home wins where the two collide — it is the user's — and the + # skel copy below still adds anything only the new image ships. + cp -a /mnt/codev-previous/home/codev/. "$HOME"/ && touch "$HOME/.codev-home-migrated" + fi + umount /mnt/codev-previous + else + echo "codev-init: could not mount $MIGRATE; the previous home was not carried over" >&2 + fi +fi + cp -Rn /etc/skel/. "$HOME"/ 2>/dev/null || true # The server runs as the image's unprivileged codev user; PID 1 and the vsock diff --git a/packages/codev-sandbox/native/vzhelper/Sources/main.swift b/packages/codev-sandbox/native/vzhelper/Sources/main.swift index 2799ecf494..eee2dbd2dc 100644 --- a/packages/codev-sandbox/native/vzhelper/Sources/main.swift +++ b/packages/codev-sandbox/native/vzhelper/Sources/main.swift @@ -25,6 +25,14 @@ struct ForwardConfig: Codable { let socketPath: String } +// A block device beyond the root disk. Today only one is ever attached: the +// outgoing rootfs of a sandbox being updated, read-only, so the incoming guest +// can copy the old home out of it. +struct DiskConfig: Codable { + let path: String + let readOnly: Bool +} + struct VmConfig: Codable { let kernel: String let initrd: String @@ -34,6 +42,8 @@ struct VmConfig: Codable { let memoryMiB: UInt64 let shares: [ShareConfig] let forwards: [ForwardConfig] + // Optional so a config written by an older provider still decodes. + let extraDisks: [DiskConfig]? } let stdoutLock = NSLock() @@ -201,7 +211,16 @@ func buildConfiguration(_ config: VmConfig) throws -> VZVirtualMachineConfigurat VZVirtualMachineConfiguration.maximumAllowedMemorySize ) let disk = try VZDiskImageStorageDeviceAttachment(url: URL(fileURLWithPath: config.disk), readOnly: false) - machine.storageDevices = [VZVirtioBlockDeviceConfiguration(attachment: disk)] + // Attachment order is the guest's device order: the root disk is /dev/vda + // and the extras follow, which is how the guest init addresses them. + var storage = [VZVirtioBlockDeviceConfiguration(attachment: disk)] + for extra in config.extraDisks ?? [] { + let attachment = try VZDiskImageStorageDeviceAttachment( + url: URL(fileURLWithPath: extra.path), readOnly: extra.readOnly + ) + storage.append(VZVirtioBlockDeviceConfiguration(attachment: attachment)) + } + machine.storageDevices = storage machine.entropyDevices = [VZVirtioEntropyDeviceConfiguration()] machine.memoryBalloonDevices = [VZVirtioTraditionalMemoryBalloonDeviceConfiguration()] // Deliberately no network device: the guest's only egress is vsock. diff --git a/packages/codev-sandbox/script/build-image.ts b/packages/codev-sandbox/script/build-image.ts index 0947220a82..653939d039 100644 --- a/packages/codev-sandbox/script/build-image.ts +++ b/packages/codev-sandbox/script/build-image.ts @@ -30,7 +30,7 @@ import { arch as hostArch } from "node:os" import { basename, join } from "node:path" import { fileURLToPath } from "node:url" import { SANDBOX_IMAGES_BASE_URL } from "../src/const" -import { parseSandboxManifest, type SandboxAsset } from "../src/manifest" +import { SANDBOX_IMAGE_FEATURE_HOME_MIGRATION, parseSandboxManifest, type SandboxAsset } from "../src/manifest" import type { SandboxArch, SandboxBackend } from "../src/provider" const imageDir = fileURLToPath(new URL("../image/", import.meta.url)) @@ -214,5 +214,11 @@ async function describeAsset(path: string): Promise { url: `${SANDBOX_IMAGES_BASE_URL}/${version}/${basename(path)}`, sha256: hash.digest("hex"), bytes: file.size, + // What the guest in this image can do, declared by the pipeline that just + // built it. The host reads it before downloading anything: an update can + // only carry the old guest's home across when the image it is updating + // *to* knows how to read the outgoing disk, and the user is told which of + // the two they are about to get. + features: [SANDBOX_IMAGE_FEATURE_HOME_MIGRATION], } } diff --git a/packages/codev-sandbox/script/e2e-vz.ts b/packages/codev-sandbox/script/e2e-vz.ts index a7c8ad270d..ed1df76b1e 100644 --- a/packages/codev-sandbox/script/e2e-vz.ts +++ b/packages/codev-sandbox/script/e2e-vz.ts @@ -6,19 +6,26 @@ // // Usage: // bun script/e2e-vz.ts --image dist-images/vz-arm64.tar.gz \ -// --bundle ../opencode/dist/node [--project dir] [--hold] +// --bundle ../opencode/dist/node [--project dir] [--hold] [--migrate] +// +// --migrate stages the pristine disk as an update's outgoing one, so the VM +// boots with a second (read-only) block device the way it does after an image +// update. On an image declaring `home-migration` the guest also copies +// /home/codev out of it; on an older one it is ignored, which is exactly what +// that flag has to prove is harmless. import { createHash } from "node:crypto" -import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises" +import { constants, copyFile, mkdir, mkdtemp, rm, stat, writeFile } from "node:fs/promises" import { tmpdir } from "node:os" import { join } from "node:path" import { fileURLToPath } from "node:url" import { checkSandboxHealth } from "../src/boot" import type { SandboxConfig } from "../src/provider" -import { createVzProvider } from "../src/vz/provider" +import { VZ_DISK_FILE, createVzProvider, vzSandboxDirs } from "../src/vz/provider" const image = requireArg("--image") const bundle = requireArg("--bundle") const hold = process.argv.includes("--hold") +const migrate = process.argv.includes("--migrate") const work = await mkdtemp(join(tmpdir(), "codev-vz-e2e-")) const projectDir = argValue("--project") ?? join(work, "project") @@ -70,6 +77,12 @@ console.time("create") await provider.create(config) console.timeEnd("create") +const dirs = vzSandboxDirs(join(work, "data"), config.name) +if (migrate) { + await copyFile(join(dirs.image, VZ_DISK_FILE), dirs.migrate, constants.COPYFILE_FICLONE) + console.log("staged a migration disk (the guest boots with /dev/vdb attached)") +} + console.time("start") const sandbox = await provider.start(config) console.timeEnd("start") @@ -77,6 +90,9 @@ console.log(`server: ${sandbox.url} (user ${sandbox.username})`) const healthy = await checkSandboxHealth(sandbox.url, { username: sandbox.username ?? "", password: sandbox.password }) console.log(`health: ${healthy ? "OK" : "FAILED"}`) +// A healthy guest has run its init to the end, which is where the copy would +// have happened, so the disk has served its purpose either way. +if (migrate) console.log(`migration disk dropped: ${(await stat(dirs.migrate).catch(() => null)) === null}`) // /global/health never touches the instance store; /path boots it, so this is // the probe that catches a guest runtime the bundle cannot actually run on diff --git a/packages/codev-sandbox/src/bwrap/provider.test.ts b/packages/codev-sandbox/src/bwrap/provider.test.ts index 167a76a614..0a34c128ec 100644 --- a/packages/codev-sandbox/src/bwrap/provider.test.ts +++ b/packages/codev-sandbox/src/bwrap/provider.test.ts @@ -1,7 +1,7 @@ import { expect, test } from "bun:test" import type { spawn } from "node:child_process" import { EventEmitter } from "node:events" -import { mkdir, mkdtemp, rm, stat, writeFile } from "node:fs/promises" +import { mkdir, mkdtemp, readdir, rm, stat, writeFile } from "node:fs/promises" import { tmpdir } from "node:os" import { join } from "node:path" import { createHash } from "node:crypto" @@ -221,3 +221,135 @@ test("create/reset/destroy manage the sandbox dirs from a staged image", async ( server.stop(true) await rm(dir, { recursive: true, force: true }) }) + +// The home is a host directory here, so carrying it across an image change is +// a matter of not deleting it — and the rootfs is replaced wholesale so files +// the new image dropped do not linger. +test("an update replaces the rootfs and keeps the home", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-bwrap-")) + const buildTar = async (version: string) => { + const tree = join(dir, `tree-${version}`) + await mkdir(join(tree, "etc"), { recursive: true }) + await writeFile(join(tree, "etc", "os-release"), `ID=codev-sandbox\nVERSION=${version}\n`) + if (version === "v1") await writeFile(join(tree, "etc", "gone-in-v2"), "old") + const tarPath = join(dir, `rootfs-${version}.tar`) + expect((await runCommand("tar", ["-cf", tarPath, "-C", tree, "etc"])).code).toBe(0) + return await Bun.file(tarPath).bytes() + } + const payloads = { v1: await buildTar("v1"), v2: await buildTar("v2") } + let active: keyof typeof payloads = "v1" + const server = Bun.serve({ port: 0, fetch: () => new Response(payloads[active]) }) + + const options: BwrapProviderOptions = { + dataDir: join(dir, "data"), + cacheDir: join(dir, "cache"), + bundleDir: join(dir, "bundle"), + platform: "linux", + resolveImage: async () => ({ + url: `http://127.0.0.1:${server.port}/rootfs-${active}.tar`, + sha256: createHash("sha256").update(payloads[active]).digest("hex"), + bytes: payloads[active].byteLength, + }), + } + const provider = createBwrapProvider(options) + const dirs = sandboxDirs(options.dataDir, config.name) + + await provider.create(config) + await writeFile(join(dirs.home, "state.db"), "guest state") + active = "v2" + await provider.update(config) + + expect(await Bun.file(join(dirs.home, "state.db")).text()).toBe("guest state") + expect(await Bun.file(join(dirs.rootfs, "etc", "os-release")).text()).toContain("VERSION=v2") + expect(await stat(join(dirs.rootfs, "etc", "gone-in-v2")).catch(() => null)).toBe(null) + + server.stop(true) + await rm(dir, { recursive: true, force: true }) +}) + +test("reports whether the published image has moved on", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-bwrap-")) + const tree = join(dir, "tree") + await mkdir(join(tree, "etc"), { recursive: true }) + await writeFile(join(tree, "etc", "os-release"), "ID=codev-sandbox\n") + const tarPath = join(dir, "rootfs.tar") + expect((await runCommand("tar", ["-cf", tarPath, "-C", tree, "etc"])).code).toBe(0) + const payload = await Bun.file(tarPath).bytes() + const server = Bun.serve({ port: 0, fetch: () => new Response(payload) }) + + let version = "1.0" + const options: BwrapProviderOptions = { + dataDir: join(dir, "data"), + cacheDir: join(dir, "cache"), + bundleDir: join(dir, "bundle"), + platform: "linux", + resolveImage: async () => ({ + url: `http://127.0.0.1:${server.port}/rootfs.tar`, + sha256: version === "1.0" ? createHash("sha256").update(payload).digest("hex") : "f".repeat(64), + bytes: payload.byteLength, + version, + }), + } + const provider = createBwrapProvider(options) + + await provider.create(config) + expect(await provider.imageStatus(config)).toEqual({ kind: "current" }) + version = "2.0" + // A host-side home always survives, so this backend never has to qualify the + // promise the update dialog makes. + expect(await provider.imageStatus(config)).toEqual({ kind: "update", version: "2.0", preservesHome: true }) + + server.stop(true) + await rm(dir, { recursive: true, force: true }) +}) + +// Nothing else ever removes a staged image, so without this a machine keeps +// every image it has downloaded for good — and deleting a sandbox is the one +// moment a user expects the disk back. An image another sandbox is still on +// has to survive it. +test("destroy drops the cached image nothing else is on", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-bwrap-")) + const buildTar = async (version: string) => { + const tree = join(dir, `tree-${version}`) + await mkdir(join(tree, "etc"), { recursive: true }) + await writeFile(join(tree, "etc", "os-release"), `ID=codev-sandbox\nVERSION=${version}\n`) + const tarPath = join(dir, `rootfs-${version}.tar`) + expect((await runCommand("tar", ["-cf", tarPath, "-C", tree, "etc"])).code).toBe(0) + return await Bun.file(tarPath).bytes() + } + const payloads = { v1: await buildTar("v1"), v2: await buildTar("v2") } + const digests = { + v1: createHash("sha256").update(payloads.v1).digest("hex"), + v2: createHash("sha256").update(payloads.v2).digest("hex"), + } + let active: keyof typeof payloads = "v1" + const server = Bun.serve({ port: 0, fetch: () => new Response(payloads[active]) }) + + const options: BwrapProviderOptions = { + dataDir: join(dir, "data"), + cacheDir: join(dir, "cache"), + bundleDir: join(dir, "bundle"), + platform: "linux", + resolveImage: async () => ({ + url: `http://127.0.0.1:${server.port}/rootfs-${active}.tar`, + sha256: digests[active], + bytes: payloads[active].byteLength, + }), + } + const provider = createBwrapProvider(options) + const other: SandboxConfig = { ...config, id: "sandbox:other", name: "other" } + + await provider.create(config) + active = "v2" + await provider.create(other) + expect((await readdir(options.cacheDir)).sort()).toEqual([digests.v1, digests.v2].sort()) + + await provider.destroy(config) + expect(await readdir(options.cacheDir)).toEqual([digests.v2]) + + await provider.destroy(other) + expect(await readdir(options.cacheDir)).toEqual([]) + + server.stop(true) + await rm(dir, { recursive: true, force: true }) +}) diff --git a/packages/codev-sandbox/src/bwrap/provider.ts b/packages/codev-sandbox/src/bwrap/provider.ts index 676e15b31d..ba63a91250 100644 --- a/packages/codev-sandbox/src/bwrap/provider.ts +++ b/packages/codev-sandbox/src/bwrap/provider.ts @@ -9,7 +9,14 @@ import { sandboxBootEnv, } from "../boot" import { createBrokerServer, type SandboxEgress } from "../broker" -import { localImageAsset, readImageMarker, stageAsset, writeImageMarker } from "../image" +import { + imageUpdateStatus, + localImageAsset, + pruneImageCache, + readImageMarker, + stageAsset, + writeImageMarker, +} from "../image" import type { SandboxAsset } from "../manifest" import { resolveGuestMounts } from "../mounts" import type { SandboxAvailability, SandboxConfig, SandboxJobOptions, SandboxProvider } from "../provider" @@ -83,6 +90,7 @@ export function createBwrapProvider(options: BwrapProviderOptions): SandboxProvi }) await unpackRootfs(image, dirs.rootfs, opts) await writeImageMarker(dirs.base, asset) + await pruneImageCache(options.dataDir, options.cacheDir, [asset.sha256]) } await mkdir(dirs.home, { recursive: true }) } @@ -231,6 +239,9 @@ export function createBwrapProvider(options: BwrapProviderOptions): SandboxProvi async destroy(config) { await rm(sandboxDirs(options.dataDir, config.name).base, { recursive: true, force: true }) + // Its marker went with its directory, so its image is referenced by + // nothing unless another sandbox is on it. + await pruneImageCache(options.dataDir, options.cacheDir, []) }, async reset(config, opts) { @@ -239,5 +250,25 @@ export function createBwrapProvider(options: BwrapProviderOptions): SandboxProvi await rm(dirs.rootfs, { recursive: true, force: true }) await prepare(config, opts) }, + + async update(config, opts) { + // The rootfs is replaced wholesale rather than unpacked over the old + // one, so files the new image dropped do not linger. The home directory + // is host-side and untouched: on this backend that is all it takes to + // carry the guest's state across. + await rm(sandboxDirs(options.dataDir, config.name).rootfs, { recursive: true, force: true }) + await prepare(config, opts) + }, + + imageStatus(config, opts) { + return imageUpdateStatus({ + dir: sandboxDirs(options.dataDir, config.name).base, + config, + resolveImage: options.resolveImage, + // Nothing about the image decides this: the home never lived in it. + preservesHome: () => true, + signal: opts?.signal, + }) + }, } } diff --git a/packages/codev-sandbox/src/controller.test.ts b/packages/codev-sandbox/src/controller.test.ts index e3340ebe83..2aba79598c 100644 --- a/packages/codev-sandbox/src/controller.test.ts +++ b/packages/codev-sandbox/src/controller.test.ts @@ -34,6 +34,8 @@ function fakeProvider(overrides?: Partial): SandboxProvider { start: async () => fakeRunning("http://127.0.0.1:4096"), destroy: async () => undefined, reset: async () => undefined, + update: async () => undefined, + imageStatus: async () => ({ kind: "current" as const }), ...overrides, } } @@ -750,3 +752,158 @@ test("restartRunning skips a sandbox that failed to start", async () => { await controller.restartRunning() expect(starts).toBe(1) }) + +function imageOf(controller: SandboxController, id: string) { + return controller.getState().sandboxes.find((item) => item.config.id === id)?.image +} + +test("an update runs the provider's update, not its reset, and says so on the job", async () => { + let persisted: SandboxConfig[] = [] + const calls: string[] = [] + const kinds: string[] = [] + const controller = createSandboxController({ + providers: { + bwrap: fakeProvider({ + reset: async () => void calls.push("reset"), + update: async () => void calls.push("update"), + }), + }, + readSandboxes: () => persisted, + writeSandboxes: (sandboxes) => { + persisted = sandboxes + }, + }) + await controller.addSandbox(config) + await waitForRuntimeKind(controller, config.id, "ready") + controller.subscribe((event) => { + for (const job of event.state.jobs) kinds.push(job.kind) + }) + await controller.updateSandbox(config.id) + expect(calls).toEqual(["update"]) + expect(kinds).toContain("update") + expect(kinds).not.toContain("reset") + await waitForRuntimeKind(controller, config.id, "ready") +}) + +// A sandbox the user stopped is re-provisioned but not brought up, exactly as +// a reset leaves it: taking an update is not a request to start. +test("updating a stopped sandbox leaves it stopped", async () => { + let persisted: SandboxConfig[] = [] + const controller = createSandboxController({ + providers: { bwrap: fakeProvider() }, + readSandboxes: () => persisted, + writeSandboxes: (sandboxes) => { + persisted = sandboxes + }, + }) + await controller.addSandbox(config) + await waitForRuntimeKind(controller, config.id, "ready") + await controller.stopSandbox(config.id) + await controller.updateSandbox(config.id) + await tick() + expect(runtimeOf(controller, config.id)).toEqual({ kind: "stopped" }) +}) + +test("an update check records each sandbox's verdict", async () => { + let persisted: SandboxConfig[] = [] + const controller = createSandboxController({ + providers: { + bwrap: fakeProvider({ + imageStatus: async (target) => + target.id === config.id ? { kind: "update", version: "2.0", preservesHome: true } : { kind: "current" }, + }), + }, + readSandboxes: () => persisted, + writeSandboxes: (sandboxes) => { + persisted = sandboxes + }, + }) + await controller.addSandbox(config) + await controller.addSandbox(other) + await controller.checkImageUpdates() + expect(imageOf(controller, config.id)).toEqual({ kind: "update", version: "2.0", preservesHome: true }) + expect(imageOf(controller, other.id)).toEqual({ kind: "current" }) +}) + +// Offline, or behind a whitelist that does not cover the storage host: the +// question went unanswered, which is not the same as "up to date". +test("a check that cannot reach the manifest keeps the previous verdict", async () => { + let persisted: SandboxConfig[] = [] + let reachable = true + const controller = createSandboxController({ + providers: { + bwrap: fakeProvider({ + imageStatus: async () => { + if (!reachable) throw new Error("getaddrinfo EAI_AGAIN") + return { kind: "update", version: "2.0", preservesHome: true } + }, + }), + }, + readSandboxes: () => persisted, + writeSandboxes: (sandboxes) => { + persisted = sandboxes + }, + }) + await controller.addSandbox(config) + await controller.checkImageUpdates() + reachable = false + await controller.checkImageUpdates() + expect(imageOf(controller, config.id)).toEqual({ kind: "update", version: "2.0", preservesHome: true }) +}) + +test("an update clears the verdict it was taken for", async () => { + let persisted: SandboxConfig[] = [] + let published = "2.0" + const controller = createSandboxController({ + providers: { + bwrap: fakeProvider({ + imageStatus: async () => + published === "2.0" ? { kind: "update", version: "2.0", preservesHome: true } : { kind: "current" }, + update: async () => { + published = "installed" + }, + }), + }, + readSandboxes: () => persisted, + writeSandboxes: (sandboxes) => { + persisted = sandboxes + }, + }) + await controller.addSandbox(config) + await controller.checkImageUpdates() + await controller.updateSandbox(config.id) + await tick() + expect(imageOf(controller, config.id)).toEqual({ kind: "current" }) +}) + +// The answer is about to change anyway, and asking mid-download would race the +// job that is rewriting the marker. +test("a check skips a sandbox with a job in flight", async () => { + let persisted: SandboxConfig[] = [] + let asked = 0 + let release: () => void = () => {} + const controller = createSandboxController({ + providers: { + bwrap: fakeProvider({ + imageStatus: async () => { + asked++ + return { kind: "current" } + }, + reset: () => new Promise((resolve) => (release = resolve)), + }), + }, + readSandboxes: () => persisted, + writeSandboxes: (sandboxes) => { + persisted = sandboxes + }, + }) + await controller.addSandbox(config) + await waitForRuntimeKind(controller, config.id, "ready") + asked = 0 + const resetting = controller.resetSandbox(config.id) + await tick() + await controller.checkImageUpdates() + expect(asked).toBe(0) + release() + await resetting +}) diff --git a/packages/codev-sandbox/src/controller.ts b/packages/codev-sandbox/src/controller.ts index ed4f819916..b8c920b4f4 100644 --- a/packages/codev-sandbox/src/controller.ts +++ b/packages/codev-sandbox/src/controller.ts @@ -1,3 +1,4 @@ +import type { SandboxImageStatus } from "./image" import type { RunningSandbox, SandboxBackend, @@ -10,7 +11,10 @@ import type { } from "./provider" export type SandboxJob = { - kind: "create" | "destroy" | "reset" + // "update" and "reset" run the same shape of work; they are separate so the + // row can say which one the user asked for, and they differ in whether the + // guest's home survives it. + kind: "create" | "destroy" | "reset" | "update" id: string // When the job was admitted to the queue, which is not when it started // running — status says which of those has happened. @@ -67,6 +71,24 @@ export function createSandboxController(options: SandboxControllerOptions) { }) } + const setImage = (id: string, image: SandboxImageStatus) => { + setState({ + sandboxes: state.sandboxes.map((item) => (item.config.id === id ? { ...item, image } : item)), + }) + } + + // Asking again after the sandbox has been re-provisioned, so the row stops + // offering an update the user has just taken. Silent on failure like the + // sweep: the sandbox is fine either way. + const refreshImageStatus = async (id: string) => { + const item = state.sandboxes.find((x) => x.config.id === id) + if (!item) return + const provider = options.providers[item.config.backend] + if (!provider) return + const status = await provider.imageStatus(item.config).catch(() => null) + if (status && state.sandboxes.some((x) => x.config.id === id)) setImage(id, status) + } + // The persisted store and state.sandboxes[].config are separate; both have // to move together (see addSandbox). const patchConfig = (id: string, patch: Partial) => { @@ -173,6 +195,31 @@ export function createSandboxController(options: SandboxControllerOptions) { } } + // Reset and update differ only in which provider call they make and in what + // the user is told; everything around it — stopping first, the stale-attempt + // guard, not starting a sandbox the user had stopped — is identical. + const reprovision = async (id: string, kind: "reset" | "update") => { + const item = state.sandboxes.find((x) => x.config.id === id) + if (!item) return + invalidateStartAttempt(id) + stopSandboxInternal(id) + setRuntime(id, { kind: "stopped" }) + const done = await runJob(kind, id, async (abort, report) => { + const provider = requireProvider(item.config.backend) + const opts = { signal: abort.signal, onProgress: report } + await (kind === "update" ? provider.update(item.config, opts) : provider.reset(item.config, opts)) + return true + }) + // A cancelled job may have discarded guest state already; leave the + // sandbox stopped instead of starting from a half-provisioned image. + if (!done) return + void refreshImageStatus(id) + // Re-provisioning a sandbox the user stopped does not bring it up — Start + // is still theirs to press. + if (state.sandboxes.find((x) => x.config.id === id)?.config.stopped) return + await startSandboxInternal(id) + } + const startSandboxInternal = async (id: string) => { const item = state.sandboxes.find((x) => x.config.id === id) if (!item) return @@ -252,6 +299,7 @@ export function createSandboxController(options: SandboxControllerOptions) { if (!created) return undefined options.writeSandboxes([...options.readSandboxes(), config]) setState({ sandboxes: [...state.sandboxes, { config, runtime: { kind: "starting" } }] }) + void refreshImageStatus(config.id) void startSandboxInternal(config.id) return config }, @@ -274,23 +322,27 @@ export function createSandboxController(options: SandboxControllerOptions) { setState({ sandboxes: state.sandboxes.filter((x) => x.config.id !== id) }) }, - async resetSandbox(id: string) { - const item = state.sandboxes.find((x) => x.config.id === id) - if (!item) return - invalidateStartAttempt(id) - stopSandboxInternal(id) - setRuntime(id, { kind: "stopped" }) - const done = await runJob("reset", id, async (abort, report) => { - await requireProvider(item.config.backend).reset(item.config, { signal: abort.signal, onProgress: report }) - return true - }) - // A cancelled reset may have discarded guest state already; leave the - // sandbox stopped instead of starting from a half-reset image. - if (!done) return - // Resetting a sandbox the user stopped re-provisions it but does not - // bring it up — Start is still theirs to press. - if (state.sandboxes.find((x) => x.config.id === id)?.config.stopped) return - await startSandboxInternal(id) + resetSandbox(id: string) { + return reprovision(id, "reset") + }, + + // Re-provisions from the image published now, keeping the guest's home. + // Same lifecycle as a reset — the sandbox goes down, is rebuilt and comes + // back — so it shares every one of its guards. + updateSandbox(id: string) { + return reprovision(id, "update") + }, + + // Asks every sandbox whether the image it was built from is still the + // published one. Sequential rather than parallel: the answers come from + // one manifest fetch the desktop memoizes, and a machine with several + // sandboxes has no reason to ask for it several times at once. + async checkImageUpdates() { + for (const item of state.sandboxes) { + // A sandbox mid-create or mid-update is about to change the answer. + if (queue.some((entry) => entry.job.id === item.config.id)) continue + await refreshImageStatus(item.config.id) + } }, // Replaces the extra-mount list. Applied hot where the provider can diff --git a/packages/codev-sandbox/src/image.test.ts b/packages/codev-sandbox/src/image.test.ts index 0521552b2a..dbffdcef8a 100644 --- a/packages/codev-sandbox/src/image.test.ts +++ b/packages/codev-sandbox/src/image.test.ts @@ -1,19 +1,23 @@ import { expect, test } from "bun:test" import { createHash } from "node:crypto" -import { mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises" +import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises" import { tmpdir } from "node:os" import { join } from "node:path" import { pathToFileURL } from "node:url" import { imageMarkerPath, + imageUpdateStatus, isAssetStaged, localImageAsset, partialAssetPath, + pruneImageCache, readImageMarker, stageAsset, stagedAssetPath, writeImageMarker, } from "./image" +import type { SandboxAsset } from "./manifest" +import type { SandboxConfig } from "./provider" const payload = new TextEncoder().encode("codev sandbox image payload") const payloadSha256 = createHash("sha256").update(payload).digest("hex") @@ -301,3 +305,104 @@ test("refuses a local image that has since been deleted", async () => { expect(stageAsset(join(dir, "cache"), asset)).rejects.toThrow("no longer available") await rm(dir, { recursive: true, force: true }) }) + +const config: SandboxConfig = { id: "sandbox:work", name: "work", backend: "bwrap", projectDir: "/work" } +const published: SandboxAsset = { + url: "https://example.invalid/codev/sandbox/2.0/bwrap-arm64.tar.gz", + sha256: "b".repeat(64), + bytes: 10, + version: "2.0", + features: ["home-migration"], +} + +function statusOptions(dir: string, overrides?: Partial[0]>) { + return { + dir, + config, + resolveImage: async () => published, + preservesHome: () => true, + ...overrides, + } +} + +// The version and the feature list ride on the asset, so a marker written now +// records which published image a guest is running and what that image can do. +test("keeps the version and features a marker was written with", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-marker-")) + await writeImageMarker(dir, published) + expect(await readImageMarker(dir)).toEqual(published) + await rm(dir, { recursive: true, force: true }) +}) + +test("reads a marker written before images carried a version", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-marker-")) + await writeFile(imageMarkerPath(dir), JSON.stringify({ url: published.url, sha256: published.sha256, bytes: 10 })) + expect(await readImageMarker(dir)).toEqual({ url: published.url, sha256: published.sha256, bytes: 10 }) + await rm(dir, { recursive: true, force: true }) +}) + +test("reports an update when the published digest has moved on", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-status-")) + await writeImageMarker(dir, { url: published.url, sha256: "a".repeat(64), bytes: 10, version: "1.0" }) + expect(await imageUpdateStatus(statusOptions(dir))).toEqual({ kind: "update", version: "2.0", preservesHome: true }) + await rm(dir, { recursive: true, force: true }) +}) + +test("reports current when the digest still matches, whatever the version says", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-status-")) + await writeImageMarker(dir, { ...published, version: "1.0" }) + expect(await imageUpdateStatus(statusOptions(dir))).toEqual({ kind: "current" }) + await rm(dir, { recursive: true, force: true }) +}) + +// The manifest is not a custom image's source, so it has no update to offer — +// and asking would compare it against an image built for someone else. +test("offers nothing for a sandbox built from a custom image", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-status-")) + const status = await imageUpdateStatus( + statusOptions(dir, { + config: { ...config, imagePath: "/images/custom.tar.gz" }, + resolveImage: async () => { + throw new Error("the manifest must not be consulted for a custom image") + }, + }), + ) + expect(status).toEqual({ kind: "custom" }) + await rm(dir, { recursive: true, force: true }) +}) + +test("reports unknown when the sandbox has no readable marker", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-status-")) + expect(await imageUpdateStatus(statusOptions(dir))).toEqual({ kind: "unknown" }) + await rm(dir, { recursive: true, force: true }) +}) + +// The macOS backend can only carry a home across when the image it is updating +// to knows how to read the outgoing disk; the dialog says which of the two the +// user is about to get. +test("passes the backend's verdict on whether the home survives", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-status-")) + await writeImageMarker(dir, { ...published, sha256: "a".repeat(64) }) + const status = await imageUpdateStatus(statusOptions(dir, { preservesHome: (asset) => asset.version === "3.0" })) + expect(status).toEqual({ kind: "update", version: "2.0", preservesHome: false }) + await rm(dir, { recursive: true, force: true }) +}) + +test("prunes staged images nothing points at any more", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-prune-")) + const dataDir = join(dir, "sandboxes") + const cacheDir = join(dir, "cache") + const stale = join(cacheDir, "c".repeat(64)) + const held = join(cacheDir, "a".repeat(64)) + const current = join(cacheDir, published.sha256) + for (const staged of [stale, held, current]) { + await mkdir(staged, { recursive: true }) + await writeFile(join(staged, "image.tar.gz"), "x") + } + // Another sandbox is still on the older image; a reset of it must not have + // to download what this machine already has. + await writeImageMarker(join(dataDir, "other"), { ...published, sha256: "a".repeat(64) }) + await pruneImageCache(dataDir, cacheDir, [published.sha256]) + expect((await readdir(cacheDir)).sort()).toEqual([held, current].map((path) => path.split("/").pop()!).sort()) + await rm(dir, { recursive: true, force: true }) +}) diff --git a/packages/codev-sandbox/src/image.ts b/packages/codev-sandbox/src/image.ts index 1922121258..3f0d19f790 100644 --- a/packages/codev-sandbox/src/image.ts +++ b/packages/codev-sandbox/src/image.ts @@ -4,8 +4,9 @@ import { mkdir, readdir, readFile, rename, rm, stat, writeFile } from "node:fs/p import { finished } from "node:stream/promises" import { basename, dirname, join } from "node:path" import { fileURLToPath, pathToFileURL } from "node:url" +import { SANDBOX_IMAGE_FEATURE_HOME_MIGRATION } from "./manifest" import type { SandboxAsset } from "./manifest" -import type { SandboxProgress } from "./provider" +import type { SandboxConfig, SandboxProgress } from "./provider" export type StageAssetOptions = { fetchImpl?: typeof fetch @@ -235,7 +236,80 @@ export async function readImageMarker(dir: string): Promise const record = parsed as Record if (typeof record.url !== "string" || typeof record.sha256 !== "string" || typeof record.bytes !== "number") return null - return { url: record.url, sha256: record.sha256, bytes: record.bytes } + return { + url: record.url, + sha256: record.sha256, + bytes: record.bytes, + // Absent in every marker written before images carried a version, which is + // why the update check compares digests and treats the version as display + // text only. + ...(typeof record.version === "string" ? { version: record.version } : {}), + ...(Array.isArray(record.features) && record.features.every((entry) => typeof entry === "string") + ? { features: record.features as string[] } + : {}), + } +} + +// What a provisioned sandbox's image is, compared with the one published now. +// "custom" is a sandbox built from a file the user supplied: the manifest is +// not its source, so it has no update to offer. "unknown" is a sandbox with no +// readable marker, which create would re-provision anyway. +export type SandboxImageStatus = + | { kind: "current" } + | { kind: "update"; version?: string; preservesHome: boolean } + | { kind: "custom" } + | { kind: "unknown" } + +// Shared by all three backends: the marker lives in the provider's own +// directory and the current asset comes from the provider's resolver, but the +// comparison itself is the same everywhere. +export async function imageUpdateStatus(options: { + dir: string + config: SandboxConfig + resolveImage: (opts?: { signal?: AbortSignal }) => Promise + // Whether an update can carry the guest's home across. Host-side homes + // (bubblewrap's directory, WSL's exported tarball) always can; the vz backend + // can only when the image it is updating to knows how. + preservesHome: (asset: SandboxAsset) => boolean + signal?: AbortSignal +}): Promise { + if (options.config.imagePath) return { kind: "custom" } + const marker = await readImageMarker(options.dir) + if (!marker) return { kind: "unknown" } + const asset = await options.resolveImage({ signal: options.signal }) + if (asset.sha256 === marker.sha256) return { kind: "current" } + return { + kind: "update", + ...(asset.version ? { version: asset.version } : {}), + preservesHome: options.preservesHome(asset), + } +} + +const SHA256_DIR = /^[0-9a-f]{64}$/ + +// Staged images are content-addressed and nothing else ever removes them, so +// without this every update leaves its predecessor behind — over a gigabyte a +// time. Keeps whatever a provisioned sandbox's marker still points at, so a +// reset does not re-download an image the machine already has, and drops the +// rest. Safe only because create/update/reset are serialized: a staged +// directory no marker names is not being written by anyone. +export async function pruneImageCache(dataDir: string, cacheDir: string, keep: string[]) { + const referenced = new Set(keep) + for (const entry of await readdir(dataDir, { withFileTypes: true }).catch(() => [])) { + if (!entry.isDirectory()) continue + const marker = await readImageMarker(join(dataDir, entry.name)) + if (marker) referenced.add(marker.sha256) + } + const staged = await readdir(cacheDir, { withFileTypes: true }).catch(() => []) + await Promise.all( + staged + .filter((entry) => entry.isDirectory() && SHA256_DIR.test(entry.name) && !referenced.has(entry.name)) + .map((entry) => rm(join(cacheDir, entry.name), { recursive: true, force: true })), + ) +} + +export function imageMigratesHome(asset: SandboxAsset) { + return !!asset.features?.includes(SANDBOX_IMAGE_FEATURE_HOME_MIGRATION) } export async function writeImageMarker(dir: string, asset: SandboxAsset) { diff --git a/packages/codev-sandbox/src/manifest.test.ts b/packages/codev-sandbox/src/manifest.test.ts index 040c8ceb74..d0b699fc7f 100644 --- a/packages/codev-sandbox/src/manifest.test.ts +++ b/packages/codev-sandbox/src/manifest.test.ts @@ -106,3 +106,31 @@ test("an unknown installer manifest schema is refused", () => { parseWslInstallerManifest({ schema: 2, version: "2.7.11", recommendedMinBuild: 19041, assets: {} }), ).toThrow("schema") }) + +// The manifest names the version once; attaching it to the asset is what lets +// a provisioned sandbox's marker record which published image it is running. +test("attaches the manifest version to the image it hands out", () => { + const manifest = parseSandboxManifest(validManifest) + expect(requireImageAsset(manifest, "vz", "arm64").version).toBe("2026.08.1") +}) + +test("carries an image's declared features through", () => { + const manifest = parseSandboxManifest({ + ...validManifest, + images: { "vz-arm64": { ...asset("vz-arm64.img"), features: ["home-migration"] } }, + }) + expect(requireImageAsset(manifest, "vz", "arm64").features).toEqual(["home-migration"]) +}) + +// Every manifest published so far has none, and an app that predates a feature +// name has to keep working when a newer pipeline adds one. +test("accepts an image that declares no features", () => { + const manifest = parseSandboxManifest(validManifest) + expect(requireImageAsset(manifest, "vz", "arm64").features).toBeUndefined() +}) + +test("rejects features that are not a list of strings", () => { + expect(() => + parseSandboxManifest({ ...validManifest, images: { "vz-arm64": { ...asset("vz.img"), features: "home" } } }), + ).toThrow("features") +}) diff --git a/packages/codev-sandbox/src/manifest.ts b/packages/codev-sandbox/src/manifest.ts index 92e84564cc..712894d2c3 100644 --- a/packages/codev-sandbox/src/manifest.ts +++ b/packages/codev-sandbox/src/manifest.ts @@ -4,8 +4,24 @@ export type SandboxAsset = { url: string sha256: string bytes: number + // The manifest's imageVersion, attached by requireImageAsset rather than + // stored per asset: one publish builds every image, so recording it on the + // asset is what lets a provisioned sandbox's marker say which published + // version it is running. + version?: string + // What the guest inside this image can do, for decisions the host has to + // make before downloading it — today only whether an update can carry the + // old guest's home across (SANDBOX_IMAGE_FEATURE_HOME_MIGRATION). + features?: string[] } +// The guest init mounts the outgoing rootfs handed to it as a migration disk +// and copies its home into the new one. Declared per image because the code +// that performs it lives in the image being updated *to*: an image published +// before this existed cannot carry a home across, and the host has to know +// that before it offers the update, not after. +export const SANDBOX_IMAGE_FEATURE_HOME_MIGRATION = "home-migration" + // The WSL runtime installers, one asset per architecture. `recommendedMinBuild` // is the Windows build Microsoft supports this WSL version on; the availability // warning reads it, so publishing a WSL version that raises the floor re-words @@ -92,7 +108,7 @@ export async function fetchSandboxManifest( export function requireImageAsset(manifest: SandboxManifest, backend: SandboxBackend, arch: SandboxArch) { const asset = manifest.images[manifestImageKey(backend, arch)] if (!asset) throw new Error(`sandbox manifest has no image for ${manifestImageKey(backend, arch)}`) - return asset + return { ...asset, version: manifest.imageVersion } } const SHA256_HEX = /^[0-9a-f]{64}$/ @@ -113,7 +129,17 @@ function parseAsset(label: string, value: unknown): SandboxAsset { if (!SHA256_HEX.test(sha256)) throw new Error(`sandbox asset ${label} needs a sha256 hex digest`) if (typeof record.bytes !== "number" || !Number.isInteger(record.bytes) || record.bytes <= 0) throw new Error(`sandbox asset ${label} needs a positive byte size`) - return { url: record.url, sha256, bytes: record.bytes } + return { url: record.url, sha256, bytes: record.bytes, ...parseFeatures(label, record.features) } +} + +// Absent on every manifest published so far, and absent means the image +// declares nothing — an unknown feature name is ignored rather than rejected, +// so a newer pipeline can add one without breaking older apps. +function parseFeatures(label: string, value: unknown) { + if (value === undefined || value === null) return {} + if (!Array.isArray(value) || value.some((entry) => typeof entry !== "string")) + throw new Error(`sandbox asset ${label} features must be a list of strings`) + return { features: value as string[] } } function parseWslInstaller(value: unknown): SandboxWslInstaller | null { diff --git a/packages/codev-sandbox/src/provider.ts b/packages/codev-sandbox/src/provider.ts index d90bf4d294..a1dbb60d50 100644 --- a/packages/codev-sandbox/src/provider.ts +++ b/packages/codev-sandbox/src/provider.ts @@ -1,3 +1,5 @@ +import type { SandboxImageStatus } from "./image" + export type SandboxBackend = "vz" | "wsl" | "bwrap" export type SandboxArch = "x64" | "arm64" @@ -74,6 +76,11 @@ export type SandboxRuntime = export type SandboxItem = { config: SandboxConfig runtime: SandboxRuntime + // How this sandbox's image compares with the published one. Absent until an + // update check has answered for it: a check that cannot reach the manifest + // leaves the previous verdict alone rather than reporting "up to date" about + // a question nobody answered. + image?: SandboxImageStatus } export type RunningSandbox = { @@ -102,6 +109,17 @@ export type SandboxProvider = { start(config: SandboxConfig, opts?: { signal?: AbortSignal }): Promise destroy(config: SandboxConfig, opts?: { signal?: AbortSignal }): Promise reset(config: SandboxConfig, opts?: SandboxJobOptions): Promise + // Re-provisions from the currently published image while carrying the + // guest's home across — the difference from reset, which is a wipe. A + // separate method rather than a flag on reset because what it takes to keep + // a home differs completely per backend: bubblewrap's is a host directory + // that simply is not deleted, WSL's is tarred out before the distro is + // unregistered, and vz hands the outgoing disk to the incoming guest. + update(config: SandboxConfig, opts?: SandboxJobOptions): Promise + // Whether the published image has moved on from the one this sandbox was + // provisioned from. Reaches the network, so it throws when the manifest + // cannot be fetched; callers treat that as "no answer", not as "current". + imageStatus(config: SandboxConfig, opts?: { signal?: AbortSignal }): Promise // Applies config.mounts to the already-running sandbox without a restart. // Only backends where that is possible implement it (WSL: the distro's // mount namespace is shared, and automounted drives need no guest work at diff --git a/packages/codev-sandbox/src/vz/protocol.ts b/packages/codev-sandbox/src/vz/protocol.ts index faed51777a..bfd824b713 100644 --- a/packages/codev-sandbox/src/vz/protocol.ts +++ b/packages/codev-sandbox/src/vz/protocol.ts @@ -8,6 +8,14 @@ export type VzShare = { readOnly: boolean } +// A block device beyond the root disk, in the order the guest sees them: the +// root disk is /dev/vda and these follow. Only used by an update, which hands +// the outgoing rootfs to the incoming guest read-only. +export type VzDisk = { + path: string + readOnly: boolean +} + export type VzForward = { direction: "host-to-guest" | "guest-to-host" vsockPort: number @@ -23,6 +31,7 @@ export type VzVmConfig = { memoryMiB: number shares: VzShare[] forwards: VzForward[] + extraDisks?: VzDisk[] } export type VzHelperEvent = diff --git a/packages/codev-sandbox/src/vz/provider.test.ts b/packages/codev-sandbox/src/vz/provider.test.ts index 3b7b686743..d5031c1c87 100644 --- a/packages/codev-sandbox/src/vz/provider.test.ts +++ b/packages/codev-sandbox/src/vz/provider.test.ts @@ -356,3 +356,147 @@ test("create rejects an image tarball missing required files", async () => { server.stop(true) await rm(dir, { recursive: true, force: true }) }) + +// The guest's home lives in the instance disk, so an update sets the outgoing +// one aside for the new guest to copy out of — but only when the image it is +// updating to knows how to read it, since otherwise the disk would sit there +// for nothing. +test("an update keeps the outgoing disk for an image that can migrate the home", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-vz-")) + const buildTar = async (version: string) => { + const tree = join(dir, `tree-${version}`) + await mkdir(tree, { recursive: true }) + await writeFile(join(tree, "vmlinux"), "kernel") + await writeFile(join(tree, "rootfs.img"), `disk-${version}`) + const tarPath = join(dir, `vz-${version}.tar`) + expect((await runCommand("tar", ["-cf", tarPath, "-C", tree, "vmlinux", "rootfs.img"])).code).toBe(0) + return await Bun.file(tarPath).bytes() + } + const payloads = { v1: await buildTar("v1"), v2: await buildTar("v2"), v3: await buildTar("v3") } + let active: keyof typeof payloads = "v1" + let features: string[] | undefined = ["home-migration"] + const server = Bun.serve({ port: 0, fetch: () => new Response(payloads[active]) }) + + const options: VzProviderOptions = { + dataDir: join(dir, "data"), + cacheDir: join(dir, "cache"), + bundleDir: join(dir, "bundle"), + helperPath: join(dir, "codev-vmm"), + platform: "darwin", + resolveImage: async () => ({ + url: `http://127.0.0.1:${server.port}/vz-${active}.tar`, + sha256: createHash("sha256").update(payloads[active]).digest("hex"), + bytes: payloads[active].byteLength, + ...(features ? { features } : {}), + }), + } + const provider = createVzProvider(options) + const dirs = vzSandboxDirs(options.dataDir, config.name) + + await provider.create(config) + await writeFile(dirs.disk, "guest state") + active = "v2" + await provider.update(config) + expect(await readFile(dirs.disk, "utf8")).toBe("disk-v2") + expect(await readFile(dirs.migrate, "utf8")).toBe("guest state") + + // An image that cannot read the outgoing disk gets no migration disk left + // behind: the update still runs, as a fresh sandbox. + await rm(dirs.migrate, { force: true }) + active = "v3" + features = undefined + await provider.update(config) + expect(await readFile(dirs.disk, "utf8")).toBe("disk-v3") + expect(await stat(dirs.migrate).catch(() => null)).toBe(null) + + server.stop(true) + await rm(dir, { recursive: true, force: true }) +}) + +// An update whose guest never came up healthy leaves the migration disk in +// place; updating again must not replace it with the disk that update itself +// provisioned, or the home it was keeping goes with it. +test("a second update leaves an unconsumed migration disk alone", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-vz-")) + const tree = join(dir, "tree") + await mkdir(tree, { recursive: true }) + await writeFile(join(tree, "vmlinux"), "kernel") + await writeFile(join(tree, "rootfs.img"), "fresh-disk") + const tarPath = join(dir, "vz.tar") + expect((await runCommand("tar", ["-cf", tarPath, "-C", tree, "vmlinux", "rootfs.img"])).code).toBe(0) + const payload = await Bun.file(tarPath).bytes() + const server = Bun.serve({ port: 0, fetch: () => new Response(payload) }) + + const options: VzProviderOptions = { + dataDir: join(dir, "data"), + cacheDir: join(dir, "cache"), + bundleDir: join(dir, "bundle"), + helperPath: join(dir, "codev-vmm"), + platform: "darwin", + resolveImage: async () => ({ + url: `http://127.0.0.1:${server.port}/vz.tar`, + sha256: createHash("sha256").update(payload).digest("hex"), + bytes: payload.byteLength, + features: ["home-migration"], + }), + } + const provider = createVzProvider(options) + const dirs = vzSandboxDirs(options.dataDir, config.name) + + await provider.create(config) + await writeFile(dirs.migrate, "the home an interrupted update was keeping") + await provider.update(config) + expect(await readFile(dirs.migrate, "utf8")).toBe("the home an interrupted update was keeping") + + server.stop(true) + await rm(dir, { recursive: true, force: true }) +}) + +// One boot with the old disk attached, and only one: the guest copies the home +// before it starts the server, so a healthy start means the copy is done. +test("start attaches a pending migration disk and drops it once the guest answers", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-vz-")) + const prepared = await preparedOptions(dir, FAKE_HELPER) + await writeFile(prepared.dirs.migrate, "previous disk") + const provider = createVzProvider(prepared.options) + + const sandbox = await provider.start({ ...config, projectDir: prepared.projectDir }) + const vmConfig = JSON.parse(await readFile(join(prepared.dirs.runtime, "vm.json"), "utf8")) + expect(vmConfig.extraDisks).toEqual([{ path: prepared.dirs.migrate, readOnly: true }]) + const boot = JSON.parse(await readFile(join(prepared.dirs.guestConfig, "boot.json"), "utf8")) + expect(boot.migrate).toBe("/dev/vdb") + expect(await stat(prepared.dirs.migrate).catch(() => null)).toBe(null) + + sandbox.stop() + await rm(dir, { recursive: true, force: true }) +}) + +test("an ordinary start attaches no extra disk and asks for no migration", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-vz-")) + const prepared = await preparedOptions(dir, FAKE_HELPER) + const provider = createVzProvider(prepared.options) + + const sandbox = await provider.start({ ...config, projectDir: prepared.projectDir }) + const vmConfig = JSON.parse(await readFile(join(prepared.dirs.runtime, "vm.json"), "utf8")) + expect(vmConfig.extraDisks).toBeUndefined() + expect(JSON.parse(await readFile(join(prepared.dirs.guestConfig, "boot.json"), "utf8")).migrate).toBeUndefined() + + sandbox.stop() + await rm(dir, { recursive: true, force: true }) +}) + +// A wipe leaves nothing to carry over, so an interrupted update's disk must not +// outlive it and get copied into the next guest. +test("reset discards a migration disk left by an interrupted update", async () => { + const dir = await mkdtemp(join(tmpdir(), "codev-vz-")) + const prepared = await preparedOptions(dir, FAKE_HELPER, { + resolveImage: async () => { + throw new Error("offline") + }, + }) + await writeFile(prepared.dirs.migrate, "previous disk") + const provider = createVzProvider(prepared.options) + await provider.reset(config).catch(() => null) + expect(await stat(prepared.dirs.migrate).catch(() => null)).toBe(null) + await rm(dir, { recursive: true, force: true }) +}) diff --git a/packages/codev-sandbox/src/vz/provider.ts b/packages/codev-sandbox/src/vz/provider.ts index 1d3f73f587..f0c443367b 100644 --- a/packages/codev-sandbox/src/vz/provider.ts +++ b/packages/codev-sandbox/src/vz/provider.ts @@ -1,11 +1,19 @@ import { spawn } from "node:child_process" -import { constants, copyFile, mkdir, rm, stat, writeFile } from "node:fs/promises" +import { constants, copyFile, mkdir, rename, rm, stat, writeFile } from "node:fs/promises" import { join } from "node:path" import { checkSandboxHealth, generateSandboxCredentials, pollSandboxHealth, sandboxBootEnv } from "../boot" import { createBrokerServer, type SandboxEgress } from "../broker" import { forwardTcpToUnix } from "../bwrap/bridge" import { runCommand, unpackRootfs } from "../bwrap/rootfs" -import { localImageAsset, readImageMarker, stageAsset, writeImageMarker } from "../image" +import { + imageMigratesHome, + imageUpdateStatus, + localImageAsset, + pruneImageCache, + readImageMarker, + stageAsset, + writeImageMarker, +} from "../image" import type { SandboxAsset } from "../manifest" import { resolveGuestMounts } from "../mounts" import { macosSandboxAvailability } from "../policy" @@ -26,6 +34,9 @@ export const VZ_SERVER_VSOCK_PORT = 4096 export const VZ_BROKER_VSOCK_PORT = 8788 export const VZ_GUEST_HOME = "/home/codev" const VZ_CMDLINE = "console=hvc0 root=/dev/vda rw init=/sbin/codev-init" +// The outgoing rootfs of an update, as the guest sees it: the root disk is +// /dev/vda and the helper attaches the extras in order after it. +const VZ_MIGRATE_DEVICE = "/dev/vdb" export type VzProviderOptions = { dataDir: string @@ -58,6 +69,9 @@ export function vzSandboxDirs(dataDir: string, name: string) { base, image: join(base, "image"), disk: join(base, "disk.img"), + // The disk an update is replacing, kept until the new guest has copied the + // old home out of it and come up healthy. + migrate: join(base, "disk.migrate.img"), runtime: join(base, "runtime"), guestConfig: join(base, "guest-config"), } @@ -89,6 +103,7 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { } if (missing.length) throw new Error(`vz image is missing ${missing.join(", ")}`) await writeImageMarker(dirs.base, asset) + await pruneImageCache(options.dataDir, options.cacheDir, [asset.sha256]) } // The pristine unpacked disk stays untouched; the instance runs a // copy-on-write clone (APFS clonefile via FICLONE, plain copy elsewhere), @@ -164,6 +179,10 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { if (broker) await broker.listenUnix(join(socketDir, "broker.sock")) if (egress?.guestConfig) await writeFile(join(dirs.guestConfig, "codev.json"), egress.guestConfig) + // An update leaves the outgoing rootfs behind for exactly one boot: the + // guest copies its home across before starting the server, and this + // start deletes it once the server answers. + const migrateDisk = await stat(dirs.migrate).catch(() => null) const credentials = generateSandboxCredentials() await writeFile( join(dirs.guestConfig, "boot.json"), @@ -173,6 +192,7 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { // Merged between the guest's global and project configs. ...(egress?.guestConfig ? { CODEV_CONFIG: "/codev-config/codev.json" } : {}), }, + ...(migrateDisk?.isFile() ? { migrate: VZ_MIGRATE_DEVICE } : {}), }), ) const serverSocket = join(socketDir, "server.sock") @@ -195,6 +215,7 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { { direction: "host-to-guest", vsockPort: VZ_SERVER_VSOCK_PORT, socketPath: serverSocket }, { direction: "guest-to-host", vsockPort: VZ_BROKER_VSOCK_PORT, socketPath: join(socketDir, "broker.sock") }, ], + ...(migrateDisk?.isFile() ? { extraDisks: [{ path: dirs.migrate, readOnly: true }] } : {}), } const configPath = join(dirs.runtime, "vm.json") await writeFile(configPath, JSON.stringify(vmConfig)) @@ -269,6 +290,12 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { ) } + // Healthy means the guest ran its init to completion, which is where the + // home copy happens — so the old disk has served its purpose. Dropping + // it while the VM still has it open is fine: the bytes go when the last + // reference closes. + if (migrateDisk?.isFile()) await rm(dirs.migrate, { force: true }) + let stopKill: ReturnType | undefined void exited.then(() => clearTimeout(stopKill)) return { @@ -295,10 +322,18 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { async destroy(config) { await rm(vzSandboxDirs(options.dataDir, config.name).base, { recursive: true, force: true }) + // The sandbox's marker went with its directory, so the image it was + // built from is now referenced by nothing — unless another sandbox is on + // it, which the prune checks for. Deleting is the one moment a user + // expects the disk back. + await pruneImageCache(options.dataDir, options.cacheDir, []) }, async reset(config, opts) { const dirs = vzSandboxDirs(options.dataDir, config.name) + // A reset is a wipe, so a migration left pending by an interrupted + // update has nothing left to carry over. + await rm(dirs.migrate, { force: true }) // Guest state lives in the instance disk. When the manifest still points // at the image this sandbox was built from, reset is just re-cloning the // pristine disk; when it has moved on, reset doubles as the upgrade path @@ -306,6 +341,36 @@ export function createVzProvider(options: VzProviderOptions): SandboxProvider { await rm(dirs.disk, { force: true }) await provision(config, await resolveAsset(config, opts), opts) }, + + async update(config, opts) { + const dirs = vzSandboxDirs(options.dataDir, config.name) + const asset = await resolveAsset(config, opts) + // The guest's home lives in the instance disk, which is exactly what is + // being replaced — so the outgoing disk is set aside for the incoming + // guest to copy out of, and start() drops it once that has happened. + // Only an image that knows how to read it makes that worth doing; with + // an older one the update still runs, as a fresh sandbox. + // + // A migration disk already there belongs to an update no boot has + // consumed yet, and holds the home this sandbox is still trying to keep; + // the disk being replaced now is the one that same update provisioned, so + // it must not overwrite it. Nothing is lost by leaving it: rename is the + // only writer, and start drops it once a guest has come up healthy. + const pending = await stat(dirs.migrate).catch(() => null) + if (imageMigratesHome(asset) && !pending?.isFile()) await rename(dirs.disk, dirs.migrate).catch(() => {}) + await rm(dirs.disk, { force: true }) + await provision(config, asset, opts) + }, + + imageStatus(config, opts) { + return imageUpdateStatus({ + dir: vzSandboxDirs(options.dataDir, config.name).base, + config, + resolveImage: options.resolveImage, + preservesHome: imageMigratesHome, + signal: opts?.signal, + }) + }, } } diff --git a/packages/codev-sandbox/src/wsl/cli.ts b/packages/codev-sandbox/src/wsl/cli.ts index 82c0604f41..675012a6bf 100644 --- a/packages/codev-sandbox/src/wsl/cli.ts +++ b/packages/codev-sandbox/src/wsl/cli.ts @@ -1,4 +1,5 @@ import { spawn } from "node:child_process" +import { createReadStream, createWriteStream } from "node:fs" import { createServer } from "node:net" export type WslResult = { @@ -56,6 +57,60 @@ export function runWsl( }) } +// Streams a host file into the distro's stdin or the distro's stdout into a +// host file, for the tar that carries a sandbox's home across an update. +// runWsl cannot: it decodes output as text, which mangles an archive. Keeping +// the file on stdio rather than as an argument also keeps Windows paths off +// the command line, so nothing has to be quoted or translated with wslpath. +export function pipeWsl( + args: string[], + opts: { + stdinFile?: string + stdoutFile?: string + wslCommand?: string + spawnImpl?: typeof spawn + timeoutMs?: number + }, +): Promise<{ code: number | null; stderr: string }> { + const spawnImpl = opts.spawnImpl ?? spawn + const child = spawnImpl(opts.wslCommand ?? "wsl.exe", args, { + stdio: [opts.stdinFile ? "pipe" : "ignore", opts.stdoutFile ? "pipe" : "ignore", "pipe"], + }) + const stderr: Buffer[] = [] + child.stderr?.on("data", (chunk: Buffer) => stderr.push(chunk)) + if (opts.stdinFile && child.stdin) createReadStream(opts.stdinFile).pipe(child.stdin) + // Closed before resolving: an archive still buffered in the OS when the + // child exits would otherwise be read back short. + const written = opts.stdoutFile && child.stdout ? drainToFile(child.stdout, opts.stdoutFile) : Promise.resolve() + return new Promise((resolve, reject) => { + const timeoutMs = opts.timeoutMs ?? DEFAULT_WSL_TIMEOUT_MS + const timeout = setTimeout(() => { + child.kill() + reject(new Error(`wsl ${args.join(" ")} timed out after ${timeoutMs}ms`)) + }, timeoutMs) + child.once("error", (error) => { + clearTimeout(timeout) + reject(error) + }) + child.once("close", (code) => { + clearTimeout(timeout) + written.then( + () => resolve({ code, stderr: decodeWslOutput(Buffer.concat(stderr)) }), + (error: unknown) => reject(error instanceof Error ? error : new Error(String(error))), + ) + }) + }) +} + +function drainToFile(source: NodeJS.ReadableStream, path: string) { + return new Promise((resolve, reject) => { + const out = createWriteStream(path) + out.once("error", reject) + out.once("close", () => resolve()) + source.pipe(out) + }) +} + // `wsl --list --quiet` prints one distro per line; entries can carry a trailing // CR and the default distro is not marked in --quiet mode. export function parseDistroList(stdout: string) { diff --git a/packages/codev-sandbox/src/wsl/provider.test.ts b/packages/codev-sandbox/src/wsl/provider.test.ts index e58ff5b345..6dd72729a9 100644 --- a/packages/codev-sandbox/src/wsl/provider.test.ts +++ b/packages/codev-sandbox/src/wsl/provider.test.ts @@ -1,6 +1,6 @@ import { expect, test } from "bun:test" import { spawn } from "node:child_process" -import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises" +import { mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises" import { tmpdir } from "node:os" import { join } from "node:path" import type { SandboxConfig } from "../provider" @@ -37,6 +37,15 @@ if (args[0] === "--unregister") { } if (args.includes("wslpath")) { process.stdout.write("/mnt/hostpath\\n"); process.exit(0) } +// Update path: the provider checks for a home, tars it out over stdout and +// tars it back in over stdin. Binary on purpose — the archive must not be +// decoded as text on its way through. +if (args.includes("test")) { process.exit(read().home === false ? 1 : 0) } +if (args.includes("tar")) { + if (args.includes("-cf")) { process.stdout.write("home-archive"); process.exit(0) } + const value = read(); value.restored = readFileSync(0, "utf8"); write(value); process.exit(0) +} + // The boot script arrives on stdin (never argv — wsl.exe corrupts quoted // spaces there); import-time configuration still uses -- /bin/sh -c. const script = args[args.length - 1] === "/bin/sh" ? readFileSync(0, "utf8") : args[args.length - 1] @@ -386,3 +395,100 @@ test("create then start reaches an authenticated health check, and destroy unreg expect(state.distros).toEqual([]) await rm(dir, { recursive: true, force: true }) }) + +// The distro cannot be imported over itself, so the home leaves first and +// comes back after — the export deliberately runs before anything is +// destroyed, so a failure there costs nothing. +test("an update rebuilds the distro and carries the guest home across", async () => { + const dir = await mkdtemp(join(tmpdir(), "wsl-test-")) + const options = await preparedOptions(dir) + const provider = createWslProvider(options) + await provider.create(config) + + // A second image: create is idempotent on a matching digest, so an update + // has to be what rebuilds the distro. + const moved = createWslProvider({ + ...options, + resolveImage: async () => ({ url: "https://example.invalid/sandbox/v2.tar.gz", sha256: "b".repeat(64), bytes: 1 }), + }) + await mkdir(join(options.cacheDir, "b".repeat(64)), { recursive: true }) + await writeFile(join(options.cacheDir, "b".repeat(64), "v2.tar.gz"), "x") + await moved.update(config) + + const state = JSON.parse(await Bun.file(join(dir, "state.json")).text()) + expect(state.imports).toBe(2) + expect(state.restored).toBe("home-archive") + // The staging archive is cleaned up, and never lived where discard would + // have deleted it mid-update. + expect(await Bun.file(join(options.dataDir, `.${config.name}.home.tar`)).exists()).toBe(false) + await rm(dir, { recursive: true, force: true }) +}) + +// An update interrupted after the export left the home in that archive and +// nothing else; the retry has to restore it rather than tar out the fresh home +// the failed run had already imported. +test("an update restores the archive an interrupted one left behind", async () => { + const dir = await mkdtemp(join(tmpdir(), "wsl-test-")) + const options = await preparedOptions(dir) + const provider = createWslProvider(options) + await provider.create(config) + await writeFile(join(options.dataDir, `.${config.name}.home.tar`), "the home the first attempt saved") + await provider.update(config) + const state = JSON.parse(await Bun.file(join(dir, "state.json")).text()) + expect(state.restored).toBe("the home the first attempt saved") + await rm(dir, { recursive: true, force: true }) +}) + +test("an update of a sandbox with no home yet simply rebuilds it", async () => { + const dir = await mkdtemp(join(tmpdir(), "wsl-test-")) + const options = await preparedOptions(dir) + const provider = createWslProvider(options) + await provider.create(config) + await writeFile(join(dir, "state.json"), JSON.stringify({ distros: [wslDistroName(config.name)], home: false })) + await provider.update(config) + const state = JSON.parse(await Bun.file(join(dir, "state.json")).text()) + expect(state.restored).toBeUndefined() + expect(state.distros).toEqual([wslDistroName(config.name)]) + await rm(dir, { recursive: true, force: true }) +}) + +test("reports an update against the published image, promising the home survives", async () => { + const dir = await mkdtemp(join(tmpdir(), "wsl-test-")) + let sha = ASSET_SHA + const provider = createWslProvider( + await preparedOptions(dir, { + resolveImage: async () => ({ + url: "https://example.invalid/sandbox/rootfs.tar.gz", + sha256: sha, + bytes: 1, + version: "2.0", + }), + }), + ) + await provider.create(config) + expect(await provider.imageStatus(config)).toEqual({ kind: "current" }) + sha = "c".repeat(64) + expect(await provider.imageStatus(config)).toEqual({ kind: "update", version: "2.0", preservesHome: true }) + await rm(dir, { recursive: true, force: true }) +}) + +// Deleting unregisters the distro and takes the cached tarball with it. The +// prune sits in destroy rather than in the discard() that update() shares: +// pruning between an update's discard and its re-provision would drop an image +// an interrupted earlier attempt had already staged. +test("destroy drops the cached image, and an update keeps it", async () => { + const dir = await mkdtemp(join(tmpdir(), "wsl-test-")) + const options = await preparedOptions(dir) + const provider = createWslProvider(options) + await provider.create(config) + expect(await readdir(options.cacheDir)).toEqual([ASSET_SHA]) + + // An update re-provisions onto the same image here, so the cache is + // untouched — nothing is downloaded twice. + await provider.update(config) + expect(await readdir(options.cacheDir)).toEqual([ASSET_SHA]) + + await provider.destroy(config) + expect(await readdir(options.cacheDir)).toEqual([]) + await rm(dir, { recursive: true, force: true }) +}) diff --git a/packages/codev-sandbox/src/wsl/provider.ts b/packages/codev-sandbox/src/wsl/provider.ts index 4717bb79db..39ad959805 100644 --- a/packages/codev-sandbox/src/wsl/provider.ts +++ b/packages/codev-sandbox/src/wsl/provider.ts @@ -1,7 +1,7 @@ import { spawn } from "node:child_process" import { mkdir, rm, stat, writeFile } from "node:fs/promises" import { release } from "node:os" -import { join } from "node:path" +import { dirname, join } from "node:path" import { SANDBOX_GUEST_LAUNCHER, checkSandboxHealth, @@ -10,7 +10,14 @@ import { sandboxBootEnv, } from "../boot" import { createBrokerServer, type SandboxEgress } from "../broker" -import { localImageAsset, readImageMarker, stageAsset, writeImageMarker } from "../image" +import { + imageUpdateStatus, + localImageAsset, + pruneImageCache, + readImageMarker, + stageAsset, + writeImageMarker, +} from "../image" import { wslFirewallScript } from "./firewall" import type { SandboxAsset } from "../manifest" import { WINDOWS_11_FIRST_BUILD, windowsBuildNumber, windowsSandboxAvailability } from "../policy" @@ -21,7 +28,7 @@ import type { SandboxJobOptions, SandboxProvider, } from "../provider" -import { parseDistroList, reserveLoopbackPort, runWsl, wslEnvSpec } from "./cli" +import { parseDistroList, pipeWsl, reserveLoopbackPort, runWsl, wslEnvSpec } from "./cli" // Guest-side paths. The bundle mount point and the launcher are baked into the // image, so they match the vz and bwrap backends exactly. @@ -33,6 +40,10 @@ const GUEST_BUNDLE = "/opt/codev" // Everything else keeps runWsl's 20s wedge guard. const WSL_IMPORT_TIMEOUT_MS = 15 * 60_000 const WSL_UNREGISTER_TIMEOUT_MS = 5 * 60_000 +// Carrying a home across is a tar of the guest's own disk; generous because a +// home holds a session database and whatever the user put there, and hitting +// the ceiling would cost them the data the update promised to keep. +const WSL_HOME_TRANSFER_TIMEOUT_MS = 10 * 60_000 export type WslProviderOptions = { dataDir: string @@ -112,6 +123,41 @@ export function createWslProvider(options: WslProviderOptions): SandboxProvider await rm(wslSandboxDirs(options.dataDir, config.name).base, { recursive: true, force: true }) } + // Reads the guest's home as a tar on the host. False means there was nothing + // to carry — a sandbox that never booted has no home yet, and a distro that + // is not registered fails the same probe — while a failed archive throws, so + // an update never destroys a distro whose home it could not save. + const exportHome = async (distro: string, archive: string) => { + const present = await wsl(["-d", distro, "-u", "root", "--", "test", "-d", GUEST_HOME]) + if (present.code !== 0) return false + await mkdir(dirname(archive), { recursive: true }) + const exported = await pipeWsl(["-d", distro, "-u", "root", "--", "tar", "-C", "/home", "-cf", "-", "codev"], { + stdoutFile: archive, + wslCommand: options.wslCommand, + spawnImpl, + timeoutMs: WSL_HOME_TRANSFER_TIMEOUT_MS, + }) + if (exported.code !== 0) + throw new Error(`could not save the sandbox home before updating: ${exported.stderr.trim() || "tar failed"}`) + return true + } + + // Unpacks over the fresh distro's own home, so files the user made win and + // anything only the new image ships survives alongside them. Run as root, + // which is what restores the guest user's ownership from the archive. + const importHome = async (distro: string, archive: string) => { + const imported = await pipeWsl(["-d", distro, "-u", "root", "--", "tar", "-C", "/home", "-xf", "-"], { + stdinFile: archive, + wslCommand: options.wslCommand, + spawnImpl, + timeoutMs: WSL_HOME_TRANSFER_TIMEOUT_MS, + }) + if (imported.code !== 0) + throw new Error(`could not restore the sandbox home after updating: ${imported.stderr.trim() || "tar failed"}`) + } + + const staged = async (archive: string) => !!(await stat(archive).catch(() => null)) + const prepare = async (config: SandboxConfig, opts?: SandboxJobOptions) => { const distro = wslDistroName(config.name) const asset = config.imagePath @@ -163,6 +209,7 @@ export function createWslProvider(options: WslProviderOptions): SandboxProvider ) await wsl(["--terminate", distro]) await writeImageMarker(dirs.base, asset) + await pruneImageCache(options.dataDir, options.cacheDir, [asset.sha256]) } return { @@ -352,6 +399,11 @@ export function createWslProvider(options: WslProviderOptions): SandboxProvider async destroy(config) { await discard(config) + // Deliberately here rather than inside discard(), which update() also + // calls: pruning between an update's discard and its re-provision would + // drop an image an interrupted earlier attempt had already staged, and + // charge the user another download for it. + await pruneImageCache(options.dataDir, options.cacheDir, []) }, async reset(config, opts) { @@ -359,6 +411,39 @@ export function createWslProvider(options: WslProviderOptions): SandboxProvider await prepare(config, opts) }, + async update(config, opts) { + const distro = wslDistroName(config.name) + // Outside the sandbox's own directory, which discard() deletes whole. + const archive = join(options.dataDir, `.${config.name}.home.tar`) + // A distro cannot be imported over itself, so the home has to leave + // before the old one is unregistered. Export first and fail there if it + // fails: at that point nothing has been destroyed yet, which is the + // whole reason for doing it in this order. + // + // An archive already sitting there belongs to an update that was + // interrupted after the export, and holds the home this sandbox is still + // trying to keep. The current distro's home cannot be worth more — it is + // either gone or the fresh one that same update imported — so it must not + // overwrite it. + const carried = (await staged(archive)) || (await exportHome(distro, archive)) + await discard(config) + await prepare(config, opts) + if (carried) await importHome(distro, archive) + await rm(archive, { force: true }) + }, + + imageStatus(config, opts) { + return imageUpdateStatus({ + dir: wslSandboxDirs(options.dataDir, config.name).base, + config, + resolveImage: options.resolveImage, + // The home is tarred out to the host before the distro goes, so no + // image is required to cooperate. + preservesHome: () => true, + signal: opts?.signal, + }) + }, + // Hot-applies the mount list to the running distro. All wsl.exe // invocations share the distro's mount namespace and the drives are // already automounted, so translation is the only work. Throws on an diff --git a/packages/desktop/src/main/sandbox/controller.ts b/packages/desktop/src/main/sandbox/controller.ts index 83c5ffa811..958804b55d 100644 --- a/packages/desktop/src/main/sandbox/controller.ts +++ b/packages/desktop/src/main/sandbox/controller.ts @@ -13,7 +13,14 @@ import { requireImageAsset, sandboxResourcesValid, } from "@codev/sandbox" -import type { SandboxArch, SandboxAvailability, SandboxBackend, SandboxConfig, SandboxProvider } from "@codev/sandbox" +import type { + SandboxArch, + SandboxAvailability, + SandboxBackend, + SandboxConfig, + SandboxManifest, + SandboxProvider, +} from "@codev/sandbox" import { getStore } from "../store" import { SANDBOXES_KEY } from "../store-keys" import { serverBundleDir } from "./bundle" @@ -38,6 +45,18 @@ export function createDesktopSandboxController(logger: ControllerLogger) { }) return { ...controller, + async initialize() { + await controller.initialize() + // A published image changes rarely and the app can stay open for days, + // so: one sweep once the launch has settled, then a slow heartbeat. + // Checking costs one manifest fetch and downloads nothing — taking the + // update is the user's call — and a machine that cannot reach the + // storage host simply keeps whatever verdict it had. + const sweep = () => + void controller.checkImageUpdates().catch((error) => logger.error("sandbox image check failed", error)) + setTimeout(sweep, SANDBOX_IMAGE_CHECK_DELAY_MS).unref() + setInterval(sweep, SANDBOX_IMAGE_CHECK_INTERVAL_MS).unref() + }, // The backend every new sandbox on this machine gets; null means the // platform has no provider and add must be rejected up front. backend: provider?.backend ?? null, @@ -59,7 +78,7 @@ function platformProvider(): SandboxProvider | undefined { cacheDir: join(app.getPath("userData"), "sandbox-images"), bundleDir: serverBundleDir(), resolveImage: async (opts?: { signal?: AbortSignal }) => { - const manifest = await fetchSandboxManifest(SANDBOX_MANIFEST_URL, { signal: opts?.signal }) + const manifest = await cachedManifest(opts?.signal) return requireImageAsset(manifest, imageBackend, arch) }, resolveEgress: resolveSandboxEgress, @@ -74,6 +93,35 @@ function platformProvider(): SandboxProvider | undefined { return undefined } +const SANDBOX_IMAGE_CHECK_DELAY_MS = 30_000 +const SANDBOX_IMAGE_CHECK_INTERVAL_MS = 6 * 60 * 60_000 + +// One update check asks every sandbox, and each of them resolves the image +// through here — on a machine with three sandboxes that is one manifest, not +// three. The window is short enough that a create still sees a publish that +// landed minutes ago, and staleness is harmless anyway: images are +// content-addressed, so an older manifest names an older image rather than a +// wrong one. +const MANIFEST_CACHE_MS = 60_000 + +function cachedManifest(signal?: AbortSignal) { + // Only the signal-less fetches an update check makes are shared: a create or + // update passes its own abort signal, and a cancelled one must not be the + // fetch every other caller is waiting on. + if (signal) return fetchSandboxManifest(SANDBOX_MANIFEST_URL, { signal }) + const now = Date.now() + if (manifestCache && now - manifestCache.at < MANIFEST_CACHE_MS) return manifestCache.manifest + const manifest = fetchSandboxManifest(SANDBOX_MANIFEST_URL) + manifestCache = { at: now, manifest } + // A failed fetch must not be served to the next caller for a minute. + void manifest.catch(() => { + if (manifestCache?.manifest === manifest) manifestCache = null + }) + return manifest +} + +let manifestCache: { at: number; manifest: Promise } | null = null + // The Windows build the published WSL installer needs, for the availability // warning. Availability is asked on every launch and its verdict is cached for // the app's lifetime, so this is bounded and cached too — and only successes diff --git a/packages/desktop/src/main/sandbox/ipc.ts b/packages/desktop/src/main/sandbox/ipc.ts index b328bac35f..88218dccea 100644 --- a/packages/desktop/src/main/sandbox/ipc.ts +++ b/packages/desktop/src/main/sandbox/ipc.ts @@ -168,6 +168,12 @@ export function registerSandboxIpcHandlers(controller: DesktopSandboxController) ipcMain.handle("sandbox-reset", (_event: IpcMainInvokeEvent, id: unknown) => controller.resetSandbox(requireSandboxIpcString("sandbox id", id)), ) + ipcMain.handle("sandbox-update", (_event: IpcMainInvokeEvent, id: unknown) => + controller.updateSandbox(requireSandboxIpcString("sandbox id", id)), + ) + // The scheduled sweep answers on its own; this is the renderer asking for a + // fresh verdict at the moment the user is looking at one. + ipcMain.handle("sandbox-check-updates", () => controller.checkImageUpdates()) ipcMain.handle("sandbox-start", (_event: IpcMainInvokeEvent, id: unknown) => controller.startSandbox(requireSandboxIpcString("sandbox id", id)), ) diff --git a/packages/desktop/src/preload/index.ts b/packages/desktop/src/preload/index.ts index 6b35ed4bcb..a5f19118a3 100644 --- a/packages/desktop/src/preload/index.ts +++ b/packages/desktop/src/preload/index.ts @@ -68,6 +68,8 @@ const api: ElectronAPI = { cancelDefault: () => ipcRenderer.invoke("sandbox-cancel-default"), remove: (id) => ipcRenderer.invoke("sandbox-remove", id), reset: (id) => ipcRenderer.invoke("sandbox-reset", id), + update: (id) => ipcRenderer.invoke("sandbox-update", id), + checkUpdates: () => ipcRenderer.invoke("sandbox-check-updates"), start: (id) => ipcRenderer.invoke("sandbox-start", id), stop: (id) => ipcRenderer.invoke("sandbox-stop", id), updateMounts: (id, mounts) => ipcRenderer.invoke("sandbox-update-mounts", id, mounts),