Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/sandbox.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Sandbox

# The runner sandbox's boundary test (`just runner-sandbox`) runs nightly on a
# GitHub-hosted runner, outside pull requests and the merge queue, so landing
# never waits on it. GitHub reports a failed scheduled run only to the account
# that last changed the cron schedule (with Actions notifications on), not to
# the repository's watchers; that is the maintainer, whose account lands the
# workflow, so they see a boundary failure.
on:
schedule:
- cron: "23 7 * * *"
workflow_dispatch:

permissions:
contents: read

concurrency:
group: sandbox-${{ github.ref }}
cancel-in-progress: true

jobs:
runner-sandbox:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: DeterminateSystems/nix-installer-action@1d87d45818068401a10cf16bdc5f00b24994a83f
with:
determinate: false
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install dependencies
run: nix develop --command just install
# The test runs bubblewrap from the Nix store, which Ubuntu's AppArmor would
# refuse the unprivileged user namespace it needs.
- name: Allow unprivileged user namespaces
run: |
if [ -e /proc/sys/kernel/apparmor_restrict_unprivileged_userns ]; then
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
fi
- name: Check the runner sandbox's boundary
run: nix develop --command just runner-sandbox
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1194,7 +1194,11 @@ language servers and formatters are off. The CLI itself, Claude Code or
commands start from a minimal environment, so its login, the opencode server's
password, and the kix tools' token stay out of their reach. This is an OS
sandbox, not a VM, and it narrows rather than removes the risk of running
agent commands on your machine.
agent commands on your machine. On Linux, `just runner-sandbox` in this
repository runs a scripted model through the pinned opencode and
sandbox-runtime in a checkout that tries to read credentials, write outside its
sandbox, plant opencode config, and reach the network, and fails if any attempt
gets through. CI runs it nightly, outside the merge queue.

On Linux the sandbox needs `bwrap` and `socat` (the flake's `runner` package
pins both) and unprivileged user namespaces, including the nested one Claude
Expand Down
7 changes: 7 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,13 @@
# `flock` (the land daemon's repository lock) and `script` (the tests' terminal), which
# self-hosted NixOS runners don't otherwise provide.
++ pkgs.lib.optionals pkgs.stdenv.hostPlatform.isLinux [ pkgs.util-linux ];
# The runner package's opencode and sandbox-runtime, for the sandbox boundary test
# (test/live/runner-sandbox.test.ts), which runs where srt does. Named as the runner
# finds them rather than put on PATH, where they would shadow the user's own opencode.
env = pkgs.lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux {
QUEST_RUNNER_OPENCODE = pkgs.lib.getExe opencode;
QUEST_RUNNER_SRT = pkgs.lib.getExe pkgs.sandbox-runtime;
};
};
formatter = pkgs.nixfmt;
checks = {
Expand Down
8 changes: 7 additions & 1 deletion justfile
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,13 @@ _test:
# Codex and Claude Code CLIs. Uses each CLI's login and spends its usage; run it
# after upgrading either CLI. Arguments go to vitest, such as `-t codex`.
review-sandbox *args:
QUEST_LIVE=1 bun run vitest run {{args}}
QUEST_LIVE=1 bun run vitest run test/live/review-sandbox {{args}}

# Check that a runner job's commands cannot escape its sandbox, with the pinned
# opencode and sandbox-runtime and a scripted model, on Linux. Needs no login;
# CI runs it nightly (.github/workflows/sandbox.yml). Run it after bumping either.
runner-sandbox *args:
QUEST_LIVE=1 bun run vitest run test/live/runner-sandbox {{args}}

# Apply formatters without changing program behavior.
fix:
Expand Down
9 changes: 8 additions & 1 deletion quest/m0/kix/sandbox/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@ Shipped in [moq-dev/quest#212](https://github.com/moq-dev/quest/pull/212)
opencode 1.19 still imports the plugins a checkout's own `opencode.json`
or `.opencode/` names, even with project config disabled, so the runner
keeps the agent from creating them and refuses a checkout that holds them.
`test/live/runner-sandbox.test.ts` checks that boundary with the pinned
opencode and srt and a scripted model. It runs in `just runner-sandbox`
and a nightly GitHub-hosted workflow (`.github/workflows/sandbox.yml`),
not in `just test` or the merge queue, whose container can't mount
`/proc` for bwrap, so a bump that weakens the boundary fails the next
nightly run. Decided 2026-11-03 (user, /quest-complete), after the merge
queue dropped [moq-dev/quest#404](https://github.com/moq-dev/quest/pull/404):
its own CI, nightly, so landing never blocks on it.
- Decided 2026-10-24 (user) in `/quest-plan`: split the remaining work into
the children below, ranked so the earliest unblock the most. Dropped
(user): read-only registry access, which Claude Code's sandbox cannot
Expand All @@ -78,6 +86,5 @@ Shipped in [moq-dev/quest#212](https://github.com/moq-dev/quest/pull/212)
## Required

- [No Unix sockets in opencode jobs](/quest/m0/kix/sandbox/seccomp.md) - srt's seccomp helper is packaged, so commands can't reach the Nix daemon, and the runner refuses to start without it
- [Boundary test in CI](/quest/m0/kix/sandbox/live-test.md) - the opencode sandbox's hostile-repository test runs on Linux CI with a scripted model
- [Trust rechecked on queue](/quest/m0/kix/sandbox/trust-on-queue.md) - a job's head trust is reread whenever it queues, so the approval decision shows without a free runner
- [Claude Code boundary test](/quest/m0/kix/sandbox/claude-boundary.md) - the same hostile-repository test for Claude Code jobs, with a fake model set in the account's settings
26 changes: 17 additions & 9 deletions quest/m0/kix/sandbox/claude-boundary.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# [M] The Claude Code sandbox's boundary test runs in CI
# [M] The Claude Code sandbox's boundary test runs nightly in CI

## Goal

Claude Code jobs get the same checked-in hostile-repository boundary test
that opencode jobs got in
[moq-dev/quest#404](https://github.com/moq-dev/quest/pull/404): a scripted
fake model, set as the account's endpoint in its home, drives the pinned
Claude Code through the runner's launch path on Linux CI, and every escape
attempt is refused.
A Claude Code bump that weakens the boundary fails CI before it lands.
Claude Code through the runner's launch path in the nightly sandbox workflow
on Linux, and every escape attempt is refused.
A Claude Code bump that weakens the boundary fails the next nightly run.

## Plan

Expand All @@ -28,7 +28,8 @@ Evidence, gathered 2026-11-02:

Work:

- Reuse #404's attempts and layout: denied reads by path and symlink,
- Reuse the attempts and layout of opencode's test,
`test/live/runner-sandbox.test.ts`: denied reads by path and symlink,
writes outside the checkout and job temp directory, planted project
settings and hooks, the network directly and through the proxy, and the
Unix-socket probe from the seccomp quest.
Expand All @@ -37,7 +38,7 @@ Work:
`TMPDIR=<CLAUDE_CODE_TMPDIR or os.tmpdir()>/claude-<uid>`, which the
runner's `TMPDIR` (`src/runner/claude.ts`) puts inside `allowWrite`; the
test confirms it instead of trusting the read.
- Point the account at the fake model in its home, as #404's `signIn`
- Point the account at the fake model in its home, as that test's `signIn`
configures opencode's provider: `ANTHROPIC_BASE_URL` in the `env` of
`$CLAUDE_CONFIG_DIR/settings.json`, which `--setting-sources user` loads.
An ambient `ANTHROPIC_BASE_URL` never reaches Claude Code, since
Expand All @@ -46,10 +47,17 @@ Work:
settings' `env` also reaches the commands, the fake key there is no
secret, so the unreadable-secret attempt reads a planted credential file
in the home instead.
- Like #404, each shell attempt first succeeds outside the sandbox, and the
test fails rather than skips on Linux when the sandbox can't start.
- Like opencode's test, each shell attempt first succeeds outside the
sandbox, and the test fails rather than skips on Linux when the sandbox
can't start.
- Run it where opencode's test runs: name it
`test/live/runner-sandbox-claude.test.ts` (or extend
`runner-sandbox.test.ts`), so `just runner-sandbox` and the nightly
`.github/workflows/sandbox.yml` pick it up, and keep it out of
`just test` and the merge queue. Decided 2026-11-03 (user,
/quest-complete) for opencode's test: its own CI, nightly, so landing
never blocks on it.

## Required

- [No Unix sockets in opencode jobs](/quest/m0/kix/sandbox/seccomp.md) - the Unix-socket probe this test reuses
- [Boundary test in CI](/quest/m0/kix/sandbox/live-test.md) - #404's attempts and layout this test reuses
33 changes: 0 additions & 33 deletions quest/m0/kix/sandbox/live-test.md

This file was deleted.

11 changes: 5 additions & 6 deletions quest/m0/opencode-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,11 @@ What the change does:
- Recheck each of the sandbox's rules on 2.x: the `shell` wrapper through
srt, the commands' environment allowlist (`keptEnvironment`),
`OPENCODE_DISABLE_PROJECT_CONFIG`, `external_directory` and `webfetch`
denies, the plugin's realpath check, and `lsp`/`formatter` off. Rerun the
sandbox's hostile-repository boundary test against 2.x.
denies, the plugin's realpath check, and `lsp`/`formatter` off. The
sandbox's boundary test, `test/live/runner-sandbox.test.ts`, runs the
pinned opencode only in `just runner-sandbox` and nightly CI, so run it
on the 2.x pin before landing; extend it to any rule 2.x changes, such as
its tool names or plugin hooks.
- Run the runner's live opencode checks that the
[Kix decision UI](/quest/m0/kix/README.md) rehearsal lists, if a runner is
available.

## Required

- [Boundary test in CI](/quest/m0/kix/sandbox/live-test.md) - the boundary test this quest reruns on 2.x is checked in and runs in CI
12 changes: 8 additions & 4 deletions src/runner/opencode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,12 +199,16 @@ export default {
* environment, which holds the server password, the tools' token in
* `OPENCODE_CONFIG_CONTENT`, and any provider key opencode exported, then
* sources the private HOME and the devShell inside the sandbox before the
* command, as Claude Code does with `CLAUDE_ENV_FILE`.
* command, as Claude Code does with `CLAUDE_ENV_FILE`. srt replaces the
* command's TMPDIR with its own `CLAUDE_CODE_TMPDIR`, or `/tmp/claude`
* without it, so that names the attempt's directory too.
*/
export function shellScript(srt: string, settings: string, sandbox: Sandbox, env: NodeJS.ProcessEnv): string {
const kept = Object.entries({ ...keptEnvironment(env), TMPDIR: sandbox.tmp }).map(([name, value]) =>
quote(`${name}=${value}`),
);
const kept = Object.entries({
...keptEnvironment(env),
TMPDIR: sandbox.tmp,
CLAUDE_CODE_TMPDIR: sandbox.tmp,
}).map(([name, value]) => quote(`${name}=${value}`));
const inner = `. ${quote(sandbox.env)} && quest_command=$QUEST_COMMAND && unset QUEST_COMMAND && eval "$quest_command"`;
return `#!/bin/sh
if [ "$#" -ne 2 ] || [ "$1" != -c ]; then
Expand Down
Loading
Loading