feat(worktree): supervise runs in systemd scopes and release automatically

.github/workflows/check.yml

@@ -29,6 +29,12 @@ jobs:
           determinate: false
       - name: Install dependencies
         run: nix develop --command just install
+      # `quest worktree run` owns processes through a systemd user scope, which the tests exercise.
+      - name: Start a systemd user manager
+        run: |
+          sudo loginctl enable-linger "$USER"
+          sudo systemctl start "user@$(id -u).service"
+          echo "XDG_RUNTIME_DIR=/run/user/$(id -u)" >> "$GITHUB_ENV"
       - name: Check
         run: nix develop --command just check
       - name: Test

README.md

@@ -404,9 +404,7 @@ guarantee or timeout: a command waits as long as the host stays busy.
 The experimental `quest worktree` commands share one disk budget across this
 user's repositories and release a checkout's disk by deleting its declared
 scratch in place. They only operate on Git worktrees they create; use the
-owning application for Codex, Claude, or other managed checkouts. Process
-supervision remains unfinished: stop agents and commands in a checkout before
-releasing it.
+owning application for Codex, Claude, or other managed checkouts.
 
 Set a shared capacity and an explicit reservation in bytes, covering the
 checkout and its expected build output. Measure representative jobs before
@@ -416,30 +414,53 @@ writers, and Git objects stored in the shared repository are not charged.
 
 ```sh
 quest worktree capacity 20000000000
-quest worktree create example --reserve 3000000000 --ref HEAD
+quest worktree create example --reserve 3000000000 --scratch node_modules dist
+quest worktree run example -- sh -c 'just install && just test'
 quest worktree status
-quest worktree release example --scratch node_modules dist
-quest worktree restore example
 ```
 
 Creation prints JSON with the ID and the detached, locked checkout path under
-`.worktrees/`. Release deletes only its `--scratch` paths, and refuses one that
-Git does not ignore, that holds tracked files or another Git worktree, or that
-is reached through a symbolic link; links inside scratch are removed, never
-followed. Missing paths
-are skipped, so release can be rerun. The checkout, its Git state, and
-untracked files stay in place, and `status` charges what remains. Restore
-reserves the checkout's disk again; then reinstall or rebuild its scratch, for
-example with `just install`.
+`.worktrees/`, and records the `--scratch` paths. `run` restores a released
+checkout, runs the command there, and once it exits kills any descendants it
+left behind. When no other run remains in the checkout, it releases it: the
+declared scratch is deleted and the checkout keeps only its source. Put a whole
+job, such as an agent session or a script that installs, builds, and tests, in
+one run, since the next run starts without scratch.
+
+`run` owns processes through a systemd user scope per run, so it needs Linux
+5.14+ with a systemd 254+ user manager and unified cgroup v2; it fails on other
+hosts.
+Work handed to another service manager, such as `systemd-run` or a container
+daemon, escapes the scope. If the supervisor itself is killed, its command keeps
+running and its scratch is kept; the next `quest worktree` command drops the run
+once its scope is empty and then releases the checkout. `status` lists each
+run's scope, which `systemctl --user stop` ends.
+
+Release deletes only declared scratch, and refuses a path that Git does not
+ignore, that holds tracked files or another Git worktree, or that is reached
+through a symbolic link; links at or inside scratch are removed, never followed.
+Missing paths are skipped, so release can be rerun. The checkout, its Git state,
+and untracked files stay in place, and `status` charges what remains. A failed
+automatic release is recorded on the checkout, which stays reserved. To manage a
+checkout by hand, stop its unsupervised work first:
+
+```sh
+quest worktree release example
+quest worktree restore example
+```
+
+`release` refuses while supervised runs are live. `restore` reserves the
+checkout's disk again; then reinstall or rebuild its scratch, for example with
+`just install`.
 
 Quest never removes a checkout. Once one is finished, remove it with
 `git worktree unlock` and `git worktree remove`, which refuses modified or
 untracked files but deletes ignored ones; `release` then drops its record. A
 failed creation stays charged until `release` drops it. Records live in
 `$QUEST_WORKTREE_HOME`, or `$XDG_STATE_HOME/quest/worktrees` (default
-`~/.local/state/quest/worktrees`). No live lock is stolen: after a crash,
-confirm the registry lock's owner has exited before removing it. This initial
-interface refuses unknown or changed checkout ownership.
+`~/.local/state/quest/worktrees`). The registry lock of a process that exited on
+this host is reclaimed; a live lock, or one held from another host, is never
+stolen. This initial interface refuses unknown or changed checkout ownership.
 
 ## Work with your agents
 

quest/m0/README.md

@@ -35,7 +35,7 @@ ownership-manifest design:
 
 ## Required
 
-- [Worktree process ownership](/quest/m0/worktree.md) - supervise checkout commands and release checkouts automatically
+- [Worktree removal](/quest/m0/worktree.md) - remove finished checkouts without losing work
 - [Agent lifecycle integration](/quest/m0/worktree-agents.md) - reclaim completed and suspended checkouts through their harness
 - [Agent workflow helpers](/quest/m0/helpers/README.md) - CLI commands for CI waits, PR status, claims, quest moves, and serialized merges
 

quest/m0/kix/runner.md

@@ -87,6 +87,6 @@ replacing the hosted Agent SDK runtime:
   preference-order control. Decided 2026-10-18 as a follow-up of the
   decision inbox (#144), folded here rather than its own quest.
 
-## Required
+## Related
 
-- [Worktree process ownership](/quest/m0/worktree.md) - reuse checkout supervision, scratch release, and disk admission
+- [Worktree removal](/quest/m0/worktree.md) - removes finished checkouts; reuse `quest worktree run` for supervision, scratch release, and disk admission

quest/m0/worktree-agents.md

@@ -37,10 +37,7 @@ merge.
   [Claude removal hooks](https://code.claude.com/docs/en/hooks#worktreeremove),
   [Codex worktrees](https://learn.chatgpt.com/docs/environments/git-worktrees).
 
-## Required
-
-- [Worktree process ownership](/quest/m0/worktree.md) - supervised commands and automatic scratch release
-
 ## Related
 
+- [Worktree removal](/quest/m0/worktree.md) - removes finished checkouts; `quest worktree run` already supervises jobs and releases scratch
 - [Own-PC runner](/quest/m0/kix/runner.md) - reuse lifecycle ownership in the hosted job adapter

quest/m0/worktree.md

@@ -1,49 +1,48 @@
-# [M] Own worktree processes and release checkouts automatically
+# [M] Remove finished quest worktree checkouts
 
 ## Goal
 
-`quest worktree` supervises the commands that run in a checkout it created,
-then releases that checkout automatically once they stop by pruning its
-declared scratch in place. A crash or cancellation is reconciled without losing
-work, stealing a live lock, or deleting scratch a running process still uses.
+`quest worktree remove` deletes a finished checkout that `quest worktree` created
+and drops its record, without losing work: it refuses while supervised runs are
+live and refuses a checkout that still holds anything not already preserved.
+`quest worktree stop` ends a crashed supervisor's leftover processes.
 
 ## Plan
 
 - Landed in [kixelated/quest#127](https://github.com/kixelated/quest/pull/127):
-  manual disk capacity and reservations, `create`, `release` (delete declared
-  scratch in place), and `restore` (reserve again; the caller rebuilds the
-  scratch) of Git-owned checkouts. The README section "Prunable local
-  checkouts" documents it as experimental and tells users to stop all work
-  before releasing.
-- Decided (user, 2026-10-21): prune in place. Release deletes only declared
-  scratch that Git ignores, that holds no tracked files or other Git worktree,
-  and that is not reached through a symlink; it never removes the checkout or
-  its Git state.
-  Checkout-independent recovery is out of scope: four review rounds of the
-  checkpoint-and-remove design kept finding data-loss edge cases, while the
-  disk cost is scratch (2026-10-15 on this repository: 1.6 MB fresh source,
-  376 MB built).
-- Decided 2026-10-21 (user): this quest brings its own process-ownership
-  backend; `quest gate` admits on CPU load and memory only and owns no
-  descendants. The Linux systemd scope backend (scopes, slot handshake,
-  `cgroup.kill` inode pinning, prerequisite probes, opt-in systemd tests) was
-  dropped from the gate and is restorable from commit 5684307 in
-  [kixelated/quest#128](https://github.com/kixelated/quest/pull/128). Process
-  groups and PID files cannot prove detached descendants stopped after a
-  supervisor crash, so hosts without a supported backend fail explicitly
-  instead of guessing.
-- Own foreground child processes and their descendants until they stop; only
-  then prune scratch and release the reservation. Record ownership so a crash
-  can be reconciled, including a stale registry lock whose owner has exited.
-  Report failed cleanup as a retained allocation, not success.
-- Release automatically at lifecycle boundaries: before a blocking handoff and
-  at completion, once meaningful edits are committed. An asynchronous question
-  with independent work still running is not a release boundary. Automatic
-  release needs the scratch declarations without a caller; recording them at
-  `create` is the expected route.
-- Add cancellation and crash fixtures: interrupt a supervised command with live
-  descendants, kill the supervisor, and reconcile. Measure retained disk after
-  complete and suspended jobs.
+  disk capacity and reservations, `create`, `release` (delete declared scratch
+  in place), and `restore`. Landed with this quest's first slice: scratch
+  declared at `create`, and `run`, which supervises a command in a systemd user
+  scope per run, kills leftover descendants once the command exits, and prunes
+  scratch when the checkout's last run stops. Crashed supervisors are
+  reconciled by the next `quest worktree` command once their scope is empty,
+  and a dead process's registry lock is reclaimed.
+- Decided (user, 2026-10-21): prune in place. Release never removes the
+  checkout or its Git state. Checkout-independent recovery is out of scope:
+  four review rounds of the checkpoint-and-remove design kept finding
+  data-loss edge cases.
+- Decided 2026-10-21 (user): process ownership is a systemd user scope, on
+  Linux only; other hosts fail explicitly. Process groups and PID files cannot
+  prove detached descendants stopped after a supervisor crash.
+- Remaining (maintainer, 2026-10-21, from
+  [kixelated/quest#127](https://github.com/kixelated/quest/pull/127)):
+  `quest worktree remove` for finished checkouts. Never force: refuse modified,
+  untracked, or undeclared ignored files, and commits no branch or remote ref
+  reaches. Treat a squash-merged branch as finished only with exact PR-head
+  evidence from the forge, behind an interface that keeps the core
+  forge-independent.
+- Remaining (from this quest's first slice): `quest worktree stop <id>` ends a
+  crashed supervisor's leftover processes through its recorded scopes, without
+  calling `systemctl --user stop` directly. A stale inspection-failure reason
+  on a checkout is cleared by the next successful inspection, rather than
+  lingering until the next successful release.
+- Out of scope: terminal Ctrl-C reaches the command twice (once from the
+  terminal, once forwarded by the supervisor). It is shared with `quest gate`
+  and planned separately in
+  [kixelated/quest#180](https://github.com/kixelated/quest/pull/180).
+- Measured 2026-10-21 on this repository: a fresh checkout is 2.9 MB, 394 MB
+  after installing dependencies and building, and 2.9 MB once `run` released
+  it. A suspended job keeps the same released footprint until it is restored.
 - Native agent lifecycle adoption stays in the agent integration quest.
 
 ## Related

src/cli/main.ts

@@ -371,14 +371,52 @@ export function main(argv: string[], out: Output): number | Promise<number> {
 		.description("Create an exclusively owned detached Git worktree under .worktrees/")
 		.requiredOption("--reserve <bytes>", "Checkout and expected build growth in bytes", positiveInteger("bytes"))
 		.option("--ref <ref>", "Initial Git revision", "HEAD")
-		.action((name: string, options: { reserve: number; ref: string }) =>
-			record(state().create(root(), name, options.reserve, options.ref)),
+		.option(
+			"--scratch <path...>",
+			"Ignored paths release deletes, such as dependencies and build output; they must hold no tracked files",
+			[],
+		)
+		.action((name: string, options: { reserve: number; ref: string; scratch: string[] }) =>
+			record(state().create(root(), name, options.reserve, options.ref, options.scratch)),
 		);
+	worktree
+		.command("run")
+		.summary("Run a command in a checkout, then release it once every process stops")
+		.description(
+			"Run a command in the checkout, restoring it first when released. The command and its descendants " +
+				"run in a new systemd user scope; once the command exits, descendants still in the scope are killed. " +
+				"When no other supervised run remains, the scratch declared at creation is pruned. Put the whole job, " +
+				"such as an agent session or a script that installs, builds, and tests, in one run.\n\n" +
+				"A supervisor that crashes leaves its run recorded; the next worktree command drops it once its scope " +
+				"is empty, and releases the checkout. Needs Linux 5.14+ with a systemd 254+ user manager and unified cgroup " +
+				"v2, and fails on other hosts. Exits with the command's status; a release that fails is reported, leaves " +
+				"the checkout reserved, and turns success into exit 1.",
+		)
+		.argument("<id>", "Checkout ID or name")
+		.argument("<command>", "Executable to run; put -- before it when any argument starts with -")
+		.argument("[args...]", "Arguments forwarded verbatim")
+		.action((id: string, command: string, args: string[]) => {
+			pending = state()
+				.run(id, [command, ...args])
+				.then(
+					({ code, failure }) => {
+						if (!failure) return code;
+						warn(failure);
+						return code || 1;
+					},
+					(error: unknown) => {
+						warn(error instanceof Error ? error.message : String(error));
+						return 1;
+					},
+				);
+		});
 	worktree
 		.command("release <id>")
-		.description("Delete declared scratch and keep the checkout; drop the record of a removed checkout")
-		.option("--scratch <path...>", "Ignored paths without tracked files, such as dependencies and build output", [])
-		.action((id: string, options: { scratch: string[] }) => record(state().release(id, options.scratch)));
+		.description(
+			"Delete the scratch declared at creation and keep the checkout; refuses while supervised runs are live, " +
+				"and drops the record of a removed checkout",
+		)
+		.action((id: string) => record(state().release(id)));
 	worktree
 		.command("restore <id>")
 		.description("Reserve a released checkout's disk again before rebuilding its scratch")