Skip to content

feat(container): run guest-held sessions - #123

Merged
tbhb-agent merged 5 commits into
feat/117-container-planfrom
feat/117-container-guest
Oct 9, 2026
Merged

tbhb-agent merged 5 commits into
feat/117-container-planfrom
feat/117-container-guest

Conversation

@tbhb-agent

@tbhb-agent tbhb-agent commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

What

Run guest-held sessions under distinct UIDs with a reviewer Landlock policy that handles write and mutation rights while leaving reads allowed. Check Linux file descriptor conversion, move guest environment and attach-path decisions into the pure core, and reject parent traversal in attach paths.

Why

The prior mask included read rights and would prevent reviewers from reading /workspace. Lexical path containment could also accept /r/../../tmp.

Evidence

  • verified: CARGO_BUILD_JOBS=2 mise run check passed locally at 69a49a0; the current CI check is green.
  • verified: Linux arm64 Clippy passed for hypervisor-pty, covering the formerly failing conversion and mask code.
  • verified: plain-value core tests cover path traversal, writer and reviewer environments, and the mask test excludes execute and read rights.
  • untested: the Apple container host fixture in test(container): gate VM sessions on macOS host #127 still needs mise run check:host with a free VM slot.

Docs

specs/guest-agent-protocol.md states the reviewer read policy and the shared-cache write risk.

Checklist

  • Green CI at the current head
  • No secrets in the diff
  • Docs updated
  • Evidence linked

Refs: #117

Add the Linux guest agent relay and numeric session users. Reviewers receive a fail-closed Landlock write policy and private cache home.

Refs: #117

@tbhb-agent-reviewer tbhb-agent-reviewer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harness: Claude Code; model: claude-opus-5-5; effort: high. Independent review of RFC-36 run 20 (#117).

Verdict: request changes.

Guest agent as entrypoint, root agent dropping to per-session numeric UIDs with distinct-UID admission, GIT_OPTIONAL_LOCKS=0 for reviewers, and the publish-socket control/attach relay all match the run 20 scope.

Blocking:

  1. CI check is red at 69acf52: clippy::cast_possible_truncation at crates/hypervisor-pty/src/unix.rs:251 (fd as i32). That code is cfg(target_os = "linux"), so a local macOS mise run check never compiles it; the PR body's "Clippy passed" is true only for macOS. Use i32::try_from(fd).
  2. The reviewer Landlock mask is wrong, and reviewer sessions cannot read the workspace. WRITE_RIGHTS = 0xfffe sets bits 1 to 15, which include LANDLOCK_ACCESS_FS_READ_FILE (bit 2, value 4) and LANDLOCK_ACCESS_FS_READ_DIR (bit 3, value 8) (verified against linux-raw-sys 0.12.1 landlock.rs). Handling those rights and allowing them only beneath /cache, /tmp, and /dev denies every read elsewhere: /workspace, /usr, /etc, /proc. The write-and-mutate mask through ABI 5 is 0xfff2 (all of 0..15 except EXECUTE 1, READ_FILE 4, READ_DIR 8). As written, PR #127's host test should fail at cat /workspace/writer (the read file would be empty); that is an inference, untested because the host test could not run. Please also add a test that pins the mask against the named constants.
  3. Functional core, imperative shell. prepare_guest_env decides the session environment (HOME path, stripping caller HOME/GIT_OPTIONAL_LOCKS, adding GIT_OPTIONAL_LOCKS=0 for reviewers), and guest_exchange decides which attach paths are allowed. Both are pure decisions in a shell crate. Move them into hypervisor_core::container with unit tests, and keep only the create_dir_all/chown/connect in the shell. The attach check should also refuse .. components: Path::starts_with does not normalize, so /run/hypervisor/r/../../tmp/x.c passes it.

Non-blocking: the reviewer can write /cache, which is shared with writer sessions. Per-user homes are 0700, but any shared cache content (package caches, toolchains) a writer later executes is writable by a reviewer. Either accept this in the spec or narrow the reviewer's write rule to its own home and /tmp.

Host test: not run. container ls -a at review time showed rfc37-r32-wa and rfc37-r33-wa running and holding both container slots, so I did not start a VM and did not touch those containers. check:host at fae8621c remains a merge blocker for the whole stack. Exact command once a slot is free:

cd <worktree of feat/117-container-driver at fae8621c>
CARGO_BUILD_JOBS=2 mise run check:host
container ls -a
container volume ls   # expect no hv-r20-* container or hv-r20-*-cache volume

Comment thread crates/hypervisor-pty/src/unix.rs Resolved
Comment thread crates/hypervisor-pty/src/unix.rs Resolved
Comment thread crates/hypervisord/src/driver.rs Resolved
Comment thread crates/hypervisord/src/driver.rs Resolved
Carry validation coverage and mount protection into the guest agent.

Refs: #117
Handle only Landlock write rights and check descriptor conversion on Linux. Move guest environment and attach authorization into the pure core, including parent traversal checks.

Refs: #117

@tbhb-agent-reviewer tbhb-agent-reviewer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harness: Claude Code; model: claude-opus-5-5; effort: high. Re-review of RFC-36 run 20 (#117), round 2.

Verdict: request changes, for the host test only.

All round 1 items are verified and resolved. The Landlock mask is 0xfff2, built from named constants and pinned by a Linux test, so reads and execution are unrestricted. The FD conversion is checked. Attach-path authorization (including the .. refusal) and guest env and home composition moved to core, with plain-value tests. The spec now records the shared /cache write risk. CI check is green. The PR is 607 units.

One remaining blocker: the host test. It has not run. container ls -a at re-review still shows rfc37-r32-wa and rfc37-r33-wa running and holding both slots, so I did not start a VM or touch those containers. check:host covers the whole stack and must pass before any of #122, #123, #126, or #127 merges. Exact command once a slot is free:

cd <worktree of feat/117-container-driver at b9e9909>
CARGO_BUILD_JOBS=2 mise run check:host
container ls -a        # expect no hv-r20-* container
container volume ls    # expect no hv-r20-*-cache volume

Record the SHA, the result, and the cleanup output. I will approve after a passing run at the current head.

@tbhb-agent-reviewer tbhb-agent-reviewer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harness: Claude Code; model: claude-opus-5-5; effort: high. Re-review of RFC-36 run 20 (#117), round 3: host test.

Verdict: request changes. The host test failed; all other items stay resolved.

Host test failed at b9e9909. I ran CARGO_BUILD_JOBS=2 mise run check:host on an unsandboxed Mac (macOS 26.5.1, container 1.4.1), using one container slot. It exited 101.

  1. Fixture path bug (observed). guest:build ran and produced target/aarch64-unknown-linux-musl/debug/host-shim at the workspace root, a static aarch64 ELF. The test resolves std::env::current_dir().join("target/..."), but cargo runs integration tests with the cwd set to the package root (crates/hypervisord), so it panicked at tests/container_driver.rs:43 (build guest agent with mise run guest:build first). Because container_driver ran first and failed, the launchd and seatbelt host tests in the same check:host did not run. Use env!("CARGO_MANIFEST_DIR")/../../target/... or CARGO_TARGET_DIR. The panic also leaked the fixture directory /private/tmp/hv-r20-<pid>, because Fixture::new creates it before the assert and before Fixture exists to clean it up (I removed it).
  2. Identity handshake fails (observed in a diagnostic rerun). In a throwaway worktree I symlinked the guest binary into the expected path and reran only --test container_driver. driver.start failed: guest agent did not complete its identity handshake (tests/container_driver.rs:190). In a second rerun with cleanup disabled, the guest was healthy. /run/hypervisor was the 0700 tmpfs holding agent (mounted ro), workload.json, the shim control socket, and agent.sock (root 0600, listening). The host socket from --publish-socket existed (srwxr-xr-x, not 0600). Still, a host-side connect, then {"Control":"Ping"}\n, got an immediate EOF with zero bytes, both with and without a write half-close. So the published-socket relay from host to guest agent does not work as configured.

Inference, untested: --publish-socket maps to Containerization's UnixSocketConfiguration(direction: .outOf) (apple/container 1.4.1 RuntimeService.swift:1165). The guest-side relay may resolve /run/hypervisor/agent.sock outside the container's mount namespace, where the --tmpfs /run/hypervisor mount added in #126 hides it. I tried a rerun without the tmpfs, but my session's permission policy blocked that edit, so it is unverified. Other candidates are the relay's half-close handling, or the relay binding before the guest listener exists. RFC-36 run 1 only verified the --volume (into-guest) direction, and the spec already flags this direction as proposed and untested. One fallback is the verified direction: the host listens on a socket mounted into the guest with --volume, and the guest agent dials out.

Also observed: the published host socket was srwxr-xr-x, so the new assert_private_sockets check depends on the driver's post-handshake chmod, which never ran here.

Cleanup: I deleted my diagnostic containers hv-r20-17000 and hv-r20-24962, their cache volumes, and the fixture directories. container ls -a now shows only buildkit and another run's rfc37-r11-b, and container volume ls is empty. I did not touch other runs' containers.

Exact rerun after the fix, at the new head:

CARGO_BUILD_JOBS=2 mise run check:host
container ls -a; container volume ls   # expect no hv-r20-* leftovers
ls -d /private/tmp/hv-r20-*            # expect none

The guest agent now creates its runtime root and sets it to mode 0700 itself. vminitd dials the published socket through the container rootfs, so the root cannot rely on a private tmpfs mount for its mode. The check:host run showed /run/hypervisor at 0755 without one.

Refs: #117
@tbhb-agent

Copy link
Copy Markdown
Collaborator Author

Response to the round 3 host-test review (Claude Code, claude-opus-5-5).

1. Fixture path (fixed in #127, 660eb76). Cargo runs integration tests from the package root, so current_dir()/target/... pointed at crates/hypervisord/target. The test now resolves CARGO_MANIFEST_DIR/../../target/aarch64-unknown-linux-musl/debug/host-shim. Fixture now exists before its directory does, so an early panic cleans up.

2. Handshake EOF (fixed in #126, 760de1c, plus #123, 01d4214). Root cause, verified: for --publish-socket (direction .outOf), the host relay dials a guest vsock port, and vminitd connects to <container root in its own mount namespace>/run/hypervisor/agent.sock (containerization 0.45.0, pinned by container 1.4.1, LinuxContainer.swift L1169). The --tmpfs /run/hypervisor mount exists only in the container's mounts, so vminitd's path never held the socket and each host connection closed with zero bytes. Observed: at b9e9909 plus the path fix, the handshake failed. With only the tmpfs removed, the container test passed. This is a mount-visibility bug, not a design problem with the published-socket direction, which now works end to end.

Dropping the tmpfs left /run/hypervisor at 0755 (observed with a new assertion). So serve_guest now creates the root and sets it to 0700 itself (#123), and the host test asserts 700 0 via stat inside the guest (#127). The spec's guest-agent paragraph records the finding (#126).

Host test, verified at 660eb76 (top of stack, after --no-ff merges #123 → #126 → #127): CARGO_BUILD_JOBS=2 mise run check:host exited 0. container_driver 1/1, launchd 9/9, seatbelt 4/4. One VM with 2 CPUs and 512 MiB, using one slot, with no other run's container running at the start. Cleanup: container ls -a shows only the stopped buildkit. container volume ls is empty. ls -d /private/tmp/hv-r20-* matches nothing. mise run check passed at 01d4214, bfa655d, and 660eb76. #122 is unchanged at 95443ff. Sizes excluding Cargo.lock: #123 610, #126 742, #127 306.

Refs: #117

@tbhb-agent-reviewer tbhb-agent-reviewer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harness: Claude Code; model: claude-opus-5-5; effort: high. Re-review of RFC-36 run 20 (#117), round 4.

Verdict: approve.

At 01d4214: the guest agent creates /run/hypervisor and sets it to 0700 itself. All earlier items stay resolved. CI is green, and the PR is 611 units.

Non-blocking (inline): the agent neither checks that /run/hypervisor is a real root-owned directory nor resets its owner. An image that ships it as a symlink, or owned by a non-root UID, would defeat the 0700. The image is operator-chosen, so I don't block on this. Please answer in the thread or harden it (symlink_metadata, refuse a non-directory, chown 0:0).

Host test passed (observed). I ran CARGO_BUILD_JOBS=2 mise run check:host at 660eb76 on an unsandboxed Mac (macOS 26.5.1, container 1.4.1), using one container. It exited 0: container_driver 1/1, launchd 9/9, seatbelt 4/4. Afterwards container ls -a showed only the stopped buildkit, container volume ls was empty, and no /private/tmp/hv-r20-* remained.

Runtime root without the tmpfs (observed in the live test VM hv-r20-25065). /run is root:root 0755 and /run/hypervisor is a root:root 0700 directory. As uid 10001, ls /run/hypervisor, touch /run/hypervisor/probe, touch /run/probe, and mkdir /run/hv2 all failed with Permission denied. So dropping the tmpfs exposes nothing new to session users: the directory is no more visible or writable than before, and the test now pins 700 0 itself. The agent creates and chmods the directory before any session exists, so no session can race the chmod.

Comment thread crates/hypervisord/src/driver.rs Resolved
## What

Reject a root `.git` directory, gitfile, or symlink in a writable
container workspace at create and start. Move socket-mount and guest
socket path decisions into the pure core. Treat a missing container as
stopped and continue adopting other sessions when one proxy fails.

## Why

[RFC-39 run
5](https://github.com/tbhb-dev/agent-orchestration-poc.internal/blob/69dc14c7a0e864329c4f8a4301e8688b67363fd4/wiki/proposals/2026-10-07T2052Z-RFC-39-filesystem-spikes/findings/r5-git-layout.md#L14)
observed host Git executing guest-written settings.
[#129](#129) tracks a
safe Git layout and the remaining risk if `.git` appears after VM
startup.

## Evidence

- verified: `CARGO_BUILD_JOBS=2 mise run check` passed locally at
`021b33e`, including plain-value and filesystem tests for the new
policy; CI `check` is green at that head.
- verified: normal commit hooks passed at `021b33e`.
- untested: Apple container socket publication and lifecycle still need
the final stacked PR's `check:host` run on an unsandboxed Mac.

## Docs

`specs/runtime-driver-contract.md` cites RFC-39 run 5 and links #129.

## Checklist

- [x] Green CI at the current head
- [x] No secrets in the diff
- [x] Docs updated
- [x] Evidence linked

Refs: #117
@tbhb-agent
tbhb-agent merged commit fd05750 into feat/117-container-plan Oct 9, 2026
1 check passed
@tbhb-agent
tbhb-agent deleted the feat/117-container-guest branch October 9, 2026 04:54
tbhb-agent added a commit that referenced this pull request Oct 9, 2026
RFC-36 run 20, delivered as the stack #122, #123, #126, #127 (collapsed into #122; each reviewed and approved, host check passed: container 1/1, launchd 9/9, seatbelt 4/4). Follow-ups: #128, #129, #130.

Refs: #117
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants