Skip to content

bug(kubernetes): /sandbox is root-owned with mode 0777 on local-path storage; Docker and Podman create it 0700 for the sandbox user #5721

Open

Description

@cv

User Story

I use OpenShell through NemoClaw, which runs OpenClaw and Hermes agents in OpenShell sandboxes on the Docker driver and on the Kubernetes driver (kind with the default local-path storage in NemoClaw's CI). On Kubernetes, the sandbox user's home, /sandbox, is a root-owned, world-writable directory without the sticky bit. Software that requires a private home refuses to start there, and the same software works on the Docker driver.

Problem Statement

On the Kubernetes driver with a provisioner that creates the workspace PVC root as root:root 0777 (rancher local-path, the default StorageClass in kind and k3s), the sandbox sees:

$ stat -c '%n %a %u:%g' /sandbox
/sandbox 777 0:0

The Docker and Podman drivers create /sandbox owned by the sandbox user with mode 0700 (Docker: docker_sandbox_bundle_archive in crates/openshell-driver-docker/src/lib.rs; Podman: workspace.directory(".", 0o700, true) in crates/openshell-driver-podman/src/isolation.rs).

On Kubernetes the agent container mounts the workspace PVC root at /sandbox (apply_workspace_persistence in crates/openshell-driver-kubernetes/src/driver.rs), so the root's owner and mode come from the provisioner. The workspace-init container (openshell-sandbox seed-workspace) runs as the sandbox UID with no capabilities. It copies the image's /sandbox contents and leaves the root alone, and it couldn't chown or chmod a root-owned directory anyway. The pod's fsGroup doesn't help: kubelet doesn't apply fsGroup to local-path's hostPath volumes, and on volume types where it does apply, the root becomes root:<gid> with the setgid bit, which is still not owned by the sandbox user.

This is the same at v0.1.9 (ce58cf8e, the version NemoClaw pins) and on main (78ba3477). I found no existing issue for the PVC root. #2761 covered the ownership of seeded contents under the former root seeder, and #5047 asks for private subpaths for caller-provided mounts.

Impact / Why This Matters

Acceptance Criteria

  • On the Kubernetes driver with local-path storage, /sandbox in a new sandbox is owned by the sandbox UID with mode 0700, as on Docker and Podman.
  • The sandbox pod stays non-root: no init container gains root or capabilities to do this.
  • Files in /sandbox survive stop and start, and existing sandboxes keep their workspace contents after a gateway upgrade.
  • A conformance test checks that /sandbox belongs to the sandbox user and isn't group- or world-writable.

Reproduction Steps

  1. Create a kind cluster (kind create cluster); its default StorageClass is local-path.
  2. Install the OpenShell v0.1.9 Helm chart with the Kubernetes driver as in the Kubernetes setup guide.
  3. openshell sandbox create --name perms -- sleep infinity
  4. openshell sandbox exec perms -- sh -c "id; stat -c '%n %a %U %u:%g' /sandbox"

Observed: /sandbox 777 root 0:0. Expected: /sandbox 700 sandbox, with the sandbox user's UID and GID, as with the Docker driver.

The same result without a gateway: in a kind cluster, a Pod with runAsUser: 10001, fsGroup: 10001, fsGroupChangePolicy: OnRootMismatch that mounts a fresh local-path PVC sees the mount root as 777 0:0. On the node, the provisioner created /var/local-path-provisioner/pvc-… as 777 0:0.

Environment

  • OpenShell: v0.1.9 (ce58cf8e9b953281352d98a637e41616295b1318), also current main
  • Kubernetes: kind v0.33.0 (kindest/node:v1.37.0), StorageClass standard (rancher.io/local-path)
  • Host: Ubuntu 24.04 (arm64) and GitHub-hosted runners (amd64 and arm64)
  • Docker driver, same image: /sandbox is 700 and owned by the sandbox user
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions