Skip to content

linux-setup.md

Latest commit

 

History

History
424 lines (359 loc) · 23.1 KB

linux-setup.md

File metadata and controls

424 lines (359 loc) · 23.1 KB
Raw

Linux hands node setup

How to turn a Linux desktop box into an oab-computer hands node with the Rust PoC in poc/reverse-attach-linux/. Written from two deployments, both verified end to end on 2026-09-28 (lending to an openab-pty session, the OpenAB Connect Screens pane, browser tools):

  • Pi node: Raspberry Pi OS / Debian 13, aarch64, the stock labwc desktop (a display exists).
  • Headless server: Ubuntu 24.04 Server, Intel N95, no desktop at all: a headless sway seat is created by the installer (--headless-seat).

Read the trust warning at the top of the README first. On Linux it is stricter, not looser: bash, mouse, key, the background-job tools (exec_start and friends, see Background jobs) and every browser tool are in the owner profile, so anyone who can attach this node under owner gets a shell as the user running the daemon. observe (the default for an attach that names no profile) serves only sys_info and screenshot. Those are the only two profiles (#59); the retired desktop runs as observe.

What you end up with

flowchart LR
    subgraph callers["Callers (anywhere on the tailnet)"]
        cli["Coding CLI / agent"]
        connect["OpenAB Connect<br/>Screens pane"]
        pod["openab-pty pod<br/>(reverse attach)"]
    end

    subgraph node["Linux hands node — one box"]
        ts["tailscale serve :8444<br/>TLS + injects Tailscale-User-Login"]
        subgraph seat["Desktop seat · systemd --user · WAYLAND_DISPLAY=wayland-0"]
            mcp["<b>oab-computer.service</b><br/>reverse-attach binary · 127.0.0.1:8795<br/>auth = allow-login <b>AND</b> bearer<br/>/mcp · /attach · /healthz<br/>sys_info · screenshot · bash · mouse · key<br/>exec_start · exec_poll · exec_list · exec_cancel"]
            pw["<b>oab-computer-pw-mcp.service</b><br/>@playwright/mcp · 127.0.0.1:8794<br/>headed distro Chromium, persistent profile"]
        end
        subgraph hands["What the tools touch"]
            grim["grim → screenshot"]
            wl["virtual pointer / wtype → mouse, key"]
            sh["bash -c as the desktop user"]
        end
    end

    cli -- "HTTPS · MCP" --> ts
    connect -- "screenshot loop" --> ts
    ts --> mcp
    mcp -- "WS dial-out<br/>ws://pod:8090/tools/attach/{session}" --> pod
    mcp -- "MCP_UPSTREAM browser=<br/>re-served, profile-filtered" --> pw
    mcp --> hands
Loading

Two systemd user units with linger enabled, both binding loopback only; tailscale serve terminates TLS and stamps the caller's Tailscale identity. Same shape as the Mac.

Prerequisites

  • A wlroots compositor as the seat (labwc, sway, wayfire): the daemon shells out to grim (screenshot) and wtype (keyboard) and holds its own virtual pointer, which need the compositor's wlr-screencopy, virtual-pointer and virtual-keyboard protocols. GNOME/Mutter and KDE do not expose these. Two ways to have one:
    • a desktop already running on seat0 (Raspberry Pi OS labwc, a sway login) — see Raspberry Pi notes for the no-monitor case;
    • none — pass --headless-seat to the installer and it runs sway with the headless wlroots backend as user unit oab-seat.service: one 1920×1080 virtual output, no input devices, no GPU or monitor needed (verified on a headless server, Ubuntu 24.04 Server).
  • The node on the tailnet with MagicDNS + HTTPS certificates enabled (needed for tailscale serve). Do not do this on a k3s node that hosts openab-pty pods: host-level tailscaled gives every pod tailnet egress as the node (measured, see docs/adr/reverse-attach.md).
  • Rust ≥ 1.85 (rustup), sudo.
sudo apt-get install -y grim wlrctl wtype nodejs npm        # + sway for --headless-seat
sudo apt-get install -y chromium                             # Debian / Raspberry Pi OS only

Debian 13 ships nodejs 20 and npm 9, which is enough for @playwright/mcp. Ubuntu has no chromium deb (snap only, unusable from a user unit); the installer downloads Playwright's Chromium instead and handles the AppArmor rule it needs (see Gotchas).

1. Tailscale

curl -fsSL https://pkgs.tailscale.com/stable/debian/$(. /etc/os-release; echo $VERSION_CODENAME).noarmor.gpg \
  | sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
curl -fsSL https://pkgs.tailscale.com/stable/debian/$(. /etc/os-release; echo $VERSION_CODENAME).tailscale-keyring.list \
  | sudo tee /etc/apt/sources.list.d/tailscale.list >/dev/null
sudo apt-get update && sudo apt-get install -y tailscale
sudo tailscale up --hostname="$(hostname)" --accept-dns=false --accept-routes=false

tailscale up prints a login URL; approve it in the browser (or pass --authkey). If it hangs with no URL at all, check curl -4 -m 8 https://controlplane.tailscale.com/ — see Gotchas.

2. Get the daemon

Prebuilt (preferred). Every v* release ships oab-computer-VERSION-linux-arm64.tar.gz and -linux-amd64.tar.gz (built and smoke-tested in CI on native runners) beside the macOS installer, with a .sha256 each:

V=0.12.0   # the first release with oab-computer-* assets (v0.11.0 and older use the old names)
A=$(dpkg --print-architecture)          # arm64 or amd64
curl -fsSLO "https://github.com/openabdev/openab-computer/releases/download/v$V/oab-computer-$V-linux-$A.tar.gz"
curl -fsSLO "https://github.com/openabdev/openab-computer/releases/download/v$V/oab-computer-$V-linux-$A.tar.gz.sha256"
sha256sum -c "oab-computer-$V-linux-$A.tar.gz.sha256"
tar xzf "oab-computer-$V-linux-$A.tar.gz" && cd "oab-computer-$V-linux-$A"
./install-linux.sh            # does steps 3–5 below; --allow-login / --no-browser / --port
./install-linux.sh --headless-seat   # same, on a box with no desktop (creates oab-seat.service)

install-linux.sh is idempotent: it keeps an existing token, detects your Tailscale login, writes both user units, installs @playwright/mcp when node/chromium are present, enables linger and tailscale serve. If it did everything, skip to Verify; steps 3–5 describe what it did.

From source (needs Rust ≥ 1.85):

git clone https://github.com/openabdev/openab-computer ~/repo/oab-computer
cd ~/repo/oab-computer/poc/reverse-attach-linux
cargo build --release              # ~30 s cold on a Pi 4/5; 2.2 MB binary
bash smoke.sh                      # 55 checks against the bundled mock runtime; all must pass

smoke.sh runs the binary with MCP_INSECURE_LOCAL=1 against mock_runtime.py on loopback. It does not need the desktop or the network, so run it first when something is off. The same smoke runs in CI on amd64 and arm64 for every push.

3. Bearer token and the daemon unit

mkdir -p ~/.config/oab-computer
head -c 32 /dev/urandom | base64 | tr -d '=+/\n' | head -c 40 > ~/.config/oab-computer/token
chmod 600 ~/.config/oab-computer/token

LOGIN=$(sudo tailscale status --json | python3 -c \
  'import sys,json;d=json.load(sys.stdin);print(d["User"][str(d["Self"]["UserID"])]["LoginName"])')

mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/oab-computer.service <<EOF
[Unit]
Description=oab-computer (Rust hands node: /mcp + /attach)
After=graphical-session.target

[Service]
ExecStart=%h/.local/oab-computer/oab-computer   # or …/target/release/oab-computer from source
Environment=BIND=127.0.0.1:8795
Environment=MCP_TOKEN_FILE=%h/.config/oab-computer/token
Environment=MCP_ALLOW_LOGIN=$LOGIN
Environment=MCP_UPSTREAM=browser=http://127.0.0.1:8794/mcp
Environment=WAYLAND_DISPLAY=wayland-0
Environment=XDG_RUNTIME_DIR=/run/user/%U
Restart=always
RestartSec=2

[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user enable --now oab-computer.service
sudo loginctl enable-linger "$USER"      # keep user units alive without a login shell

curl -s 127.0.0.1:8795/healthz           # ok
curl -s -o /dev/null -w '%{http_code}\n' -X POST 127.0.0.1:8795/mcp -d '{}'   # 401

Configured checks are AND-combined, exactly like the Swift daemon: a request must carry the bearer and arrive with an allow-listed Tailscale-User-Login. That header is only trustworthy because BIND is loopback and tailscale serve overwrites it; never bind to a LAN or tailnet address. A loopback request without the header is refused (deny … no Tailscale-User-Login header in journalctl --user -u oab-computer), which is why even local POST /attach goes through the https://…:8444 URL below.

MCP_UPSTREAM may be set before the browser unit exists: a down upstream just means no browser_* tools until it answers.

4. Browser tools (@playwright/mcp)

mkdir -p ~/.local/oab-computer/pw-mcp && cd ~/.local/oab-computer/pw-mcp
npm install --save-exact @playwright/mcp@0.0.82        # same pin as poc/pw-mcp (macOS)
cp ~/repo/oab-computer/poc/reverse-attach-linux/pw-mcp.sh ~/.local/oab-computer/pw-mcp.sh
chmod +x ~/.local/oab-computer/pw-mcp.sh

cat > ~/.config/systemd/user/oab-computer-pw-mcp.service <<'EOF'
[Unit]
Description=Playwright MCP for the oab-computer hands node (loopback :8794)
After=graphical-session.target

[Service]
ExecStart=%h/.local/oab-computer/pw-mcp.sh
Environment=XDG_RUNTIME_DIR=/run/user/%U
Environment=WAYLAND_DISPLAY=wayland-0
Environment=DISPLAY=:0
Restart=always
RestartSec=3

[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user enable --now oab-computer-pw-mcp.service

# Block camera/mic/notification/geolocation prompts: YouTube raised an xdg-desktop-portal
# "Allow app to use the Camera?" dialog that nothing but `mouse` could dismiss.
sudo mkdir -p /etc/chromium/policies/managed
echo '{"DefaultMediaStreamSetting": 2, "DefaultNotificationsSetting": 2, "DefaultGeolocationSetting": 2}' \
  | sudo tee /etc/chromium/policies/managed/oab-computer.json >/dev/null
systemctl --user restart oab-computer-pw-mcp

pw-mcp.sh uses the distro Chromium (--executable-path /usr/bin/chromium) so nothing is downloaded, launches it headed into the seat, keeps a persistent profile under ~/.local/oab-computer/pw-profile, and lists --allowed-hosts in both host and host:port forms (Playwright answers 403 to everything otherwise). Chromium runs with its sandbox on; check with tr '\0' '\n' </proc/$(pgrep -f 'chromium.*user-data-dir' | head -1)/cmdline | grep -c no-sandbox → 0.

5. Expose it

sudo tailscale serve --bg --https=8444 http://127.0.0.1:8795
sudo tailscale serve status
# https://<host>.<tailnet>.ts.net:8444 (tailnet only)
# |-- / proxy http://127.0.0.1:8795

The Playwright port is not served; only the daemon re-serves it, filtered by profile.

6. Verify from another tailnet machine

TOKEN=$(ssh <host> cat ~/.config/oab-computer/token)
U=https://<host>.<tailnet>.ts.net:8444
curl -s $U/healthz                                                  # ok
curl -s -o /dev/null -w '%{http_code}\n' -X POST $U/mcp -H 'Authorization: Bearer nope' -d '{}'   # 401
curl -s -X POST $U/mcp -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
  | python3 -c 'import sys,json;print([t["name"] for t in json.load(sys.stdin)["result"]["tools"]])'

Expect the five local tools plus 32 browser_* (owner surface; direct /mcp is always owner).

OpenAB Connect Screens pane: Add a Mac screen with Name <host>, URL https://<host>.<tailnet>.ts.net:8444/mcp, Display 0, the token above, Max FPS 1. Frames are JPEG, encoded in the daemon because Debian's grim has no libjpeg: about 50 KB for a 1080p desktop at Connect's quality 0.6, against about 2 MB as PNG.

Lend to an openab-pty session:

sequenceDiagram
    participant H as Human (CLI / Connect)
    participant N as hands node :8444
    participant R as openab-pty runtime :8090
    participant A as agent in the session
    H->>N: POST /attach {runtime, session, profile, ttl_secs, admin_credential}
    N->>R: POST /admin/sessions/{session}/tools-attach {ttl_secs}
    R-->>N: {secret, expires_in}
    N-->>H: 202 grant {id, state: idle}
    N->>R: WS GET /tools/attach/{session} · Bearer secret
    R-->>N: 101 (state: attached)
    A->>R: tools/list via $OPENAB_TOOLS_MCP_URL
    R->>N: tools/list (over the WS)
    N-->>R: owner: 9 local tools + 32 browser_* · observe: sys_info screenshot
    A->>R: tools/call screenshot
    R->>N: tools/call screenshot
    N-->>R: image/jpeg
    Note over N,R: close 4010 / DELETE /attach/{id} → stop · 1000 → redial with backoff
Loading 
curl -s -X POST -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' $U/attach \
  -d '{"runtime":"ws://<pod-tailnet-ip>:8090","session":"<name>","profile":"owner",
       "ttl_secs":3600,"admin_credential":"<openab-pty admin credential>"}'
# → 202 {"id":"grant-…","profile":"owner","requested_profile":"owner","state":"idle",…}
#   GET $U/attach lists, DELETE $U/attach/<id> revokes

"profile" is owner (full control) or observe (look only). A request without it gets observe; up to 0.8.2 it got owner. Any other name is a 400 profile must be one of [owner, observe]. A request for the retired desktop gets observe, and the grant shows requested_profile: desktop and a warning; a grant stored as desktop resumes as observe the same way. See tool-profiles.md.

OpenAB Connect stores the runtime's admin credential in the macOS Keychain (service dev.openab.connect, account = pod name). Inside the session, $OPENAB_TOOLS_MCP_URL then lists, under owner, sys_info screenshot bash mouse key exec_start exec_poll exec_list exec_cancel + 32 browser_* + the runtime's instance_status.

Operate

systemctl --user status oab-computer oab-computer-pw-mcp
journalctl --user -u oab-computer -f            # one line per deny / upstream error
systemctl --user restart oab-computer           # grants are in memory: re-lend afterwards
sudo tailscale serve --https=8444 off               # stop exposing
WAYLAND_DISPLAY=wayland-0 XDG_RUNTIME_DIR=/run/user/$(id -u) wlr-randr    # outputs the tools see

Grants survive a daemon restart (#12): live grants are kept in ~/.local/state/oab-computer/grants.json (mode 600; override with MCP_GRANTS_FILE, or MCP_GRANTS_FILE=off to disable), and on start every grant still inside its deadline is re-dialled under its original id — journalctl --user -u oab-computer shows grants: resuming <id>. The file holds the attach secrets, at the same trust level as the bearer token file. A grant whose runtime has forgotten it (pod replaced) ends with handshake_rejected_401; lend again.

A request or stored grant that names the retired desktop profile runs as observe (#59). Each such mapping is logged and appended to ~/.local/state/oab-computer/audit.jsonl (mode 600; override with MCP_AUDIT_FILE, or MCP_AUDIT_FILE=off to disable). See Upgrading from 0.8.x.

Background jobs

Under owner, exec_start / exec_poll / exec_list / exec_cancel run long work without one long call: same arguments and structuredContent as on macOS (README Tools table), with bash -c instead of zsh -f -c. On a reverse attach bash refuses timeout_secs over 100 s (the runtime fails a call after 120 s), and both that refusal and the initialize instructions point the agent here. Under observe neither bash nor the job tools exist, and a long bash call is an unknown tool, not a refusal that names it.

  • Logs: $XDG_STATE_HOME/oab-computer/jobs/<job_id>.out / .err (default ~/.local/state/…; directory mode 700, files 600). The job writes to them directly, so they are never truncated and can be followed with tail -f. exec_poll reads them by byte offset (stdout_since / stderr_since → stdout_next / stderr_next), at most 1 MiB per stream per call.
  • States: running while the shell or anything it left in the job's process group runs (server &), then exited (the shell ended on its own, exit_code = its status) or killed (cancelled, timed out or ended by a signal, exit_code = 128 + signal). timeout_secs 0, the default, means none.
  • Cancel: exec_cancel sends KILL (default) or TERM to the job's process group (each job runs in its own session), children the shell left behind included. The shell is reaped only once the group is empty, and the group is signalled only before that, so a recycled pid is never hit (#72). Only a process that leaves the group (setsid or setpgid, as daemonizing code usually does) escapes the job and its cancel; a plain double fork stays in the group.
  • Lifetime: jobs belong to the daemon, not to a connection. When a reverse-attach socket drops, its in-flight calls are cancelled, but a job it started keeps running and the next attach can poll it. A daemon restart forgets the jobs: their processes keep running (the unit uses KillMode=process) and their logs stay on disk. Running jobs plus the 10 most recently finished are listed; older finished jobs are forgotten. Seven days after a job finishes, its logs are deleted and the job is forgotten even if it is among those 10; a running job's logs are kept whatever their age. GC runs at start-up and on each exec_start.
ls -lt ~/.local/state/oab-computer/jobs/            # <job_id>.out / .err
tail -f ~/.local/state/oab-computer/jobs/<job_id>.out

Headless server notes (Ubuntu 24.04)

What a box with no desktop needed beyond the Pi, all handled by install-linux.sh --headless-seat:

  • Seat: sway with WLR_BACKENDS=headless WLR_LIBINPUT_NO_DEVICES=1 WLR_RENDERER=pixman as oab-seat.service; ~/.config/sway/config declares output HEADLESS-1 1920x1080. sway ignores WAYLAND_DISPLAY and takes the first free socket name — usually wayland-1 — so the installer reads the socket back from /run/user/<uid> and writes that into both units.
  • Chromium: Playwright's own build (npx playwright install chromium), since Ubuntu's is a snap. Ubuntu 24.04 restricts unprivileged user namespaces with AppArmor and Chromium's sandbox aborts with No usable sandbox!; the installer writes /etc/apparmor.d/playwright-chromium granting userns to that binary path (the distro-endorsed fix — the sandbox stays on; do not reach for --no-sandbox). Playwright's Chromium also defaults to X11 and the headless seat has no Xwayland ("launched a headed browser without having a XServer"), so pw-config.json adds --ozone-platform=wayland.
  • Fonts and locale: a server has neither; the first page rendered as tofu boxes, the second in Arabic (no Accept-Language, so example.com picked one). fonts-dejavu-core, fonts-liberation, fonts-noto-cjk, fonts-noto-color-emoji plus --lang=en-US / locale: en-US / LANG=en_US.UTF-8.
  • Node from fnm/nvm: /usr/bin/node was a symlink into ~/.local/share/fnm/... with no npm on the system PATH; PW_NODE_BIN in the unit carries that directory.
  • Pod-host rule still applies: The headless server runs k3s (ARC controller, traefik) and host tailscaled. That is fine only because it hosts no openab-pty pods; it must never start to.

Raspberry Pi notes

  • Raspberry Pi OS's stock desktop is already a headless seat. With no monitor plugged in, labwc keeps a 1920×1080 output alive (wlr-randr: NOOP-1 "Headless output 2"), the panel and file manager run on it, and grim captures it. Nothing from the "sway headless" spike (#23) is needed. Plug a monitor in and the geometry changes; sys_info and the captured frame follow. mouse keeps the old layout until the virtual pointer reconnects, and while that pointer is live screenshot reports the same layout for points and the region clip, so the two agree.
  • labwc also runs an Xwayland on :0, which is why the browser unit sets DISPLAY=:0 alongside WAYLAND_DISPLAY.
  • The Debian Chromium is arm64-native and fast enough for browser_snapshot-style work; a first page load is ~10 s on a Pi 4, ~1 s warm on example.com.
  • ~/.cargo/bin is not on the PATH of non-login shells (ssh host cargo … fails with "command not found"); use export PATH=$HOME/.cargo/bin:$PATH in scripts.
  • 8 GB RAM is plenty: daemon ≈ 5 MB, Playwright MCP ≈ 80 MB, headed Chromium 300–600 MB.

Gotchas

  • Tailscale login never prints a URL and curl https://controlplane.tailscale.com/ times out while github/8.8.8.8 work: a consumer router (TP-Link Deco here) was dropping the whole 192.200.0.0/24 control-plane range for one wifi client, keyed on a months-old association. sudo nmcli con down <profile>; sudo nmcli con up <profile> fixed it instantly; no reboot.
  • nmcli needs sudo on Raspberry Pi OS ("Not authorized to control networking").
  • Anything that talks to the seat needs WAYLAND_DISPLAY + XDG_RUNTIME_DIR. An ssh-started or systemd-started process has neither; the daemon defaults them to wayland-0 / /run/user/<uid> when unset, the units set them explicitly, and pw-mcp.sh adds DISPLAY.
  • grim -t jpeg → "jpeg support disabled" on Debian's build. The daemon never asks grim for JPEG: it takes raw PPM from grim and encodes the JPEG itself.
  • A seat with no physical input silently drops transient virtual devices. Headless sway and a Pi with no mouse advertise no pointer/keyboard capability, so a wlrctl/wtype call that creates a device, sends and exits within milliseconds is lost: Chromium never binds wl_pointer/wl_keyboard in time, yet the tool exits 0 and mouse/key report ok: true. The daemon therefore holds a persistent virtual pointer (its own Wayland connection, absolute motion over the output layout) and a long-lived wtype -s keyboard anchor from startup; the journal logs seat: virtual pointer up / virtual keyboard anchor up, and swaymsg -t get_seats shows capabilities 3. wtype -s overflows a C int above ~35 min and exits at once, so the anchor sleeps 30 min and is respawned.
  • Chromium's ctrl+l focuses the omnibox but typed text may append rather than replace; send ctrl+a before typing a URL.
  • Playwright --allowed-hosts needs the host:port form or every request is 403.
  • A stale upstream Mcp-Session-Id must be dropped before re-initialize, or the upstream 404s the initialize too and the browser tools never come back after a Playwright restart. Fixed in the daemon; symptom was tools/list shrinking to the five local tools.
  • The daemon's /attach refuses loopback callers without Tailscale-User-Login by design; use the tailscale serve URL even from the node itself.
  • pkill -f inside ssh host 'sh -c …' kills the sh -c whose command line matches. Use pkill -x <name> — but -x matches at most 15 characters of the process name, so pkill -x oab-computer silently matches nothing.
  • Chromium No usable sandbox! / Trace/breakpoint trap on Ubuntu 24.04+: AppArmor userns restriction; see Headless server notes.
  • Headed browser "without having a XServer" on a headless seat: Chromium picked X11; add --ozone-platform=wayland (the installer's pw-config.json does).