Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/actions/flybot-tests/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ runs:
shell: bash
working-directory: core
run: npm test
- name: Plane tests
shell: bash
working-directory: plane
run: npm test
- name: Plugin lint
if: inputs.plugins != 'false'
shell: bash
Expand Down
1 change: 1 addition & 0 deletions docs/subsystems/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Detail pages for each FlyBot subsystem. Each page lists `owned_paths`, what the
| project | [project.md](project.md) | `plugins/project/` |
| social | [social.md](social.md) | `plugins/social/` |
| install | [install.md](install.md) | `core/scripts/`, arch-doc tests, `plugins/README.md`, the container image (`Dockerfile`, `compose.yaml`, `docker/`) |
| plane | [plane.md](plane.md) | `plane/` — management-plane skeleton (metadata, split credentials, lifecycle contract, compose materialization; not in the image) |
| team | [team.md](team.md) | `docs/subsystems/team.md` — **planned contract only** (no code paths; multi-human is not shipped) |
| interop | [interop.md](interop.md) | `dev/interop/`, `scripts/lint-interop.mjs` — offline development harness only |
| live-e2e | [live-e2e.md](live-e2e.md) | `dev/live-e2e/`, `scripts/lint-live-e2e.mjs` — real-model browser suite (#204) |
Expand Down
50 changes: 50 additions & 0 deletions docs/subsystems/plane.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
subsystem: plane
owned_paths:
- plane/
---

# plane (management-plane skeleton)

The first slice of the management server for self-hosted FlyBot (#3113, parent #3110): accounts,
workspaces and instance metadata, the two separated credentials, the create/start/stop/restart
lifecycle, and an abstract lifecycle backend with one compose materializer. It is a skeleton on
purpose: no container is started in this slice.

## Owns

- `plane/`: the `@flybot/plane` package — store, credentials, lifecycle contract, HTTP router,
CLI, tests and its README. The three replaceable working defaults (granularity, storage,
backend) are at the top of [plane/README.md](../../plane/README.md).

## Does not own

- The FlyBot runtime inside the instance (Agent, sessions, memory, scheduling) — that stays
`core/` and the plugins.
- The shipped image, `Dockerfile`, `compose.yaml`, `docker/` and the pack whitelist — this
package is not in the image and not in `docker/profile.mjs`'s bundles.
- Cross-instance social traffic — Bot Social and `plugins/social`; the plane will configure an
instance's connection later, not carry messages.
- Upgrade, recycle, backup and recovery implementation — present only as `NotImplementedError`
on the lifecycle contract.

## Invariants

- The plane never runs an Agent. It records an instance, and a `LifecycleBackend` decides where
it runs; the first backend writes a compose project and starts nothing.
- Two credentials never interchange: `op_` is accepted on `/api/*` only, `ctl_` is what the
plane hands an instance. Only sha256 is persisted, and comparison is timing-safe.
- Every instance route checks the operator token first, then `instance.ownerId ===
token.accountId`; a mismatch is `403`, an unknown id is `404`, no token is `401`.
- `(workspaceId, name)` is unique inside an owner's workspace, never globally; a clash is
`DuplicateInstanceError` / HTTP 409.
- `plane.json` is written atomically (temporary file + rename), mode 0600, directory 0700, and
carries `version: 1` for migration.
- `upgrade` and `recycle` fail loudly (`NotImplementedError`) instead of pretending to succeed.

## Tests and gates

`npm --prefix plane test` (20 cases) runs in the L1 `plane` suite registered in
[`docs/testing-layers.json`](../testing-layers.json) and in the `Plane tests` step of
[`.github/actions/flybot-tests/action.yml`](../../.github/actions/flybot-tests/action.yml). Root
`npm run verify` includes it. The suite needs no Docker, network, socket or model.
9 changes: 9 additions & 0 deletions docs/testing-layers.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,15 @@
"command": "npm --prefix plugins/workflow test",
"note": "Peer plugin; its CI job also runs the plugin lint and fails a skipped test."
},
{
"id": "plane",
"layer": "L1",
"paths": [
"plane/test/**/*.test.js"
],
"command": "npm --prefix plane test",
"note": "Management-plane skeleton: credential separation, ownership, create/start/stop, compose materialization; no network and no Docker."
},
{
"id": "scripts",
"layer": "L1",
Expand Down
1 change: 1 addition & 0 deletions docs/testing-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ The tables below are generated: edit `docs/testing-layers.json`, then run
| `plugins/project` | L1 | `plugins/project/test/**/*.test.js` | `npm --prefix plugins/project test` |
| `plugins/timeline` | L1 | `plugins/timeline/test/**/*.test.js` | `npm --prefix plugins/timeline test` |
| `plugins/workflow` | L1 | `plugins/workflow/test/**/*.test.js` | `npm --prefix plugins/workflow test` |
| `plane` | L1 | `plane/test/**/*.test.js` | `npm --prefix plane test` |
| `scripts` | L1 | `.github/scripts/*.test.mjs`<br>`policy/*.test.mjs` | `node --test ".github/scripts/*.test.mjs" "policy/*.test.mjs"` |
| `interop` | L1 | `dev/interop/**/*.test.mjs` | `npm run test:interop` |
| `patch-payload` | none (see reason) | `patches/ds-bot/files/**` | — Generated copies of the files the patches carry. They are compared against `core/` by `node scripts/ds-bot-patches.mjs check` (L2) and never run on their own; the code under test is the copy in `core/`. |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"node": ">=22"
},
"scripts": {
"verify": "npm --prefix core run lint && node scripts/lint-plugins.mjs && node core/scripts/build-client.mjs --check && node scripts/ds-bot-patches.mjs check && node scripts/ds-bot-patches.mjs canonical && npm --prefix core test && npm --prefix plugins/social test && npm --prefix plugins/tree test && npm --prefix plugins/timeline test && npm --prefix plugins/workflow test && npm --prefix plugins/project test && node --test \".github/scripts/*.test.mjs\" \"policy/*.test.mjs\" && npm run lint:interop && npm run test:interop && npm run lint:live-e2e && npm run test:live-e2e",
"verify": "npm --prefix core run lint && node scripts/lint-plugins.mjs && node core/scripts/build-client.mjs --check && node scripts/ds-bot-patches.mjs check && node scripts/ds-bot-patches.mjs canonical && npm --prefix core test && npm --prefix plane test && npm --prefix plugins/social test && npm --prefix plugins/tree test && npm --prefix plugins/timeline test && npm --prefix plugins/workflow test && npm --prefix plugins/project test && node --test \".github/scripts/*.test.mjs\" \"policy/*.test.mjs\" && npm run lint:interop && npm run test:interop && npm run lint:live-e2e && npm run test:live-e2e",
"lint:interop": "node scripts/lint-interop.mjs",
"test:interop": "node --import ./dev/interop/no-network.mjs --test \"dev/interop/test/*.test.mjs\"",
"demo:interop": "node --import ./dev/interop/no-network.mjs dev/interop/demo.mjs",
Expand Down
76 changes: 76 additions & 0 deletions plane/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# @flybot/plane — management-plane skeleton

**Three working defaults, all replaceable — none of them is an owner decision (#3113):**

1. **Granularity:** one FlyBot instance per **owner**, not per workspace or per Agent. The
issue's body suggested this as the closest thing to today's "one instance per person"; split
it further once the workspace product exists.
2. **Storage:** a single JSON document (`<dataDir>/plane.json`, `version: 1`) for accounts,
workspaces and instance metadata. It is in the data directory, not the Agent container's
volume, and the `version` field is there so a later slice can migrate it. No cloud database
is bound.
3. **Backend:** compose is only the **first** lifecycle path. `LifecycleBackend` is the
contract; `ComposeBackend` materializes a compose project and starts nothing. A cloud or
other orchestrator is another backend, not a rewrite.

Everything below follows from those three; if the owner picks a different granularity, storage
or orchestrator, only the default changes.

## What this first slice does

- Records accounts, workspaces and instances (`plane/src/store.mjs`), atomically (temporary
file + rename), file mode 0600, directory mode 0700.
- Splits the two credentials (`plane/src/credentials.mjs`): `op_<32 hex>` operator tokens for
the API, `ctl_<32 hex>` instance control tokens. Only sha256 is stored; both are checked with
`timingSafeEqual`; `redact()` is for logs. Neither token type verifies as the other.
- Creates, starts, stops and restarts an instance (`index.js`), with per-owner uniqueness on
`(workspaceId, name)` and `DuplicateInstanceError` on a clash.
- Routes the HTTP surface as a pure function (`plane/src/server.mjs`): `POST/GET
/api/instances`, `GET /api/instances/:id`, `POST /api/instances/:id/{start,stop}`. Every
instance route checks the operator token first, then `instance.ownerId === token.accountId`.
- Materializes an instance's compose project (`plane/src/lifecycle.mjs`): `compose.yaml` plus
`control.env` (0600), loopback port, project `flybot-<id>`. It does **not** run `docker
compose up`; `runner` is an unused injection point for the next slice.
- `upgrade` and `recycle` are on the contract and throw `NotImplementedError`.

## API

```js
import { createPlane, ComposeBackend } from '@flybot/plane'

const plane = createPlane({ dataDir: '/var/lib/flybot/plane', backend: new ComposeBackend({ instancesDir: '/var/lib/flybot/plane/instances' }) })
const instance = await plane.createInstance({ workspaceId: 'ws-1', name: 'main', ownerId: 'acct-1' })
await plane.startInstance(instance.id)
await plane.stopInstance(instance.id)
```

`createInstance()` returns the instance and, exactly once, `controlToken`. `getInstance()` and
`listInstances()` never include the token or its hash. `createPlane` accepts `{ dataDir,
store, backend, clock }`; `clock` is injectable for deterministic tests.

## CLI

```sh
node plane/bin/plane.mjs token --data-dir /var/lib/flybot/plane [--account owner_local]
node plane/bin/plane.mjs serve --data-dir /var/lib/flybot/plane [--port 3090]
```

`token` prints the operator token once (only its sha256 is kept). `serve` binds
`127.0.0.1:<port>` and answers `401` to an unauthenticated `GET /api/instances`.

## Not in this slice

No container is started, stopped or upgraded for real; no management UI; no public gateway; no
backup/recovery; no Bot Social wiring (the instance's social connection is a later package). The
package does not ship in the FlyBot image and is not registered in `docker/profile.mjs`'s
bundles.

## Tests

```sh
npm --prefix plane test
```

20 cases: ownership/403, 401, per-owner listing, duplicate creation (409), persistence, 400s,
start/stop/restart, unknown id (404), credential separation and storage, compose materialization,
idempotency and the zero-spawn guarantee.
66 changes: 66 additions & 0 deletions plane/bin/plane.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/usr/bin/env node
// Operator CLI. `token` is the one place a plaintext operator token is printed — once.
import { resolve } from 'node:path'
import { join } from 'node:path'
import { openStore } from '../src/store.mjs'
import { mintOperatorToken } from '../src/credentials.mjs'
import { ComposeBackend } from '../src/lifecycle.mjs'
import { createPlane } from '../index.js'
import { serve } from '../src/server.mjs'

const USAGE = `usage:
node plane/bin/plane.mjs token --data-dir <dir> [--account <id>]
node plane/bin/plane.mjs serve --data-dir <dir> [--port 3090]

token mint an operator token (printed once; only its sha256 is stored)
serve run the management-plane API on 127.0.0.1:<port>`

function parseFlags(argv) {
const flags = {}
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i]
if (!arg.startsWith('--')) continue
const [key, inline] = arg.slice(2).split('=')
if (inline !== undefined) flags[key] = inline
else if (argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')) flags[key] = argv[i + 1], (i += 1)
else flags[key] = true
}
return flags
}

function dataDirFrom(flags) {
if (typeof flags['data-dir'] !== 'string' || flags['data-dir'] === '') {
console.error('plane: --data-dir <dir> is required')
process.exit(2)
}
return resolve(flags['data-dir'])
}

const [command, ...rest] = process.argv.slice(2)
const flags = parseFlags(rest)

if (command === 'token') {
const dataDir = dataDirFrom(flags)
const account = typeof flags.account === 'string' ? flags.account : 'owner_local'
const store = openStore(dataDir)
const token = mintOperatorToken(store, account)
// stdout only: this is the mint, not a log line.
console.log(`account: ${account}`)
console.log(`token: ${token}`)
console.error('plane: this token is shown once and is not recoverable; store it outside the repository.')
} else if (command === 'serve') {
const dataDir = dataDirFrom(flags)
const port = flags.port === undefined ? 3090 : Number(flags.port)
if (!Number.isInteger(port) || port < 1 || port > 65535) {
console.error('plane: --port must be an integer between 1 and 65535')
process.exit(2)
}
const store = openStore(dataDir)
const backend = new ComposeBackend({ instancesDir: join(dataDir, 'instances') })
const plane = createPlane({ dataDir, store, backend })
serve({ port, plane, store })
console.log(`plane: listening on http://127.0.0.1:${port}`)
} else {
console.error(USAGE)
process.exit(2)
}
103 changes: 103 additions & 0 deletions plane/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// Public surface of @flybot/plane.
//
// const plane = createPlane({ dataDir, backend })
// const instance = await plane.createInstance({ workspaceId, name, ownerId })
//
// The plane owns metadata, ownership and credentials; a LifecycleBackend owns where the
// instance actually runs. This package starts no HTTP server and no container by itself —
// `server.mjs` is a pure router plus an opt-in node:http adapter, and the compose backend
// only writes files.
import { randomBytes } from 'node:crypto'
import { openStore, InstanceNotFoundError } from './src/store.mjs'
import { mintControlToken, hashToken } from './src/credentials.mjs'
import { MemoryBackend } from './src/lifecycle.mjs'

export { DuplicateInstanceError, InstanceNotFoundError, openStore } from './src/store.mjs'
export { LifecycleBackend, MemoryBackend, ComposeBackend, NotImplementedError, hostPortFor } from './src/lifecycle.mjs'
export {
mintOperatorToken,
mintControlToken,
verifyOperator,
verifyControl,
hashToken,
redact,
} from './src/credentials.mjs'
export { handle, serve } from './src/server.mjs'

function assertNonEmpty(value, field) {
if (typeof value !== 'string' || value.trim() === '') throw new TypeError(`${field} is required`)
}

// The record the plane hands out: the control-token hash never leaves the store.
function publicRecord(record) {
if (!record) return null
const { controlTokenHash, ...rest } = record
return rest
}

export function createPlane({ dataDir, store, backend = new MemoryBackend(), clock = () => Date.now() } = {}) {
const theStore = store ?? (dataDir ? openStore(dataDir) : null)
if (!theStore) throw new TypeError('createPlane needs a dataDir or a store')
const now = () => new Date(clock()).toISOString()

function applyOutcome(id, outcome) {
const patch = { updatedAt: now() }
if (outcome?.state) patch.state = outcome.state
if (outcome?.health) patch.health = outcome.health
if (outcome?.hostPort !== undefined) patch.hostPort = outcome.hostPort
if (outcome?.composePath !== undefined) patch.composePath = outcome.composePath
return publicRecord(theStore.updateInstance(id, patch))
}

async function createInstance({ workspaceId, name, ownerId } = {}) {
assertNonEmpty(workspaceId, 'workspaceId')
assertNonEmpty(name, 'name')
assertNonEmpty(ownerId, 'ownerId')

const createdAt = now()
theStore.ensureWorkspace({ id: workspaceId, ownerId, createdAt })

const controlToken = mintControlToken()
const record = {
id: `inst_${randomBytes(8).toString('hex')}`,
ownerId,
workspaceId,
name,
state: 'stopped',
health: 'unknown',
createdAt,
updatedAt: createdAt,
controlTokenHash: hashToken(controlToken),
}
theStore.insertInstance(record)

const outcome = await backend.create(publicRecord(record), { controlToken })
const instance = applyOutcome(record.id, outcome)
// The plaintext control token is returned exactly once, to the caller that created it.
return { ...instance, controlToken }
}

async function startInstance(id) {
const record = theStore.getInstance(id)
if (!record) throw new InstanceNotFoundError(id)
const outcome = await backend.start(publicRecord(record))
return applyOutcome(id, outcome)
}

async function stopInstance(id) {
const record = theStore.getInstance(id)
if (!record) throw new InstanceNotFoundError(id)
const outcome = await backend.stop(publicRecord(record))
return applyOutcome(id, outcome)
}

function getInstance(id) {
return publicRecord(theStore.getInstance(id))
}

function listInstances(ownerId) {
return theStore.listInstances(ownerId).map(publicRecord)
}

return { createInstance, startInstance, stopInstance, getInstance, listInstances }
}
18 changes: 18 additions & 0 deletions plane/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"name": "@flybot/plane",
"version": "0.0.0",
"private": true,
"description": "Management-plane skeleton: instance metadata, split credentials, an abstract lifecycle backend and a compose materializer. No runtime dependencies; does not ship in the FlyBot image.",
"type": "module",
"license": "Apache-2.0",
"main": "index.js",
"bin": {
"plane": "bin/plane.mjs"
},
"engines": {
"node": ">=22"
},
"scripts": {
"test": "node --test \"test/*.test.js\""
}
}
Loading
Loading