Add @mieweb/os-cloud-provider: mieweb deploy --target mieweb (#475)

[runleveldev]

Closes #475. Stacked on #476 (volumes, #421). This PR targets 421-volumes and should be retargeted to main once #476 merges.

What's in it

packages/os-cloud-provider: the @mieweb/deploy-contract DeployProvider for os.mieweb.org. It's written in TypeScript: Node runs the sources directly in development, and tsc builds dist/ for publishing. The contract is installed with pnpm's git subdirectory syntax, pinned to the mieweb/cloud#14 commit.

• deploy
  • Looks the container up by hostname (wrangler.jsonc name), then creates or updates it. A 409 from a concurrent create is retried as an update.
  • If the image or GPU requirement changes, it deletes and recreates the container; the rw /mnt/data volume is kept across the recreate.
  • It exposes one HTTP service plus SSH, polls the job (Ctrl-C cancels), and returns the VMID as the resource id.
  • It calls the Manager only when the configuration actually changed.
• Code sync. After updating the container, deploy copies the local worktree into /opt/app/src over the container's SSH port, then restarts app.service. It honors .gitignore files (nested ones too) and skips .git/; staged/committed status doesn't matter.
  • It's pure JS (ssh2, ignore, tar-stream), so no local ssh or rsync is needed.
  • Only changed files are sent (compared by size and mtime), streamed as a tar into sudo tar -x. Files deleted locally are deleted remotely; gitignored remote paths are left alone.
  • Auth: ssh-agent, then ~/.ssh keys (with a passphrase prompt), then a password prompt. Host keys are trusted on first use and pinned in ~/.mieweb/known_hosts.
• destroy, whoami, login, logout: token from MIEWEB_OS_TOKEN, or the per-instance cache in ~/.mieweb/os.json written by login. logout revokes the key on the Manager. dev and tail are intentionally omitted.
• Compatibility with a Manager that predates volumes: the provider warns and skips the volume. Read-only requests are retried on dropped connections.

Manager

• GET/POST /api/v1/auth/cli/callback: the loopback login handoff (§4.3).
  • Without a session, it sends you through the normal sign-in first.
  • With a session, the GET only shows a confirmation page; it never creates a key.
  • The POST requires the session and a CSRF token, and API-key-only requests are rejected.
  • Only a port is accepted from the client; the redirect host is hard-coded to 127.0.0.1 and the CLI's state value is echoed back. The key is returned in the URL fragment, so it never appears in a request line.
• Security fix: GET /sites/:id/containers/new, which any container creator can call, returned the full ExternalDomain rows, including cloudflareApiKey/cloudflareApiEmail. It now returns only {id, name, siteId}.
• Service models now declare onDelete: CASCADE, matching the migrations. Without it, deleting a service in an update failed with a foreign-key error on SQLite.
• LoginPage does a full page load for same-origin /api/... redirects, so sign-in can return to the CLI route.

images/cloud: built on nodejs. MinIO (built from source at a pinned release, since MinIO no longer publishes binaries), sqld (checksum-verified download), and Valkey run as systemd units bound to 127.0.0.1, with their data under /mnt/data. app.service installs dependencies only when package.json or the lockfile changed, runs build if present, then starts the app on $PORT. The image is added to the bake file and build-images.yml.

Docs: docs/users/mieweb-cli-deploy.md, the image docs, the package README, and examples/mieweb.jsonc.

Testing

• Provider (pnpm test): 67 tests.
  • Contract conformance, both structural and live, against a fake Manager.
  • The SSH client against an in-process ssh2 server: password fallback, key login, host-key pinning.
  • The file sync against a directory-backed fake remote.
• Manager jest: new cli-auth and containers.self-service suites. agents.checkin.test.js crashes the same way on the unmodified 421-volumes branch, so that crash wasn't introduced here.
• Live against make dev (SQLite + simulated hypervisor): create, update, recreate on image change, destroy, live conformance, and the browser login → whoami → logout round trip.
• Live against manager.os.mieweb.org (site 1, image pushed to registry.os.mieweb.org/opensource-server/cloud:provider-test):
  • A fresh deploy created the container, waited for SSH, synced the code, and served the app in 61 s.
  • Code-only redeploys took about 5 s. The test container was destroyed afterwards.
• Image: booted under systemd; all four units came up, and the app reached MinIO, libSQL and Valkey.

Follow-ups

• Make ghcr.io/mieweb/opensource-server/cloud public. :latest only exists after a release.
• Switch @mieweb/deploy-contract to ^0.2.1 once it's published to npm.
• The login flow and volume persistence are untested on production, which doesn't have Support user-defined shared volumes for containers (replace hardcoded quick_and_dirty mp0) #421 or this route yet.
• The full-stack Proxmox compose run is documented but hasn't been run.
• One unreproduced run of the provider suite showed 4 failures; the following 11 runs were clean.

[runleveldev]

@mieweb/os-cloud-provider can't be installed from git yet — root cause + one-line fix

Found while wiring the provider into @mieweb/cli (mieweb/cloud, stacked on mieweb/cloud#14). Installing this package from git (github:mieweb/opensource-server#<sha>&path:/packages/os-cloud-provider) fails for every consumer, so mieweb deploy --target mieweb can't be set up outside this repo.

Root cause. The package exports only a built dist/, so pnpm has to prepare it: install its dependencies, then run prepack → tsc. Without a packageManager pin, pnpm runs that step with npm. npm doesn't understand the &path: in this package's own contract dependency (github:mieweb/cloud#2bd2f78…&path:/packages/deploy-contract) and warns npm-package-arg ignoring unknown key "2bd2f78…&path". It then installs the private mieweb-cloud monorepo root in place of @mieweb/deploy-contract, and tsc fails with TS2307: Cannot find module '@mieweb/deploy-contract' (exit 2).

Verified test matrix (installing df9fbef into an empty app; the "pin" rows use a local copy with only the pin added):

Consumer pnpm	This package pins packageManager	Result
10.17.1	no	❌ prepared with npm → contract missing → tsc fails
10.17.1	yes	❌ pnpm 10 ignores the pin; still npm
12.8.1	no	❌ still npm; also blocked by blockExoticSubdeps (on by default in 12)
12.8.1	yes	✅ prepared with pnpm install, dist/ built, createProvider(env) imports, supports('mieweb') === true

Fix for this PR — pin the package manager in packages/os-cloud-provider/package.json:

{
  "name": "@mieweb/os-cloud-provider",
  "version": "0.1.0",
  "packageManager": "pnpm@12.8.1",
  // …
}

Consumer side (worth a note in the package README until the contract is on npm). The app installing the provider needs pnpm ≥ 12 and, in pnpm-workspace.yaml:

blockExoticSubdeps: false   # this package has a git-hosted subdependency (the contract)
allowBuilds:
  "@mieweb/os-cloud-provider@https://codeload.github.com/mieweb/opensource-server/tar.gz/<sha>#path:/packages/os-cloud-provider": true

After mieweb/cloud#14 merges and @mieweb/deploy-contract is published, switching the dependency to ^0.2.1 removes the git-hosted subdependency, and with it the blockExoticSubdeps exception. The pin is still required as long as this package is consumed from git.