Skip to content

Second machine from a snapshot: init --from-snapshot, on-branch restore, core modules, venv check, doctor on Linux - #15

Open
plur9 wants to merge 3 commits into
mainfrom
fix/second-machine-from-snapshot
Open

plur9 wants to merge 3 commits into
mainfrom
fix/second-machine-from-snapshot

Conversation

@plur9

@plur9 plur9 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Setting up Datacore on a second machine from a snapshot, fixed so it works without hand repairs. Every item below comes from a real headless Ubuntu 24.04 second-machine install made from a snapshot of an existing installation. The install worked in the end, but only after moving files by hand, and the first datacore update reported "37 module(s) failed" when nothing was broken.

Fixes

1. Core modules are not failures. Modules that ship inside the root Datacore repo (gtd, research, outbox, goals, decisions, github, analytics, tab-capture, voice-terminal, whatsapp) have no .git of their own. datacore update reported each one as "Not a git repository" and counted it as failed. A new moduleKind() tells them apart: the module has its own clone, the root repo tracks it (core), or neither (local). update now reports core modules as "part of Datacore, updated with it". snapshot create records them as source: core, and restore and diff skip them. They used to be written as local, so restore warned that it "cannot restore" them.

2. Root repo "not in registry". On a new machine .datacore/registry/repositories.yaml does not exist: it is private and gitignored, and the snapshot did not carry it. The sync transport then refused the root repo. update now makes sure the registry names the root under <root>. When there is no registry file it creates a minimal one and says so in one plain sentence (copy the full file from your other machine, or use --with-registry). When the file lacks the entry it adds it and keeps comments. It never touches a file it cannot parse. The transport now also gets an explicit --root. New: snapshot create --with-registry carries the private registry files (repositories, infrastructure, principals). The snapshot is then written owner-only. Restore writes those files only where none exist.

3. Restore left modules and spaces detached. Restore checked out exact commits, so on the real install 27 modules and 7 of 10 spaces were on a detached HEAD, and every git pull failed. Restore now runs git switch -C <branch> <commit> with the upstream set, so each repo is on its branch at the snapshot commit. update also repairs a module that is already detached: it switches it back to its default branch before pulling. If HEAD has commits on no branch, it changes nothing and gives the command to keep them. Clone URLs from a lock file now go through execFile, not a shell string.

4. init checks python3 -m venv first. On Debian and Ubuntu, venv needs python3.X-venv. Without it the MCP server could not start, and init only found out after setup. checkVenvSupport() builds a throwaway venv with pip during the system-setup step and prints the exact command (sudo apt install python3.12-venv). In an interactive run it waits for you to install it and checks again. The venv error from update now carries the same hint.

5. datacore init --from-snapshot <file>. This clones the root from the snapshot's recorded source. Then, before any space or module step, it restores the snapshot's spaces and modules, on their branches. It installs the restored modules' own dependencies after the venv exists. It never creates a template space the snapshot provides: if that space's clone failed, init reports an error and creates no template. Interactive init now asks once, before anything is installed, whether Datacore already runs on another machine. If so, it explains datacore snapshot create --settings there and --from-snapshot here. snapshot restore also warns when a non-git directory blocks a space, which is the template collision that hit the real install.

6. space --help. It said "Create, join, list and audit" but listed only create and list. join and audit both had handlers but were missing from the action list. Both are listed now, and the help points to the snapshot route for setting up a whole machine.

Also, from the same install:

  • datacore update dirtied the root repo on every run. It rewrote the tracked .datacore/modules/package-lock.json. It now installs with npm ci from the lock file, puts the lock file back byte for byte, and skips the install when nothing has changed.
  • doctor on a headless server. It showed "✗ Claude Desktop config" on Linux, where there is no Claude Desktop, and ended with "Missing recommended dependencies" while every dependency was present: every status that was not ready got that message, including a degraded ledger. The Claude Desktop row now appears only where the app is installed, and .mcp.json is read from the data dir. The ledger rows are printed, and the closing line matches the real status.
  • Fixed the existing typecheck error in ensureRecallHook, which blocked npm run verify.

Tests (written first, seen failing, then the code)

  • tests/second-machine.test.ts (21 tests, real git repos): core module detection, update, snapshot and diff; restore on-branch for modules and spaces; update pulls a restored module forward; detached repair, and refusal when commits would be lost; registry created, added, left alone, and not parsed; --root passed; private registry carried only on request and written only where absent; template-collision warning.
  • tests/second-machine-setup.test.ts (18 tests): venv check and install hints; personalSpacePlan never chooses the template when the snapshot provides the space; root clone URL order; an unreadable snapshot refused before anything changes; --from-snapshot parsing; second-machine help text; doctor on Linux and on a Mac with and without the app; doctor verdicts; space actions and help.
  • tests/python-env.test.ts (+3): npm ci with a lock file, lock file stays byte-identical, no reinstall when nothing changed.

Results: bun test 197 pass, 0 fail (155 before). tsc --noEmit clean. scripts/prepublish-verify.sh passes (types, tests, build, dist version, doctor from dist, no --rebase).

Smoke test, not part of the suite: in an isolated HOME with local bare repos for the root, a personal space and one module, init --from-snapshot cloned the root from the snapshot source and added upstream. It restored the real personal space (no template) and the module, both on main at the snapshot commits, tracking origin. Then update moved a detached module back to main and pulled it to the tip.

Not covered: no second run on a real Ubuntu machine yet. The interactive question and the venv re-check prompt are covered by code review only. --settings is still carried in the snapshot but not applied by restore, as before.

Releasing (the owner's step, not done here)

Nothing was published. A release needs:

  1. Bump package.json (suggest 2.7.0: new flags init --from-snapshot and snapshot create --with-registry).
  2. Run npm run verify.
  3. Run npm publish.
  4. Re-run the second-machine install on a fresh Ubuntu 24.04 machine with the published CLI.

Snapshots made by older CLIs still restore. Their core modules were recorded as local and are skipped because they already arrive with the root.

🤖 Generated with Claude Code

plur9 and others added 3 commits October 23, 2026 19:32
…n registry

- Modules shipped inside the root repo (gtd, research, outbox, ...) are
  core: update reports them as such instead of 'Not a git repository' and
  no longer counts them as failed; snapshot records them as source 'core'
  and restore/diff skip them.
- snapshot restore puts modules and spaces ON their branch at the snapshot
  commit (switch -C, upstream set) instead of a detached HEAD.
- update moves an already-detached module back to its default branch
  before pulling, and refuses without changing anything if commits would
  be left behind.
- update makes sure the transport registry names the root repo (<root>),
  creating a minimal repositories.yaml on a machine that has none and
  saying so plainly; passes --root to the transport.
- snapshot create records the root repo's source; --with-registry carries
  the private registry files, written on restore only where absent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tor on Linux

- checkVenvSupport builds a throwaway venv before init relies on one and
  names the exact fix (sudo apt install python3.X-venv on Debian/Ubuntu);
  the venv failure message in update carries the same hint.
- ensureModuleDeps installs with npm ci from the tracked lock file, puts
  the lock back byte for byte, and skips the install when nothing changed,
  so update no longer leaves the root repo dirty.
- doctor skips the Claude Desktop check where there is no Claude Desktop
  (Linux, or app not installed), reads .mcp.json from the data dir, prints
  the ledger rows, and only says 'Missing recommended dependencies' when
  one is missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- datacore init --from-snapshot <file> clones the root from the snapshot's
  source, restores its spaces and modules before anything else, installs
  the modules' dependencies, and never creates a template space the
  snapshot provides (a failed clone is an error, not a template).
- Interactive init asks once whether Datacore already runs on another
  machine and explains snapshot create --settings there and
  --from-snapshot here.
- space --help lists join and audit (both existed, neither was listed) and
  points to the snapshot route for a whole machine; init and snapshot help
  describe the new flags.
- Fixes a pre-existing typecheck error in ensureRecallHook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant