Refuse to create a mirror from scratch when it already exists elsewhere

[yarikoptic-gitmate]

AsyncDataset.ensure_installed() runs datalad create whenever the target directory isn't an installed dataset. So a Dandiset registered in the superdataset but not installed gets re-created with a new datalad-id and a one-commit history. That happens to an uninstalled submodule in a fresh clone of dandi/dandisets, and to a mirror directory removed by hand on drogon. The next push then collides with the real repository on GitHub, or overwrites it if forced. Zarr mirrors missing from zarr_root have the same problem.

This is hazard H1/H2 from the CI-mirroring plan (docs/ci-mirroring-plan.md on claude/jolly-meitner-v8jo47), split out as its own PR. It's a prerequisite for running the update from a partial checkout, and it also protects drogon today.

Changes

• ensure_installed() takes a new before_create hook. It is awaited only when the dataset is about to be created, and raising from it cancels the creation. An installed mirror never reaches it.

• Dandisets: raise MirrorMissingError if the superdataset's .gitmodules registers the ID, or if gh_org/<id> exists on GitHub.

• Zarrs: raise if the Dandiset already has a submodule for that Zarr at the asset path, or if zarr_gh_org/<zarr_id> exists. This also covers backup-zarrs, which syncs into --partial-dir.

• New GitHub.repo_exists(). A 404 is an answer here (not retried, not logged as an error), but GitHub also returns 404 for a private repo the caller can't see, so a 404 is only trusted when:

  • there is a token at all (otherwise it raises);
  • for a classic token, the reported X-OAuth-Scopes include repo, which is needed to see private repos (otherwise it raises).

  A bad token gets 401, which fails through get_repo()'s usual path. Fine-grained and App tokens don't report scopes; they are the same token we push those repos with, so they can see them.

The GitHub checks cost one GET per brand-new Dandiset or Zarr.

Testing

• New test/test_mirror_guard.py (marked ai_generated), which needs no Docker. It checks that:
  • a registered-but-uninstalled Dandiset is refused and nothing is created;
  • a Dandiset or Zarr whose repo is already on GitHub is refused;
  • a new Dandiset gets through the checks;
  • an installed dataset never runs the check;
  • repo_exists() returns the right answer for 200 and 404, and refuses to answer without a token or with a classic token lacking repo.
• Without the fix, the first test goes straight on to datalad create.
• Passing locally: flake8 src test, mypy src test, the new tests, test_zarrbargo.py, test_dirty_detection.py, test_util.py, test_aioutil.py, test_cfg_dandiset.py, test_quiescence.py.
• Not run: the Docker-based test_core.py / test_zarr.py. CI will cover those.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KoaLJfNiVVnwSDHNMg2bpK

[yarikoptic-gitmate]

CI note: the test (3.x, py) jobs fail in fixture setup, not in this change. docker compose pull for dandi-cli's test archive fails with minio Error unauthorized: access to the requested resource is not authorized. The image is quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z, which dandi/dandi-cli#1921 switched to on 2026-09-14 after minio/minio was deleted from Docker Hub. That registry now refuses the pull too.

All 17 errors are Docker-backed tests. Everything else passes, including the new test_mirror_guard.py. main last went green on 2026-09-20, before the pull started failing, and the docs-only #125 fails the same way.

There's no fix yet: dandi-cli's docker-compose.yml needs another MinIO source, such as a mirrored image, bitnami/minio, or one built from source. I'm triggering one re-run to confirm.

---

Generated by Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.39%. Comparing base (90aacaf) to head (e63bc39).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #124      +/-   ##
==========================================
+ Coverage   76.95%   77.39%   +0.43%     
==========================================
  Files          19       19              
  Lines        3328     3375      +47     
  Branches      505      517      +12     
==========================================
+ Hits         2561     2612      +51     
+ Misses        571      565       -6     
- Partials      196      198       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.