Repository navigation
Add an sshfs backend to eval-under - #13
Open
yarikoptic wants to merge 11 commits into
Open
yarikoptic wants to merge 11 commits into
yarikoptic wants to merge 11 commits into
Conversation
yarikoptic-gitmate
force-pushed
the
claude/sshfs-backend
branch
from
September 28, 2026 20:01
91bcf2f to
c0ccb14
Compare
sshfs is the filesystem people actually reach for when they mount a
remote over ssh, and it breaks git-annex in a way no local filesystem
does. This adds it as a backend so a report against it can be reproduced
directly.
Two modes:
- Loopback (default): a throwaway sshd on the first free port at or
above 2222, with its own host key, authorized_keys and pid file inside
the run's scratch directory, and a fresh backing directory
sshfs-mounted back over it. The system sshd is not used and
~/.ssh/authorized_keys is never written to.
- `--host` mounts a real remote using the caller's ssh config, so a
reporter's own server and mount options can be used verbatim.
Knobs for the options reports actually turn on: `--no-cache`,
`--workaround`, `--opt`, `--port`, `--user`, `--remote-dir`, plus the
common `--mount-point` / `--set-home` / `--keep`.
Notes on the two non-obvious pieces:
- Dropping privileges deliberately calls `sudo -u` literally rather than
going through the `${SUDO[@]}` array the other backends use: that array
is empty when we are already root, which is exactly the case that needs
the drop.
- sshd runs with `UsePAM yes`. With it off, sshd refuses any account whose
shadow entry is locked (`!`), which is the normal state for service and
CI accounts, and the mount fails for a reason that looks nothing like
its cause. Its log is dumped on mount failure for the same reason.
`bin/ci/run-under.sh` refuses sshfs together with a root-requiring target
(pjdfstest): a FUSE mount belongs to whoever mounted it, there is no
--no-root-squash equivalent, and the run would measure privilege rather
than the filesystem. No matrix cell and no badge -- the backend exists for
on-demand reproduction, and GOTCHAS.md records what it does to hardlink
identity, mtime granularity and fifos before anyone reads a red result as
a filesystem bug.
Split out of the filesystem-probing branch, which had grown too large to
review as one change.
Verified: shellcheck clean, all 28 bats tests pass (the four "every
installed backend" cases now cover this backend), the mount works as root
and via the privilege-drop path, and the refusal above exits 2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
The backend was on-demand only, so nothing watched it. Add it as a matrix row: `sshfs (loopback) / git-annex test` and `sshfs (loopback) / git testsuite`, taking the matrix from 20 cells to 22. Not four cells. stress-ng and pjdfstest are needs-root, and a FUSE mount belongs to whoever mounted it -- there is no --no-root-squash equivalent the way there is for NFS -- so those two could only ever report on privilege rather than on the filesystem. run-under.sh already refuses that pair with exit 2; the matrix now expresses the same rule as data. Rather than a hand-kept exclusion list, the backend row carries `no-root: true` and `cell_enabled()` in matrix.sh derives the gap from it plus the target's existing `needs-root`. So the reason lives in one place, a future unprivileged backend gets the behaviour for free, and the grid cannot silently grow a cell that measures privilege. Every consumer asks `cell_enabled` instead of assuming backends x targets: - matrix-json.sh omits the pair, so the workflow never schedules it; - gen-readme-matrix.sh prints `n/a` instead of a badge (README regenerated); - update-status.py keeps it out of status.json, so it publishes no permanently-unknown badge; - render-report.py renders an explicit gap rather than an "unknown" badge, which would read as "not measured yet". `sshfs-git-annex` is expected red and is annotated as such on the report page and in GOTCHAS.md: invisible hardlinks (link() succeeds, st_ino differs, nlink=1) break `git annex add`'s post-link verification. `sshfs-git` is the control -- same mount, a suite that never hardlinks into an object store. Its result is not predicted here; the first run measures it, and GOTCHAS.md gets the answer either way. Verified: shellcheck clean and 28/28 bats; matrix-json.sh emits 22 cells with exactly the two sshfs ones; update-status.py's cell set agrees and omits sshfs-stress-ng / sshfs-pjdfstest; and render-report.py, run over a fabricated 22-cell status.json, writes 22 cell badges plus overall, shows two n/a gaps in the sshfs row, and carries the expected-red note on sshfs-git-annex. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
The `sshfs / git testsuite` cell came back red, and the previous commit
had called it "the control: a suite that never hardlinks into an object
store". That was wrong, and measuring it turned up a bug in our own
reporting.
Root cause of the cell: `git clone <local path>` hardlinks each object
and then sanity-checks the result, comparing st_mode/st_ino/st_dev/
st_size/st_uid/st_gid against the source (builtin/clone.c). sshfs
synthesises st_ino per path, so the check fails and the clone dies with
"hardlink different from source". Plain `git clone` of a local path does
not work on sshfs at all -- a broader statement than the git-annex one,
and the same mechanism. It accounts for t1507-rev-parse-upstream (20),
t1013-read-tree-submodule (58) and t0035-safe-bare-repository (2), each
of which clones or adds a submodule in setup. A second, independent cause
is the absence of unix sockets: t0301-credential-cache (37, "unable to
bind ... Operation not permitted") and t0052-simple-ipc (9/9).
`-o disable_hardlink` fixes both this and the git-annex failure, which is
the counter-intuitive part worth telling a reporter: sshfs then fails
link() with EPERM instead of pretending, and every caller here has a copy
fallback that only an honest failure reaches. Measured, with an ext4
control:
default disable_hardlink ext4
git annex add (locked) ok ok ok
git annex add, annex.addunlocked=true FAIL ok ok
git clone <local path> FAIL ok ok
The reporting bug: `not ok N ... # TODO known breakage` is git's
test_expect_failure -- a TAP TODO directive that prove counts as an
expected result, not a failure. dump-failure-logs.sh counted those, so
this cell reported 351 failed assertions where prove saw 166, and the two
worst-looking scripts (t1517-outside-repo at 104, t0450-txt-doc-vs-help
at 54) are ones prove reports as *ok*. That sends triage after failures
that do not exist; it inflated every git cell, vfat included, not just
this one. Both the count and the detail listing now exclude the
directive, matching prove.
GOTCHAS.md gets the mechanisms, the workaround table, a single-script
reproduction recipe, and -- deliberately -- the list of small failures
that are *not* explained (t0003-attributes 6, t1091 8, t0610 3, t0021 3,
ten scripts with one each), so nobody assumes they share a cause.
Method: built the pinned git v2.55.0 locally and ran the whole t0*/t1*
range under the sshfs backend (174 scripts, 10366 assertions) plus an
ext4 control, which passes the five scripts under suspicion. Reproduced
the clone failure minimally, read git's check in clone.c, and confirmed
link() returns EPERM under disable_hardlink. One hypothesis was tested
and rejected rather than left implied: synthesised inode numbers do not
collide at 4000 entries, so that is not behind the whole-suite failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
The rebase dropped my copies of this rule from update-status.py and render-report.py: master now enumerates cells once, in matrix_cells() in bin/ci/evals.py, which is a better home for it than the two duplicates the pre-rebase branch had. cell_enabled() states the rule -- a `no-root` backend has no cell for a `needs-root` target, because such a cell could only report on privilege rather than on the filesystem -- and matrix_cells() skips those pairs, so status.json, the badges and the report page agree with the workflow instead of publishing a permanently-unknown badge. render-report.py draws an explicit `n/a` gap for them; an "unknown" badge would read as "not measured yet". The shell side already has the same rule in matrix.sh, and run-under.sh refuses the pair outright. Verified: both sides report 22 cells with exactly sshfs-git-annex and sshfs-git, and neither reports sshfs-stress-ng or sshfs-pjdfstest; the README grid shows two n/a entries; run-checks.sh passes in full. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
The rebase put this branch on master's known-issues machinery, which judges a cell per test rather than by the suite's exit code. The two new sshfs cells had no entries, so they would have reported `failing-new` even though their failures are the documented findings this backend exists to surface. The prose that used to say so by hand in GOTCHAS.md belongs in evals/known-issues.yaml now, where the verdict, the status page and GOTCHAS all read it from one place. Measured, not asserted: ran the cell here (`run-under.sh sshfs n/a git` then `check-cell.sh`), which reported 185 failures across 30 scripts, and attributed the scripts by the message in their own logs rather than by guessing: - `sshfs-git-local-clone-hardlink` (114 failures, 16 scripts, fs-divergence) -- `git clone <local path>` hardlinks each object then compares st_ino/st_dev/... against the source (builtin/clone.c), and sshfs synthesises st_ino per path, so the clone dies with `fatal: hardlink different from source`. Every script here clones or adds a submodule in setup. Plain `git clone` of a local path does not work on sshfs at all; --no-hardlinks, file:// and `-o disable_hardlink` do. - `sshfs-git-no-unix-sockets` (46, fs-limitation) -- `bind()` on the mount fails with EPERM, so the credential-cache daemon never starts and simple-ipc finds no server. Mirrors the existing vfat entry; `unix-socket=no` in fs-capabilities.sh predicts it. - `sshfs-git-untriaged` (25, needs-triage) -- kept deliberately separate: neither signature appears in these scripts' logs, so folding them into the mechanisms above would be a guess. Candidates noted (1-second mtime granularity, no xattr/fifo). With these, the cell judges `failing-known` -> conclusion=success: 9734 pass, 185 fail all known, 0 new. GOTCHAS.md's generated section is regenerated from them, and run-checks.sh passes in full. Still outstanding: `sshfs (loopback) / git-annex test` has no entry yet, so it will report `failing-new` on its first run here. Its test ids are best taken from CI's own verdict -- CI runs a git-annex daily build, this container has 10.20240129, and ids from the wrong build would mask real failures rather than document them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
The first draft of these three entries was derived from a run in this development container, and CI disagreed with it: `sshfs (loopback) / git testsuite` reported `failing-new` with 11 failures no entry covered, while 28 of the ids I had listed passed there (4 under `sshfs-git-local-clone-hardlink`, 24 of the 25 under `sshfs-git-untriaged`). The mechanisms were right; the assertion numbers were not mine to guess. The container runs as root. That both flips permission-dependent assertions and renumbers the scripts that define tests conditionally, so a locally derived `script#N` can name a different assertion than the same `script#N` on a runner -- which is exactly what happened: my `t0003-attributes.sh#41` passes in CI while `#48` fails, and `t0450`'s `#167,347` became `#797`. This is the same reason the git-annex entry is still outstanding rather than invented here, applied to the cell where I had not applied it. Reseeded from run 36476334300: - `sshfs-git-local-clone-hardlink`: 114 -> 110 ids, dropping t0001#28, t0003#41, t0610#48 and t1091#47. - `sshfs-git-no-unix-sockets`: unchanged. It reproduced exactly, 46/46, which is what a mechanism that owns two whole scripts should do. - `sshfs-git-untriaged`: 25 -> 12 ids. Only t1092#55 survived; the 11 new failures join it. Three are now named from their own assertions, since CI's log quotes them: t0003#48 "builtin object mode attributes work (dir and regular paths)", t0061#6 "run_command can run a script without a #! line", t0061#18 "run_command is asked to abort gracefully" -- so the mode bits execve() and builtin_objectmode read join mtime granularity and xattr/fifo as the candidates to check first. Also recorded: t0003#48 is in `vfat-git-untriaged` too, and the t0061 and t1091/t1300 assertions here sit beside the ones listed there, so some of this is likely one non-POSIX cause shared with vfat rather than anything sshfs invented. Worth noting these are sshfs's own: the same run's `NFS (localhost) / git testsuite` is green with no nfs git entries at all, so none of the 11 is an environment or build artefact. Verified by replaying CI's exact failure set (all 168 ids, reconstructed and cross-checked per script against the job's own summary table -- zero mismatches) through `known_issues.py check`: `failing-known`, known 168, new 0, no now-passing notices, exit 0, where before it was `failing-new` with 11. `run-checks.sh` passes in full, and GOTCHAS.md is regenerated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
My previous commit reseeded these entries from CI run 36476334300 and predicted the cell would go green. It did not: run 36485450259, on near-identical code, reported 18 residual failures with **no overlap** with the 12 that run reported. Every id I had just added passed, and ids I had just removed as "now passing" -- t0003#41, t0017#4, t1430#26, t1700#9 -> #10-15 -- were back. The residue is not a fixed set of divergences, so no `script#N` list can cover it, and the explanation I committed last time (root renumbering the scripts) was wrong: locally t0003 yields 24,25,29,32,33,34, exactly CI's ids. What is actually true, measured: - The two mechanism entries are deterministic. `local-clone-hardlink` reproduced 110/110 and `no-unix-sockets` 46/46 on both CI runs, and t0003's six hardlink assertions failed in all six local runs. - The residue is flaky at a low per-test rate. Six local runs of `t0003 t0017 t0040 t1700`: t0017#4 failed in one, t0040#37 and t1700#10-15 and t0003#41/#48 in none, though CI has flagged each. - It needs concurrent load. `prove --jobs 1` is no cleaner than `--jobs 4`, and 14 runs of t0017 on its own were all clean. - Some of it cannot be filesystem semantics at all: t0040#37 "OPT_CALLBACK() and OPT_BIT() work" and t0017#4 "test-tool env-helper --type=ulong" parse arguments and environment variables, and t0450 compares documentation against `-h` output. What they share is capturing output into `>out`/`2>err` in the trash directory on the mount and grepping it, which points at the mount losing or delaying writes under load. So this commit stops pretending the list gates anything. The entry now carries the union observed across both CI runs, labelled as a sample, plus the measurements above and the untried candidate that would actually resolve it: mounting with `-o attr_timeout=0 -o entry_timeout=0` (possibly with `-o max_conns=N` or `-o sync_read`) and re-measuring, which if it works makes the cell deterministic and properly xfail-able. This does NOT make the cell green -- it will still report `failing-new` whenever a run draws a flake outside the sample, and that is now the honest state rather than a hidden one. run-checks.sh passes and GOTCHAS.md is regenerated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
Two things this branch owed. `dump-failure-logs.sh` had no `sshfs)` arm, so every failing sshfs cell printed `unknown backend: sshfs` and no diagnostics at all -- visible in the git-annex cell's log, which is the one place they would have helped. The new arm prints any surviving `/tmp/eval-under-sshfs-*.scratch/sshd.log` (the backend removes its scratch dir on teardown, so the log is only there when teardown was skipped -- a crash, or --keep), any leftover `mount -t fuse.sshfs`, which is itself the finding when a suite timed out, and fuse-tagged dmesg. And the `cache=no` candidate the last commit left untried is now measured: five runs of `t0*.sh` at `--jobs 4`, counting failures beyond the 64 assertions this subset's two mechanisms own. Baseline drew 4, 8 and 8; `cache=no` drew 1 and 2, at about 7% more wall clock. So sshfs's caching is implicated, and `cache=no` is worth having as a mitigation -- but it does not make the cell green, because known_issues.py sets `failing-new` on the first uncovered failure and has no notion of a flake budget. One flake per run still fails the cell most runs. That turns the remaining question into a decision rather than a patch, and the note in evals/known-issues.yaml now says so: cover the whole cell with `tests: ["*"]` the way `vfat-pjdfstest` does (green, but it masks the 110 and 46 findings along with any future regression), make the cell non-gating, or give the verdict machinery a flake budget -- which belongs in the machinery, not in this backend's PR. Not choosing one here on my own. shellcheck clean over 26 scripts; run-checks.sh passes; GOTCHAS.md regenerated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
…ell-wide Two decisions from the PR discussion, plus the rebase onto d10a71d (master gained the mtime-stability and git-annex-linkannex targets, so sshfs now has four cells rather than two). **`-o disable_hardlink` for the git-annex target only**, set in run-under.sh. Without it that cell measured nothing at all: every test group died in setup cloning a local repo, and `testremote type git` then hung until the 2400s timeout, so the verdict was `incomplete` with 0 results (run 36567303570). Measured here, A/B on the same mount: without: fatal: hardlink different from source ... rc=128 with: CLONE OK rc=0 and `git annex test -p "testremote type git"` -- the group that hung for 40 minutes -- then reports `All 126 tests passed (6.29s)`, with `add` passing on both unlocked branches, i.e. `failed to link to annex` gone too. (Local git-annex is 10.20240129; CI runs a daily build, so CI is the real test.) Deliberately not applied to two other cells, and GOTCHAS.md now says so: `git testsuite` keeps hardlinks because the failure there IS the finding (110 assertions), and `git-annex linkAnnex loop` keeps them because that target exists to measure what linkAnnex does -- disabling them would have it measure the copy fallback instead. So the sshfs cells do not all mount the same filesystem, which the doc flags. **The git cell is now gated by one whole-cell entry**, `sshfs-git-flaky-residue` with `tests: ["*"]`, replacing the id list that reported `failing-new` twice. Four CI runs of the same cell on near-identical code: the two mechanisms reproduced exactly 110 and 46 every time, while the residue drew 12, 18, 11 and 12 with no overlap between runs. `-o cache=no` cuts it about fourfold (baseline 4, 8, 8 vs 1, 2 over `t0*.sh`) but not to zero, and known_issues.py fails a cell on the first uncovered failure, so that is not enough. The cost is masking, and it is real: a regression in the two mechanisms would no longer fail this cell. What keeps it visible is that known_issues.py credits every matching issue, so the mechanism entries still report their own counts in the verdict table and in GOTCHAS, and the whole-cell entry renders as "(whole cell)" rather than silently. The narrower fix is a flake budget in the verdict machinery -- useful for vfat and nfs too -- and the entry's notes say that belongs in its own change. Verified: a synthetic cell with three ids never seen before now reports known 5 / new 0 -> failing-known, exit 0, with the mechanisms still itemised. Also: `flaky` added to the tag vocabulary, since the file had no way to say this; and tests/test_matrix_json.py's cell count now follows cell_enabled() instead of a bare backends x targets product, which also pins matrix.sh's bash implementation of that rule against evals.py's Python one. run-checks.sh passes in full (shellcheck 30 scripts, pyflakes, bats, known-issues, 33 unit tests); README's matrix and GOTCHAS' generated section regenerated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
yarikoptic-gitmate
force-pushed
the
claude/sshfs-backend
branch
from
October 3, 2026 03:26
f6cc4fb to
ef1a7df
Compare
First CI verdict for the `git-annex linkAnnex loop` cell that master's #11 added, and it is the crispest statement of this backend's central result: ok 1 - unlock 0/200 rounds failed not ok 2 - add-unlocked 200/200 rounds failed (100.00%) in 21s Locked operations are fine. Only the path that needs a hardlink to be *observable* fails, and it fails every single round with `f<N> failed to link to annex` -- so this is not a slow or racy mount, it is the synthesised-inode divergence, stated without the noise the git cell suffers from. Recorded as `sshfs-linkannex-unlocked-add` with `tests: [add-unlocked]`: one id, no whole-cell masking, because there is nothing intermittent here to absorb. The entry also records why this cell deliberately does NOT mount with `-o disable_hardlink` while `sshfs / git-annex test` does: with the option git-annex falls back to copying, so the loop would report a reassuring 0% and measure the fallback instead of linkAnnex. Verified by replaying CI's exact verdict (fail add-unlocked, pass unlock) through `known_issues.py check`: known 1, new 0 -> failing-known, exit 0. Where the other sshfs cells stand on this run (37093228458), for the record: - `git testsuite`: GREEN. The whole-cell `sshfs-git-flaky-residue` entry does its job. - `mtime stability`: GREEN with no entry at all. I had expected sshfs's 1-second granularity to trip it; it does not. - `git-annex test`: the disable_hardlink change moved it from `0 result(s)`/incomplete -- a 40-minute hang into the 2400s timeout -- to 838 results, 835 passing, in 59s. Three named failures remain (`Repo Tests v10 locked`: `concurrent get of dup key regression`, `fix`, `export and import of subdir`), deliberately not recorded yet: the git cell taught me to confirm a failure set across two runs before pinning it, and this is one run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
`sshfs / git-annex test` reported the same three failures out of 838 on
two consecutive CI runs -- 37093228458 and 37094765390, both `pass 835 ·
fail 3`, the same names -- so unlike the git cell's residue this set does
not wander and is listed by name rather than covered cell-wide:
Tests.Repo Tests v10 locked.concurrent get of dup key regression
Tests.Repo Tests v10 locked.fix
Tests.Repo Tests v10 locked.export and import of subdir
Waiting for that second sample before writing anything was the point:
the git cell cost two pushes to learn that one run's failure set is not
evidence of a fixed one.
The obvious worry was that these are artefacts of `-o disable_hardlink`
-- the option this branch now sets for this target makes `link()` fail
outright, so it could plausibly break operations that hardlink for
legitimate reasons. Tested both ways on the same mount:
without the option: both die in setup `add` (the `failed to link to
annex` cascade that stops the suite)
with the option: setup succeeds, and each test reaches its own
assertion -- `fix` fails at `fix of moved file
failed with unexpected exit code`, `export and
import of subdir` at `git commit failed with
unexpected exit code`
So the option did not create these three; it let the suite run far
enough to reveal them. `concurrent get of dup key regression` was not
probed locally and rests on CI's verdict alone, measured twice -- the
entry says so rather than implying otherwise.
Tagged needs-triage, not attributed: the two symptoms sit at different
points and need not share a cause. To be split as causes are found, the
way vfat-git-untriaged is meant to be.
Verified by replaying CI's verdict through `known_issues.py check`:
known 3, new 0 -> failing-known, exit 0. run-checks.sh passes in full;
GOTCHAS.md regenerated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #5, which had grown too large to review as one change. This is the sshfs backend on its own, on top of current master; #5 keeps the probing / capabilities / survey work and has dropped its sshfs wiring.
Why sshfs
It is what people actually reach for to mount a remote over ssh, and it breaks git in a way no local filesystem does. SFTP's
ATTRScarries no inode number and no link count, so sshfs synthesisesst_inoper path and reportsnlink=1. Callers that hardlink and then verify the link cannot see that it happened, and they fail.Two such callers, both measured here against an ext4 control:
-o disable_hardlinkgit annex add(locked)git annex add,annex.addunlocked=truegit clone <local path>Plain
git cloneof a local path does not work on sshfs at all —builtin/clone.chardlinks each object and comparesst_mode/st_ino/st_dev/st_size/st_uid/st_gidagainst the source. That is broader than the git-annex failure and the same mechanism.-o disable_hardlinkfixes both, which is the counter-intuitive part worth telling a reporter: sshfs then failslink()withEPERMinstead of pretending, and both git and git-annex have copy fallbacks that only an honest failure reaches. A filesystem advertising hardlinks it cannot express is worse than one admitting it has none.What's here
bin/eval-under-sshfs, in two modes:authorized_keysand pid file inside the run's scratch directory — with a fresh backing directory sshfs-mounted back over it. The system sshd is not used and~/.ssh/authorized_keysis never written to.--host: mounts a real remote using the caller's ssh config, so a reporter's own server and mount options can be used verbatim.Flags for the options reports actually turn on:
--no-cache,--workaround,--opt,--port,--user,--remote-dir, plus the common--mount-point/--set-home/--keep.Wiring:
install-backend.shgrows aninstall_sshfs, andrun-under.shgrows the backend case plus a refusal — sshfs together with a root-requiring target exits 2 rather than producing a meaningless red result.In the matrix: two cells, not four
The matrix goes from 20 cells to 22 —
sshfs (loopback) / git-annex testandsshfs (loopback) / git testsuite. Both are red, both for the reasons above, and both are annotated as expected on the report page and inGOTCHAS.md.Not four.
stress-ngandpjdfstestareneeds-root, and a FUSE mount belongs to whoever mounted it, so those cells could only report on privilege rather than on the filesystem.Rather than a hand-kept exclusion list, the backend row carries
no-root: trueandcell_enabled()inmatrix.shderives the gap from that plus the target's existingneeds-root. The reason lives in one place, a future unprivileged backend gets the behaviour for free, and the grid cannot silently grow a cell that measures privilege. Every consumer askscell_enabledinstead of assuming backends × targets:matrix-json.shgen-readme-matrix.shn/ainstead of a badge (README regenerated)update-status.pystatus.json— no permanently-unknownbadgerender-report.pyA reporting bug this turned up
not ok N ... # TODO known breakageis git'stest_expect_failure— a TAP TODO directive that prove counts as an expected result, not a failure.dump-failure-logs.shcounted those, so the sshfs git cell reported 351 failed assertions where prove saw ~170, and the two worst-looking scripts (t1517-outside-repoat 104,t0450-txt-doc-vs-helpat 54) are ones prove reports as ok — triage aimed at failures that do not exist. This inflated every git cell, vfat included. Both the count and the detail listing now exclude the directive; the same cell reports 179 after the fix.Two non-obvious pieces in the backend
sudo -uliterally rather than going through the${SUDO[@]}array the other backends use. That array is empty when we are already root, which is exactly the case that needs the drop — routing through it made the documentedsudo bin/eval-under sshfsinvocation fail with-u: command not found.UsePAM yes. With it off, sshd refuses any account whose shadow entry is locked (!) — the normal state for service and CI accounts — and the mount fails for a reason that looks nothing like its cause. Its log is dumped on mount failure for the same reason.Verification
bin/ci/run-checks.sh: shellcheck clean, 28/28 bats — including the four "every installed backend" cases, which now cover this backend.nlink=1with distinct synthesised inodes for two names of one file.t0*/t1*range under the backend (174 scripts, 10366 assertions) plus an ext4 control that passes; the clone failure was then reproduced minimally. One hypothesis was tested and rejected rather than left implied: synthesised inode numbers do not collide at 4000 entries.sshfs-stress-ng/sshfs-pjdfstestare absent as theno-rootrule intends.GOTCHAS.mdnames the small failures that are not explained (t0003-attributes6,t10918,t06103,t00213, ten scripts with one each), so nobody assumes they share a cause.Note for whoever merges second
#5 also touches
update-status.pyandrender-report.py(to filter its on-demandcapabilitiestarget), in the same enumeration this PR changes. Expect a small conflict in those two files for whichever of #5 / #13 lands second; both changes are additive filters on the same loop and compose cleanly.🤖 Generated with Claude Code
https://claude.ai/code/session_01E1fDiGVWKywpbhVps5hzQK