diff --git a/docs/README-lanes.md b/docs/README-lanes.md index 713b14b9..a6caed0a 100644 --- a/docs/README-lanes.md +++ b/docs/README-lanes.md @@ -4147,9 +4147,19 @@ one passed `lane-end`. * reads every leg, and every dependency nested in one (an initialized submodule of the leg's own, as `code/upstream/openRepoShape` will be), as a **`leg` row** — - `clean`, `dirty`, `unpushed`, `dirty+unpushed`, `absent` (not checked out here) - or `unreadable`. A leg is the assembly's, shared by every lane the assembly - homes, as the assembly's own checkout is: **no `--yes` acts on a leg + `clean`, `dirty`, `unpushed`, `dirty+unpushed`, `absent` (not checked out here, + or not there at all) or `unreadable` (a `.git` of its own that git cannot read, + or a leg directory this account cannot search, #255). A dependency nested in a + leg that git lists as not initialized has no row where its directory is missing, + or empty and searchable: it was never initialized. But git lists a dependency it + cannot search as not initialized too, so where its directory, or one above it, + cannot be searched (or it holds a `.git` that git cannot read), it is an + `unreadable` row, counted: a read that failed is never a clean dependency (#255). + One whose directory holds a checkout that git can read (a clone made by hand, + which git does not count as initialized) is read like an initialized one: it + holds the lane's work all the same. + A leg is the assembly's, shared by every lane the assembly homes, as the + assembly's own checkout is: **no `--yes` acts on a leg checkout** (its branches are another matter: see `--branches` below). It is COUNTED (exit 3, and `lane-end`'s gate) only where its HEAD's own `Lane:` trailer is this lane's and it holds edits or commits origin lacks — the rule the sweep @@ -4217,8 +4227,9 @@ leg's repository (a paired tree, or one `lane-worktrees add --checkout checkout of the tree's own repository keeps it: a dependency nested in a leg (`--checkout /code/upstream/`) is no declared leg. Any other tree keeps the lane's `dir`, as before. A leg that is not checked out here (never initialized, or -deinitialized) names no tree. A leg with a `.git` of its own that git cannot read is -`unreadable`, the sweep's word: one `- UNREADABLE LEG: …` line in WRITERS and a note on +deinitialized) names no tree. A leg with a `.git` of its own that git cannot read, or +a leg directory this account cannot search (#255), is `unreadable`, the sweep's word: +one `- UNREADABLE LEG: …` line in WRITERS and a note on stderr, and counted on stderr's `legs: N of the assembly's legs under unreadable` line, never in WRITERS' `(N found)`. The lane is not held `SWAPPING` for the leg itself. A polled tree that could then be of that leg's repository is one it cannot @@ -4240,8 +4251,9 @@ as a writer, just as the checkout's own registrations elsewhere are not; beside what the checkout holds. Every worktree git registers in a checked-out leg that no sidecar names is a `TREE` row: `unmanaged`, or `stale-registration` where its directory is gone, naming the leg. A checkout in the paired root that nothing else named -is `unmanaged`. A leg with a `.git` of its own that git cannot read is a `TREE` row -`unreadable` that requires recovery: the sweep's rule and word for its `leg` row, which +is `unmanaged`. A leg with a `.git` of its own that git cannot read, or a leg directory +this account cannot search, is a `TREE` row `unreadable` that requires recovery: +the sweep's rule and word for its `leg` row, which it counts. The verdict is not `indeterminate` for it, so the sweep still lists and counts that leg. Nothing is adopted, deleted or overwritten. @@ -4257,6 +4269,12 @@ cannot be listed, or a leg's directory under a feature cannot be searched. For t same directories, and where the lane's inventory cannot be read (which paired trees it names would then be unknown), `lane-handoff` gives the same three signs: a `- NOT POLLED: …` line in WRITERS, a note on stderr, and the lane left `SWAPPING`. +So does an inventory that holds a record that is not a whole schema-1 one (#255): +`lane-trees` gives such a record its id, its schema (``, ``, an +unknown one, or #252's ``) and any path it keeps, and the note names each +one by those three. It is never read as a record, as the sweep never reads one, so a +paired tree that only it names is not taken as this lane's writer, and the handoff +files no record over it. **A checkout with no `project.yaml` is asked nothing**, and one whose `project.yaml` names no leg but itself is asked only `project-legs`, which answers 8. The handoff's output, top block and inventory, and the reconciliation, are byte for diff --git a/docs/triad-lane-tooling-analysis.md b/docs/triad-lane-tooling-analysis.md index 0dfa34f2..d8aa606f 100644 --- a/docs/triad-lane-tooling-analysis.md +++ b/docs/triad-lane-tooling-analysis.md @@ -696,7 +696,7 @@ landing. format. `resume` records the tree it rebuilds with `lanes-edit.sh set-lane-tree --checkout /`, which covers both the parked and the resumed state. Both are workBenches changes, to be filed as one workBenches issue - that cites this item and is linked from the T018 PR. **workBenches issue: to be + that cites this item and is linked from the T018 PR. **workBenches issue: opensoft/workBenches#166, filed and linked.** The PR does not wait for it. Until it lands, `docs/README-lanes.md` (T018 section) says what to check before `lane-end`. 12. **The paired root in `lane-handoff` (T018, the pre-PR review's M1).** **Ruled diff --git a/lane-handoff b/lane-handoff index 3505e726..e1a43cee 100755 --- a/lane-handoff +++ b/lane-handoff @@ -2138,7 +2138,7 @@ poll_writer() { # return 0 } -# ------------------------------------------------ 3a. A TRIAD'S LEGS (T018) +# ------------------------------------------------ 3b. A TRIAD'S LEGS (T018) # # opensoft/openRepoTools#186, plan task T018. THE LANE'S CHECKOUT IS AN # ASSEMBLY when its own `project.yaml` declares a leg other than itself — the @@ -2233,13 +2233,27 @@ fi # poll looked everywhere it looks. A tree this handoff then cannot place is # another matter (`writer_checkout`): it is named, not recorded, and the lane # stays SWAPPING for it, as for any tree no checkout can be vouched for. +# +# SO IS A LEG DIRECTORY THIS ACCOUNT CANNOT SEARCH (#255, openRepoShape-2's +# SHOULD on #246): whether a `.git` stands in it cannot be tested, and the +# `.git` test fails there exactly as for a leg that is not checked out, so +# the leg was passed over IN SILENCE while an unpushed commit of this lane's +# could be in it. It takes the same path, in the same words as the sweep's +# `probe_leg` and `lane-reconcile`. A leg directory that is not there at all +# is still passed over (`-d`), as a leg that is not checked out. legs_unreadable=""; legs_unreadable_count=0 if [ "$legs_state" = known ]; then while IFS="$tab" read -r lg_repo lg_decl lg_abs; do [ -n "$lg_abs" ] && [ -d "$lg_abs" ] || continue lg_top="$(git -C "$lg_abs" rev-parse --show-toplevel 2>/dev/null TAB `, one a line: the top block writes each leg's line # once the poll has said whether any tree was withheld (N1), and @@ -2278,15 +2292,29 @@ paired_note_unread() { # # were read: which paired trees are already this lane's, and the checkout each # recorded tree was filed under (`writer_checkout`). ` TAB `, one # a line. A checkout with no `project.yaml` never asks it. +# +# A ROW IS A RECORD ONLY WHERE IT IS A WHOLE SCHEMA-1 ONE (#255): its schema, +# `lane-trees`' thirteenth field, is `1` - the schema `lane-worktrees` reads too +# (`_discover`). Every other row keeps no field but its id, its path where the +# file named one, and its schema (``, an unknown one, ``, and +# #252's ``), so it is no record of whose a tree is: read as one, it +# made the paired tree it names this lane's writer on the strength of a record +# nobody could read, and the poll then tried to file a record of its own for +# that tree - over a malformed one, which went SWAPPED in silence, or against +# one of another schema, which `set-lane-tree` refuses and this handoff then +# reported as a refused write. So such a row is NAMED, by its id, its schema and +# any path it keeps, and the lane stays SWAPPING for it, exactly as for an +# inventory that could not be read at all; it never reaches `paired_inv`. if [ "$legs_state" = known ]; then pi_out=""; pi_rc=0 pi_out="$(LANES_NO_FETCH=1 "$LANES_EDIT" lane-trees "$lane" 2>/dev/null = 2 && $2 != "" { print $2 "\t" $10 }')" - pi_bad="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && $2 == "" { printf "%s ", $1 }')" + paired_inv="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && $2 != "" && $13 == "1" { print $2 "\t" $10 }')" + pi_bad="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && ($2 == "" || $13 != "1") { + printf "%s%s (schema %s%s)", (n++ ? ", " : ""), $1, ($13 == "" ? "" : $13), ($2 == "" ? "" : ", path " $2) }')" [ -z "$pi_bad" ] || - paired_note_unread "this lane's worktree inventory holds record(s) that could not be read (${pi_bad% }), so a paired tree only they name could not be told from another lane's" ;; + paired_note_unread "this lane's worktree inventory holds record(s) that could not be read ($pi_bad), so a paired tree only they name could not be told from another lane's" ;; 8) : ;; *) paired_note_unread "this lane's worktree inventory could not be read (\`$LANES_EDIT lane-trees $lane\` exited $pi_rc), so a paired tree only it names could not be told from another lane's" ;; esac diff --git a/lane-worktrees b/lane-worktrees index 342b2955..35e2d0ea 100755 --- a/lane-worktrees +++ b/lane-worktrees @@ -2036,24 +2036,85 @@ def probe_leg(leg: Leg) -> None: """Is a checkout of its own there? An uninitialized leg is an empty directory of the assembly's, and git answers the ASSEMBLY there - which is no checkout of the leg's. A `.git` of its own that git cannot read is a - read that failed, never an absent leg.""" + read that failed, never an absent leg. + + SO IS A LEG DIRECTORY THIS ACCOUNT CANNOT SEARCH (#255, openRepoShape-2's + SHOULD on #246): whether a `.git` stands in it cannot be tested, and + `os.path.lexists` answers False for a path it cannot reach exactly as for + one that is not there - so the leg read `absent`, counted for nobody, and + `lane-end` could end a lane over an unpushed commit in it. The `.git` is + asked of `lstat` itself: only "no such file" (or "not a directory") is a + leg with no `.git`; any other failure is a read that failed, the same + `read_error` - `unreadable`, counted - in the words `lane-handoff` and + `lane-reconcile` use for it. A leg directory that is not there at all is + still `absent` (`leg.exists`).""" if not leg.exists: return proc = git(leg.path, "rev-parse", "--show-toplevel") if proc.returncode == 0 and real(proc.stdout.strip()) == leg.real: leg.checkout = True - elif os.path.lexists(os.path.join(leg.path, ".git")): - leg.read_error = ("it has a .git of its own and git could not read it: " - + (_first_line(proc.stderr) if proc.returncode != 0 else - f"git answers {proc.stdout.strip()} there")) + return + said = (_first_line(proc.stderr) if proc.returncode != 0 else + f"git answers {proc.stdout.strip()} there") + try: + os.lstat(os.path.join(leg.path, ".git")) + except (FileNotFoundError, NotADirectoryError): + return + except OSError: + leg.read_error = ("its directory cannot be searched, so whether it has a .git of its " + "own could not be tested, and git could not read it: " + said) + return + leg.read_error = "it has a .git of its own and git could not read it: " + said + + +def probe_uninitialized(leg: Leg) -> None: + """A dependency `git submodule status` lists as `-`: `read_error` is set + where its directory, or one above it, cannot be searched, and `checkout` + where it holds a checkout of its own that git can read (#255). + `probe_leg` alone cannot see a dependency beneath a directory this + account cannot search: `Leg.exists` is `os.path.isdir`, which answers + False for a path it cannot reach exactly as for one that is not there, so + it would read `absent`. The path is asked of `stat` itself: only "no such + file" (or "not a directory") is a dependency that is not there, which sets + nothing; any other failure is a read that failed. A directory that is + there is `probe_leg`'s: with no search it is unreadable through its + `lstat` of the `.git`; holding a checkout git can read - a clone made by + hand, which git does not count as initialized - it is a checkout; empty + and searchable, it is a dependency never initialized and sets nothing.""" + try: + os.stat(leg.path) + except (FileNotFoundError, NotADirectoryError): + return + except OSError as err: + leg.read_error = ("a directory above it cannot be searched, so whether it is there " + f"could not be tested: {err.strerror or err}") + return + probe_leg(leg) def assembly_checkouts(helper, assembly: str) -> tuple: """([Leg], declared, why-not) - every leg `/project.yaml` - declares, and every dependency nested in one (the initialized submodules - `git submodule status --recursive` names in each leg), in that order; - `declared` is whether the manifest declares any leg at all, which is what - makes `/worktrees` the paired root.""" + declares, and every dependency nested in one (the submodules + `git submodule status --recursive` names in each leg: the initialized + ones, and a `-` line only as below), in that order; `declared` is whether + the manifest declares any leg at all, which is what makes + `/worktrees` the paired root. + + A DEPENDENCY GIT LISTS AS NOT INITIALIZED (A `-` LINE) IS NOT ALWAYS + NOTHING TO READ (#255, lane 3's review). Git lists a dependency whose + directory it cannot search - mode 000, or read and no search, or one + beneath a directory it cannot search - as `-` too, and exits 0, exactly as + for one never initialized. Skipped unread, it got no row, the sweep + exited 0, and `lane-end` ended the lane over that dependency's unpushed + commit. So a `-` line is probed (`probe_uninitialized`) and gets a row + where the probe is a read that failed, or where its directory holds a + checkout git can read - a clone made by hand, which git does not count as + initialized, and which holds the lane's work all the same: it is read like + an initialized one. A directory that is not there, or an empty one that + can be searched, is a dependency never initialized and still adds none. + The path on a `-` line is the whole of the rest of the line: git prints a + describe suffix only on a ` ` or `+` line (`print_status` in + builtin/submodule--helper.c), so none is trimmed from it.""" rows, why = project_legs(helper, assembly) if why: return [], False, why @@ -2069,17 +2130,25 @@ def assembly_checkouts(helper, assembly: str) -> tuple: return out, True, (f"the dependencies nested in the {decl} leg ({path}) could not be " "listed (git submodule status failed): " + _first_line(sub.stderr)) for line in sub.stdout.splitlines(): - if not line.strip() or line.startswith("-"): - continue # declared and not initialized: nothing to read + if not line.strip(): + continue parts = line[1:].split(" ", 1) if len(parts) < 2: continue rel = parts[1] - if rel.endswith(")") and " (" in rel: + if not line.startswith("-") and rel.endswith(")") and " (" in rel: rel = rel[:rel.rindex(" (")] nested = Leg(os.path.normpath(os.path.join(path, rel)), f"{decl}/{rel}", "", assembly, nested_in=decl) - probe_leg(nested) + if line.startswith("-"): + # Declared and not initialized: nothing to read - unless git + # could not search its directory, which it lists the same way, + # or a checkout of its own stands there (a hand clone). + probe_uninitialized(nested) + if not (nested.read_error or nested.checkout): + continue + else: + probe_leg(nested) out.append(nested) return out, bool(rows), "" diff --git a/lanes-edit.sh b/lanes-edit.sh index 7e537c5a..72b74e41 100755 --- a/lanes-edit.sh +++ b/lanes-edit.sh @@ -14194,12 +14194,24 @@ EOF # TREE row `unreadable`, which requires recovery, as the sweep's `leg` row is # `unreadable` and counted. It is NO `indeterminate`: the sweep counts that # leg and goes on (exit 3), and a verdict it refuses on would hide the count. + # SO IS A LEG DIRECTORY THIS ACCOUNT CANNOT SEARCH (#255, openRepoShape-2's + # SHOULD on #246): whether a `.git` stands in it cannot be tested, and the + # `.git` test fails there exactly as for a leg that is not checked out, so it + # was passed over in silence. It is the same `unreadable` row, in the words + # of the sweep's `probe_leg` and `lane-handoff`; a leg directory that is not + # there at all is still passed over (`-d`). if [ "$lrc_lgrc" = 0 ]; then while IFS="$lrc_tab" read -r lrc_lrepo lrc_ldecl lrc_labs; do [ -n "$lrc_labs" ] && [ -d "$lrc_labs" ] || continue lrc_ltop="$(git -C "$lrc_labs" rev-parse --show-toplevel 2>/dev/null None: + """A commit of triad-1's, on a branch no origin holds, in the dependency + nested in the code leg (`code/upstream/dep`).""" + dep = t["dep"] + b.git("checkout", "-q", "-b", "in-dep", cwd=dep) + b.commit(dep, "worked in the nested dependency", {"n.md": "n\n"}, lane="triad-1") + + +@NEEDS_UPSTREAM +def test_255_a_nested_dependency_directory_that_cannot_be_searched_is_unreadable_and_holds_lane_end( + triad): + """#255, LANE 3'S REVIEW (finding 1): `git submodule status --recursive` + lists a nested dependency whose directory this account cannot search as + `-` - not initialized - and exits 0, and `assembly_checkouts` skipped every + `-` line before it probed anything. The dependency got no row, the code + leg read `clean`, the dry run exited 0 and `lane-end` ENDED the lane over + the lane's unpushed commit in it - what `probe_leg` stopped for a declared + leg. A `-` line is now probed, and the dependency is `unreadable` and + counted, with no permission at all and with read but no search; the sweep + exits 3 and `lane-end` refuses, naming it. Searchable again, it is the + dependency holding the lane's commit.""" + b, t = triad + if os.geteuid() == 0: + pytest.skip("root searches a directory whatever its mode") + dep = t["dep"] + _work_in_the_nested_dependency(b, t) + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert proc.returncode == 3, proc.stdout + proc.stderr + leg = porcelain_rows(proc.stdout, "leg")[str(dep)] + assert leg["disposition"] == "unpushed" and leg["retire"] == "retire", leg + for mode in (0, 0o600): + os.chmod(dep, mode) + try: + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + end = b.tool("lane-end", "triad-1") + finally: + os.chmod(dep, 0o755) + assert proc.returncode == 3, (oct(mode), proc.stdout + proc.stderr) + legs = porcelain_rows(proc.stdout, "leg") + assert list(legs) == [str(t["spec"]), str(t["code"]), str(dep)], (oct(mode), legs) + leg = legs[str(dep)] + assert leg["disposition"] == "unreadable" and leg["retire"] == "retire", (oct(mode), leg) + assert f"a dependency nested in the code leg (code/upstream/dep): {UNSEARCHABLE}" \ + in leg["why"], (oct(mode), leg) + assert end.returncode == 2, (oct(mode), end.stdout + end.stderr) + assert "lane triad-1 is NOT ended" in end.stderr and str(dep) in end.stderr, end.stderr + assert "1 leg checkout(s)" in end.stderr, end.stderr + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert proc.returncode == 3, proc.stdout + proc.stderr + leg = porcelain_rows(proc.stdout, "leg")[str(dep)] + assert leg["disposition"] == "unpushed" and leg["retire"] == "retire", leg + + +@NEEDS_UPSTREAM +def test_255_a_nested_dependency_beneath_a_directory_that_cannot_be_searched_holds_lane_end( + triad): + """The same, with the dependency's PARENT (`code/upstream`) unsearchable + and the dependency itself left as it is. Git lists it as `-` and exits 0 + here too, and `Leg.exists` - `os.path.isdir` - is False for a path it cannot + reach exactly as for one that is not there, so `probe_leg` alone would read + it `absent`. It holds whichever way git answers: the dependency is an + `unreadable` row (the sweep exits 3), or - where `git submodule status` + itself fails - the sweep refuses to list the dependencies nested in the + leg (exit 2); either way `lane-end` refuses and the lane is NOT ended.""" + b, t = triad + if os.geteuid() == 0: + pytest.skip("root searches a directory whatever its mode") + dep = t["dep"] + parent = dep.parent + assert parent == t["code"] / "upstream" + _work_in_the_nested_dependency(b, t) + for mode in (0, 0o600): + os.chmod(parent, mode) + try: + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + end = b.tool("lane-end", "triad-1") + finally: + os.chmod(parent, 0o755) + assert proc.returncode in (2, 3), (oct(mode), proc.stdout + proc.stderr) + if proc.returncode == 3: + leg = porcelain_rows(proc.stdout, "leg")[str(dep)] + assert leg["disposition"] == "unreadable" and leg["retire"] == "retire", (oct(mode), leg) + assert "a dependency nested in the code leg (code/upstream/dep): git could not be " \ + "read through it (" in leg["why"], (oct(mode), leg) + else: + assert "could not be listed (git submodule status failed)" in proc.stdout, proc.stdout + assert end.returncode == 2, (oct(mode), end.stdout + end.stderr) + assert "lane triad-1 is NOT ended" in end.stderr, end.stderr + assert "nothing of lane triad-1's is left on disk" not in end.stderr, end.stderr + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert porcelain_rows(proc.stdout, "leg")[str(dep)]["disposition"] == "unpushed", proc.stdout + + +@NEEDS_UPSTREAM +def test_255_a_nested_dependency_never_initialized_adds_no_row(triad): + """A dependency the code leg declares and nobody initialized is a `-` + line git prints for it exactly as for an unreadable one, and it is still + nothing to read: with its directory empty and searchable - or not there at + all - the sweep has no row for it and exits 0, and `lane-end` passes. Only + a read that FAILED is a row (#255); an uninitialized dependency is not one.""" + b, t = triad + code, dep = t["code"], t["dep"] + b.git("submodule", "deinit", "-q", "-f", "upstream/dep", cwd=code) + sub = b.run("git", "-C", code, "submodule", "status", "--recursive") + assert sub.returncode == 0 and sub.stdout.startswith("-"), sub.stdout + sub.stderr + assert dep.is_dir() and not any(dep.iterdir()) + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert proc.returncode == 0, proc.stdout + proc.stderr + legs = porcelain_rows(proc.stdout, "leg") + assert list(legs) == [str(t["spec"]), str(code)], legs + assert str(dep) not in proc.stdout, proc.stdout + dep.rmdir() + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert list(porcelain_rows(proc.stdout, "leg")) == [str(t["spec"]), str(code)], proc.stdout + assert str(dep) not in proc.stdout, proc.stdout + dep.mkdir() + end = b.tool("lane-end", "triad-1") + assert end.returncode == 0, end.stdout + end.stderr + assert "close-out gate: nothing of lane triad-1's is left on disk" in end.stderr, end.stderr + + +@NEEDS_UPSTREAM +def test_255_a_nested_dependency_git_lists_as_not_initialized_that_holds_a_checkout_is_read(triad): + """#255, LANE 3'S RULING ON THE REVIEW'S FINDING 1: a dependency git lists + as `-` whose directory holds a checkout `probe_leg` can read - a clone made + by hand into the directory of a deinitialized dependency, which git does + not count as initialized - holds the lane's work all the same. It was + skipped with every `-` line, so the sweep exited 0 and `lane-end` ENDED + the lane over the lane's unpushed commit there. It is read like an + initialized dependency now: the sweep has a row for it and exits 3, and + `lane-end` refuses. An empty directory still adds none (the test above).""" + b, t = triad + code, dep = t["code"], t["dep"] + b.git("submodule", "deinit", "-q", "-f", "upstream/dep", cwd=code) + b.git("clone", "-q", b.remotes / "triad-dep.git", dep) + sub = b.run("git", "-C", code, "submodule", "status", "--recursive") + assert sub.returncode == 0 and sub.stdout.startswith("-"), sub.stdout + sub.stderr + b.git("checkout", "-q", "-b", "in-dep", cwd=dep) + b.commit(dep, "worked in a hand-cloned dependency", {"n.md": "n\n"}, lane="triad-1") + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert proc.returncode == 3, proc.stdout + proc.stderr + legs = porcelain_rows(proc.stdout, "leg") + assert list(legs) == [str(t["spec"]), str(code), str(dep)], legs + assert legs[str(dep)]["disposition"] == "unpushed" and legs[str(dep)]["retire"] == "retire", legs + assert legs[str(dep)]["branch"] == "in-dep", legs + end = b.tool("lane-end", "triad-1") + assert end.returncode == 2, end.stdout + end.stderr + assert "lane triad-1 is NOT ended" in end.stderr and str(dep) in end.stderr, end.stderr + assert "1 leg checkout(s)" in end.stderr, end.stderr + + +@NEEDS_UPSTREAM +def test_255_a_nested_dependency_whose_path_ends_in_a_parenthesis_is_read_by_that_path(triad): + """git prints a describe suffix, ` (v1)`, only on a ` ` or `+` line + (`print_status`), never on a `-` line, so nothing is trimmed from the + path of a `-` line. A dependency never initialized at `upstream/dep (1)` + ends in what the trim would cut: read as `upstream/dep`, it became a second + row for the initialized dependency beside it and left `dep (1)` unread - + here at mode 000, where it must be the `unreadable` row.""" + if os.geteuid() == 0: + pytest.skip("root searches a directory whatever its mode") + b, t = triad + code, dep = t["code"], t["dep"] + odd = code / "upstream" / "dep (1)" + b.git("submodule", "add", "-q", b.remotes / "triad-dep.git", "upstream/dep (1)", cwd=code) + b.git("submodule", "deinit", "-q", "-f", "upstream/dep (1)", cwd=code) + sub = b.run("git", "-C", code, "submodule", "status", "--recursive") + assert any(ln.startswith("-") and ln.endswith(" upstream/dep (1)") + for ln in sub.stdout.splitlines()), sub.stdout + assert odd.is_dir() and not any(odd.iterdir()) + + def leg_paths(proc): + return [ln.split("\t")[3] for ln in proc.stdout.splitlines() if ln.startswith("leg\t")] + + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert leg_paths(proc) == [str(t["spec"]), str(code), str(dep)], proc.stdout + os.chmod(odd, 0) + try: + proc = b.sweep("triad-1", "--dry-run", "--porcelain") + finally: + os.chmod(odd, 0o755) + assert proc.returncode == 3, proc.stdout + proc.stderr + assert leg_paths(proc) == [str(t["spec"]), str(code), str(dep), str(odd)], proc.stdout + leg = porcelain_rows(proc.stdout, "leg")[str(odd)] + assert leg["disposition"] == "unreadable" and leg["retire"] == "retire", leg + assert "a dependency nested in the code leg (code/upstream/dep (1))" in leg["why"], leg + + @NEEDS_UPSTREAM def test_t018_dependencies_a_leg_nests_that_cannot_be_listed_refuse_the_sweep(triad): """THE SWEEP'S OWN REFUSAL, UNSHADOWED: `git submodule status --recursive` diff --git a/tests/test_triad_lane_tooling_handoff.py b/tests/test_triad_lane_tooling_handoff.py index eee65abd..28578298 100644 --- a/tests/test_triad_lane_tooling_handoff.py +++ b/tests/test_triad_lane_tooling_handoff.py @@ -39,6 +39,7 @@ import difflib import os +import shlex import shutil import subprocess import sys @@ -789,6 +790,274 @@ def test_t018_a_leg_tree_add_made_is_withheld_where_the_legs_git_is_an_empty_dir assert inventory(b, "triad-1") == want +# ====================================================================== #255 + +@NEEDS_UPSTREAM +def test_255_a_leg_directory_that_cannot_be_searched_is_unreadable_in_the_handoff_and_reconcile(triad): + """#255, OPENREPOSHAPE-2'S SHOULD ON #246: A LEG DIRECTORY THIS ACCOUNT + CANNOT SEARCH failed the handoff's and the reconciliation's `.git` test + as a leg that is not checked out does, so both passed over it IN SILENCE: + no UNREADABLE LEG line, no TREE row, and the tree `add --checkout spec` + made was re-recorded under the ASSEMBLY, over the leg `add` had recorded. + Now it takes the path of a leg git cannot read, in the sweep's words, with + no permission at all and with read but no search: named and counted, its + tree withheld `(NOT recorded)` and its record kept, the lane SWAPPING, and + one `unreadable` TREE row that requires recovery. Searchable again, it + names nothing and its tree is recorded under it.""" + b, t = triad + if os.geteuid() == 0: + pytest.skip("root searches a directory whatever its mode") + a, spec = t["assembly"], t["spec"] + made = b.tool("lane-worktrees", "add", "triad-1", "s1", "--checkout", spec) + assert made.returncode == 0, made.stderr + lane_spec = Path(made.stdout.strip()) + made = b.tool("lane-worktrees", "add", "triad-1", "root") + assert made.returncode == 0, made.stderr + lane_root = Path(made.stdout.strip()) + want = {str(lane_spec): str(spec), str(lane_root): str(a)} + assert inventory(b, "triad-1") == want + words = (f"the spec leg of {a}/project.yaml (opensoft/triad-spec): git could not be read " + "through it (its directory cannot be searched, so whether it has a .git of its own " + "could not be tested, and git could not read it: ") + for mode in (0, 0o600): + os.chmod(spec, mode) + try: + proc = handoff(b, "triad-1", a, "a leg directory that cannot be searched") + rec = b.tool("lanes-edit.sh", "lane-reconcile", "triad-1") + finally: + os.chmod(spec, 0o755) + assert proc.returncode == 0, (oct(mode), proc.stdout + proc.stderr) + assert f"the leg {spec} is unreadable: {words}" in proc.stderr, (oct(mode), proc.stderr) + assert f"legs: 1 of the assembly's legs under {a} unreadable" in proc.stderr, proc.stderr + assert f"writers polled: 2 under {a}" in proc.stderr, proc.stderr + assert f"WRITER {lane_spec} — branch s1" in proc.stdout, proc.stdout + assert (f"the worktree inventory entry for {lane_spec} was NOT written: the assembly has " + "1 unreadable leg(s)") in proc.stderr, proc.stderr + assert f"the worktree inventory entry for {lane_root} was NOT written" not in proc.stderr + assert "lifecycle: SWAPPING" in proc.stdout, (oct(mode), proc.stdout) + writers = top_block(b, "triad-1").split("**WRITERS at ", 1)[1] + assert f"- UNREADABLE LEG: `{spec}` is unreadable, {words}" in writers, (oct(mode), writers) + assert writers.count("- UNREADABLE LEG: ") == 1, writers + assert f"- `{lane_spec}` (NOT recorded) — branch `s1`" in writers, writers + assert f"- `{lane_root}` — branch `root`" in writers, writers + assert inventory(b, "triad-1") == want + assert rec.returncode == 0, rec.stderr + rows = reconcile_rows(rec.stdout) + assert rows[real(spec)][0] == "unreadable", (oct(mode), rows) + assert rows[real(spec)][1].startswith(words), (oct(mode), rows[real(spec)]) + assert field(rec.stdout, "TREES")[2] == "1 require recovery", (oct(mode), rec.stdout) + assert field(rec.stdout, "VERDICT")[0] != "indeterminate", rec.stdout + + proc = handoff(b, "triad-1", a, "searchable again") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert f"the leg {spec} is unreadable" not in proc.stderr, proc.stderr + assert "UNREADABLE LEG" not in top_block(b, "triad-1") + assert "lifecycle: SWAPPED" in proc.stdout, proc.stdout + assert inventory(b, "triad-1") == want + rec = b.tool("lanes-edit.sh", "lane-reconcile", "triad-1") + assert real(spec) not in reconcile_rows(rec.stdout), rec.stdout + assert field(rec.stdout, "TREES")[2] == "0 require recovery", rec.stdout + + +def lone_sidecar(b: Bench, lane: str) -> Path: + """The lane's one inventory record, as `set-lane-tree` filed it.""" + files = sorted((b.projects / ".lane-state" / lane / "trees").glob("*.yaml")) + assert len(files) == 1, files + return files[0] + + +def trees_row(b: Bench, lane: str, path: Path, env: dict | None = None) -> list: + """The `lane-trees` row whose path is `path`, as the handoff's `LANES_EDIT` gives it.""" + helper = (env or {}).get("LANES_EDIT", str(b.bin / "lanes-edit.sh")) + proc = b.run(helper, "lane-trees", lane, env=env) + assert proc.returncode == 0, proc.stderr + rows = [ln.split(US) for ln in proc.stdout.splitlines() if ln.split(US)[1:2] == [str(path)]] + assert len(rows) == 1, proc.stdout + return rows[0] + + +def a_paired_tree_a_broken_record_names(b: Bench, t: dict) -> tuple: + """A paired spec tree with a writer's first edit and nothing committed - so + no trailer finds it - recorded in triad-1's inventory with its leg as its + checkout: (the tree, its record, the record's bytes).""" + a, spec = t["assembly"], t["spec"] + fresh = leg_tree(b, spec, a / "worktrees" / "050-r" / "spec", "050-r") + (fresh / "draft.md").write_text("a writer's first edit\n") + rec = b.tool("lanes-edit.sh", "set-lane-tree", "triad-1", fresh, "--checkout", spec) + assert rec.returncode == 0, rec.stderr + sidecar = lone_sidecar(b, "triad-1") + text = sidecar.read_text() + assert text.startswith("schema: 1\n") and f"\npath: {fresh}\n" in text, text + return fresh, sidecar, text + + +def assert_the_record_is_named_and_not_polled(b: Bench, t: dict, proc, fresh: Path, + sidecar: Path, before: bytes, schema: str, + path: str | None = None) -> None: + """THE HANDOFF NAMES A RECORD IT COULD NOT READ (#255): its id, schema and + path in the NOT POLLED note, on stderr and in the top block; the tree that + record names is not taken as this lane's writer; nothing is filed over the + record; and the lane stays SWAPPING. `path` is the path the record keeps + (the tree, unless told otherwise); `""` is a record that keeps none, which + the note then names by its id and schema alone.""" + a = t["assembly"] + kept = str(fresh) if path is None else path + note = (f"this lane's worktree inventory holds record(s) that could not be read " + f"({sidecar.stem} (schema {schema}{', path ' + kept if kept else ''})), so a paired " + "tree only they name could not be told from another lane's") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert note in proc.stderr, proc.stderr + assert f"- NOT POLLED: {note}" in top_block(b, "triad-1"), top_block(b, "triad-1") + assert "lifecycle: SWAPPING" in proc.stdout, proc.stdout + assert f"WRITER {fresh} —" not in proc.stdout, proc.stdout + assert f"PAIRED ROOT, not this lane's: {fresh} — branch 050-r" in proc.stdout, proc.stdout + assert f"writers polled: 0 under {a}" in proc.stderr, proc.stderr + assert sidecar.read_bytes() == before, sidecar.read_text() + + +@NEEDS_UPSTREAM +@pytest.mark.parametrize("schema", ["2", "10"]) +def test_255_an_unknown_schema_record_is_named_in_the_handoffs_not_polled_note(triad, schema): + """#255: THE INVENTORY'S UNREADABLE RECORDS WERE ONLY THE ONES WITH NO + PATH. A record of another schema keeps its path and nothing else + (`lane-trees`' `${ltl_s:-}` row), so the handoff read it as a record + with an empty checkout: its paired tree became this lane's writer, and the + poll's record for it was refused by `set-lane-tree`, which writes over no + record of another schema - reported as a refused write, with nothing said + of the record. Now a row whose schema - the thirteenth field - is not `1` + is named in the NOT POLLED note and never read as a record, as the sweep + reads it (`_discover`). Whole again, the record makes the tree this + lane's writer, and the lane is SWAPPED. THE SCHEMA IS MATCHED WHOLE: `10` + begins with a `1` but is not schema 1, so a rule that matched the schema by + its first character would read this record as a whole schema-1 one.""" + b, t = triad + a, spec = t["assembly"], t["spec"] + fresh, sidecar, text = a_paired_tree_a_broken_record_names(b, t) + sidecar.write_text(f"schema: {schema}\n" + text[len("schema: 1\n"):]) + before = sidecar.read_bytes() + row = trees_row(b, "triad-1", fresh) + assert row[0] == sidecar.stem and row[12] == schema and row[9] == "", row + proc = handoff(b, "triad-1", a, "a record of another schema") + assert_the_record_is_named_and_not_polled(b, t, proc, fresh, sidecar, before, schema) + + sidecar.write_text(text) + proc = handoff(b, "triad-1", a, "a whole record again") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert f"WRITER {fresh} — branch 050-r" in proc.stdout, proc.stdout + assert "NOT POLLED" not in proc.stdout + proc.stderr + assert "lifecycle: SWAPPED" in proc.stdout, proc.stdout + assert inventory(b, "triad-1") == {str(fresh): str(spec)} + + +@NEEDS_UPSTREAM +def test_255_a_malformed_record_is_named_in_the_handoffs_not_polled_note(triad): + """#255, AND #252'S `` ROW: a schema-1 record that is not a + whole one keeps its id, its path and the schema `` + (`lanes-edit.sh` at 36520b9, `lane_trees_list`). Here the record is cut to + its first four lines, and `LANES_EDIT` is a `lanes-edit.sh` that passes + every call through but answers `lane-trees` with #252's row for it, so + this holds before #252 lands and after. The handoff names it in the NOT + POLLED note, takes its tree for no writer of this lane's, files nothing + over it, and leaves the lane SWAPPING.""" + b, t = triad + a, spec = t["assembly"], t["spec"] + fresh, sidecar, text = a_paired_tree_a_broken_record_names(b, t) + sidecar.write_text("".join(text.splitlines(keepends=True)[:4])) + before = sidecar.read_bytes() + fake = b.root / "fake-lanes-edit.sh" + fake.write_text( + "#!/bin/sh\n" + f"real={shlex.quote(str(b.bin / 'lanes-edit.sh'))}\n" + "if [ \"${1-}\" = lane-trees ]; then\n" + " out=\"$(\"$real\" \"$@\")\"; rc=$?\n" + " [ \"$rc\" = 0 ] || { [ -z \"$out\" ] || printf '%s\\n' \"$out\"; exit \"$rc\"; }\n" + f" printf '%s\\n' \"$out\" | P={shlex.quote(str(fresh))} awk -F'\\037' '$2 != ENVIRON[\"P\"]'\n" + f" printf '%s\\037%s\\037\\037\\037\\037\\037\\037\\037\\037\\037\\037\\037\\n' " + f"{shlex.quote(sidecar.stem)} {shlex.quote(str(fresh))}\n" + " exit 0\n" + "fi\n" + "exec \"$real\" \"$@\"\n") + fake.chmod(0o755) + env = {"LANES_EDIT": str(fake)} + row = trees_row(b, "triad-1", fresh, env) + assert row[0] == sidecar.stem and len(row) == 13 and row[12] == "", row + assert not any(row[2:12]), row + proc = b.tool("lane-handoff", "--lane", "triad-1", "--transcript", "none", "--agent", "claude", + "a malformed record", cwd=a, env=env) + assert_the_record_is_named_and_not_polled(b, t, proc, fresh, sidecar, before, "") + + sidecar.write_text(text) + proc = handoff(b, "triad-1", a, "a whole record again") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert f"WRITER {fresh} — branch 050-r" in proc.stdout, proc.stdout + assert "NOT POLLED" not in proc.stdout + proc.stderr + assert "lifecycle: SWAPPED" in proc.stdout, proc.stdout + assert inventory(b, "triad-1") == {str(fresh): str(spec)} + + +@NEEDS_UPSTREAM +def test_255_a_record_with_a_tree_id_and_no_path_is_named_in_the_handoffs_not_polled_note(triad): + """#255, `pi_bad`'S EMPTY-PATH ARM: a record that files a tree id and no + path - #219's path-less tree - is a row with an empty path. A row with no + path names no tree, so it is no record of whose a tree is, whatever its + schema: before #252 the real `lanes-edit.sh` prints it as a WHOLE schema-1 + row (every field but the path), which only the empty-path rule names; with + #252 it is ``, which the schema rule names as well. Either way + the handoff names it by its id and its schema - there is no path to name - + takes the tree beside it for no writer of this lane's, files nothing over + the record, and leaves the lane SWAPPING. Whole again, the record makes the + tree this lane's writer.""" + b, t = triad + a, spec = t["assembly"], t["spec"] + fresh, sidecar, text = a_paired_tree_a_broken_record_names(b, t) + no_path = "".join(ln for ln in text.splitlines(keepends=True) if not ln.startswith("path: ")) + assert no_path != text and "\npath:" not in no_path, no_path + sidecar.write_text(no_path) + before = sidecar.read_bytes() + lt = b.tool("lanes-edit.sh", "lane-trees", "triad-1") + assert lt.returncode == 0, lt.stderr + rows = [ln.split(US) for ln in lt.stdout.splitlines() if ln.split(US)[0] == sidecar.stem] + assert len(rows) == 1, lt.stdout + row = rows[0] + assert row[1] == "" and row[12] in ("1", ""), row + proc = handoff(b, "triad-1", a, "a record with no path") + assert_the_record_is_named_and_not_polled(b, t, proc, fresh, sidecar, before, row[12], path="") + + sidecar.write_text(text) + proc = handoff(b, "triad-1", a, "a whole record again") + assert proc.returncode == 0, proc.stdout + proc.stderr + assert f"WRITER {fresh} — branch 050-r" in proc.stdout, proc.stdout + assert "NOT POLLED" not in proc.stdout + proc.stderr + assert "lifecycle: SWAPPED" in proc.stdout, proc.stdout + assert inventory(b, "triad-1") == {str(fresh): str(spec)} + + +@NEEDS_UPSTREAM +def test_255_an_unreadable_record_is_named_in_the_handoffs_not_polled_note(triad): + """#255: a record file this account cannot read is a row of its own, named + by its file (`lane-trees`' ``, and no path). The handoff names + it in the NOT POLLED note - by its id and its schema, there being no path - + takes the tree beside it for no writer of this lane's, files nothing over + it and leaves the lane SWAPPING.""" + b, t = triad + if os.geteuid() == 0: + pytest.skip("root reads a mode-000 file") + a = t["assembly"] + fresh, sidecar, text = a_paired_tree_a_broken_record_names(b, t) + before = sidecar.read_bytes() + sidecar.chmod(0) + try: + lt = b.tool("lanes-edit.sh", "lane-trees", "triad-1") + proc = handoff(b, "triad-1", a, "an unreadable record") + finally: + sidecar.chmod(0o644) + assert lt.returncode == 0, lt.stderr + rows = [ln.split(US) for ln in lt.stdout.splitlines() if ln.split(US)[0] == sidecar.stem] + assert len(rows) == 1 and rows[0][1] == "" and rows[0][12] == "", lt.stdout + assert_the_record_is_named_and_not_polled(b, t, proc, fresh, sidecar, before, "", + path="") + + # ===================================================================== #280 #: A `lanes-edit.sh` that is the bench's own for every verb but `lane-state`, and