The handoff names every inventory record that is not a whole schema-1 one, and a leg directory this account cannot search, or a dependency nested in one, is unreadable and never absent (fixes #255)

docs/README-lanes.md

@@ -4147,9 +4147,19 @@ one passed `lane-end`.
 
 * reads every leg, and every dependency nested in one (an initialized submodule of
   the leg's own, as `code/upstream/openRepoShape` will be), as a **`leg` row** —
-  `clean`, `dirty`, `unpushed`, `dirty+unpushed`, `absent` (not checked out here)
-  or `unreadable`. A leg is the assembly's, shared by every lane the assembly
-  homes, as the assembly's own checkout is: **no `--yes` acts on a leg
+  `clean`, `dirty`, `unpushed`, `dirty+unpushed`, `absent` (not checked out here,
+  or not there at all) or `unreadable` (a `.git` of its own that git cannot read,
+  or a leg directory this account cannot search, #255). A dependency nested in a
+  leg that git lists as not initialized has no row where its directory is missing,
+  or empty and searchable: it was never initialized. But git lists a dependency it
+  cannot search as not initialized too, so where its directory, or one above it,
+  cannot be searched (or it holds a `.git` that git cannot read), it is an
+  `unreadable` row, counted: a read that failed is never a clean dependency (#255).
+  One whose directory holds a checkout that git can read (a clone made by hand,
+  which git does not count as initialized) is read like an initialized one: it
+  holds the lane's work all the same.
+  A leg is the assembly's, shared by every lane the assembly homes, as the
+  assembly's own checkout is: **no `--yes` acts on a leg
   checkout** (its branches are another matter: see `--branches` below). It is
   COUNTED (exit 3, and `lane-end`'s gate) only where its HEAD's own `Lane:` trailer
   is this lane's and it holds edits or commits origin lacks — the rule the sweep
@@ -4217,8 +4227,9 @@ leg's repository (a paired tree, or one `lane-worktrees add --checkout
 checkout of the tree's own repository keeps it: a dependency nested in a leg
 (`--checkout <assembly>/code/upstream/<dep>`) is no declared leg. Any other tree keeps
 the lane's `dir`, as before. A leg that is not checked out here (never initialized, or
-deinitialized) names no tree. A leg with a `.git` of its own that git cannot read is
-`unreadable`, the sweep's word: one `- UNREADABLE LEG: …` line in WRITERS and a note on
+deinitialized) names no tree. A leg with a `.git` of its own that git cannot read, or
+a leg directory this account cannot search (#255), is `unreadable`, the sweep's word:
+one `- UNREADABLE LEG: …` line in WRITERS and a note on
 stderr, and counted on stderr's `legs: N of the assembly's legs under <dir> unreadable`
 line, never in WRITERS' `(N found)`. The lane is not held `SWAPPING` for the leg
 itself. A polled tree that could then be of that leg's repository is one it cannot
@@ -4240,8 +4251,9 @@ as a writer, just as the checkout's own registrations elsewhere are not;
 beside what the checkout holds. Every worktree git registers in a checked-out leg that
 no sidecar names is a `TREE` row: `unmanaged`, or `stale-registration` where its
 directory is gone, naming the leg. A checkout in the paired root that nothing else named
-is `unmanaged`. A leg with a `.git` of its own that git cannot read is a `TREE` row
-`unreadable` that requires recovery: the sweep's rule and word for its `leg` row, which
+is `unmanaged`. A leg with a `.git` of its own that git cannot read, or a leg directory
+this account cannot search, is a `TREE` row `unreadable` that requires recovery:
+the sweep's rule and word for its `leg` row, which
 it counts. The verdict is not `indeterminate` for it, so the sweep still lists and
 counts that leg. Nothing is adopted, deleted or overwritten.
 
@@ -4257,6 +4269,12 @@ cannot be listed, or a leg's directory under a feature cannot be searched. For t
 same directories, and where the lane's inventory cannot be read (which paired trees it
 names would then be unknown), `lane-handoff` gives the same three signs: a
 `- NOT POLLED: …` line in WRITERS, a note on stderr, and the lane left `SWAPPING`.
+So does an inventory that holds a record that is not a whole schema-1 one (#255):
+`lane-trees` gives such a record its id, its schema (`<unreadable>`, `<none>`, an
+unknown one, or #252's `<malformed>`) and any path it keeps, and the note names each
+one by those three. It is never read as a record, as the sweep never reads one, so a
+paired tree that only it names is not taken as this lane's writer, and the handoff
+files no record over it.
 **A checkout with no `project.yaml` is asked nothing**, and one whose
 `project.yaml` names no leg but itself is asked only `project-legs`, which answers 8.
 The handoff's output, top block and inventory, and the reconciliation, are byte for

docs/triad-lane-tooling-analysis.md

@@ -696,7 +696,7 @@ landing.
     format. `resume` records the tree it rebuilds with `lanes-edit.sh set-lane-tree
     <lane> <path> --checkout <assembly>/<leg>`, which covers both the parked and the
     resumed state. Both are workBenches changes, to be filed as one workBenches issue
-    that cites this item and is linked from the T018 PR. **workBenches issue: to be
+    that cites this item and is linked from the T018 PR. **workBenches issue: opensoft/workBenches#166, filed and
     linked.** The PR does not wait for it. Until it lands, `docs/README-lanes.md` (T018
     section) says what to check before `lane-end`.
 12. **The paired root in `lane-handoff` (T018, the pre-PR review's M1).** **Ruled

lane-handoff

@@ -2138,7 +2138,7 @@ poll_writer() {   # <path>
 	return 0
 }
 
-# ------------------------------------------------ 3a. A TRIAD'S LEGS (T018)
+# ------------------------------------------------ 3b. A TRIAD'S LEGS (T018)
 #
 # opensoft/openRepoTools#186, plan task T018. THE LANE'S CHECKOUT IS AN
 # ASSEMBLY when its own `project.yaml` declares a leg other than itself — the
@@ -2233,21 +2233,35 @@ fi
 # poll looked everywhere it looks. A tree this handoff then cannot place is
 # another matter (`writer_checkout`): it is named, not recorded, and the lane
 # stays SWAPPING for it, as for any tree no checkout can be vouched for.
+#
+# SO IS A LEG DIRECTORY THIS ACCOUNT CANNOT SEARCH (#255, openRepoShape-2's
+# SHOULD on #246): whether a `.git` stands in it cannot be tested, and the
+# `.git` test fails there exactly as for a leg that is not checked out, so
+# the leg was passed over IN SILENCE while an unpushed commit of this lane's
+# could be in it. It takes the same path, in the same words as the sweep's
+# `probe_leg` and `lane-reconcile`. A leg directory that is not there at all
+# is still passed over (`-d`), as a leg that is not checked out.
 legs_unreadable=""; legs_unreadable_count=0
 if [ "$legs_state" = known ]; then
 	while IFS="$tab" read -r lg_repo lg_decl lg_abs; do
 		[ -n "$lg_abs" ] && [ -d "$lg_abs" ] || continue
 		lg_top="$(git -C "$lg_abs" rev-parse --show-toplevel 2>/dev/null </dev/null || :)"
 		if [ -z "$lg_top" ] || [ "$(physical_of "$lg_top")" != "$(physical_of "$lg_abs")" ]; then
-			{ [ -e "$lg_abs/.git" ] || [ -L "$lg_abs/.git" ]; } || continue
+			if [ -e "$lg_abs/.git" ] || [ -L "$lg_abs/.git" ]; then
+				lg_what="it has a .git of its own and git could not read it"
+			elif [ ! -x "$lg_abs" ]; then
+				lg_what="its directory cannot be searched, so whether it has a .git of its own could not be tested, and git could not read it"
+			else
+				continue
+			fi
 			if [ -n "$lg_top" ]; then
 				lg_err="git answers $lg_top there"
 			else
 				lg_err="$(git -C "$lg_abs" rev-parse --show-toplevel 2>&1 >/dev/null </dev/null \
 					| awk 'NF { sub(/^[ \t]+/, ""); sub(/[ \t]+$/, ""); print; exit }')"
 				lg_err="${lg_err:-no answer}"
 			fi
-			lg_why="the $lg_decl leg of $dir/project.yaml${lg_repo:+ ($lg_repo)}: git could not be read through it (it has a .git of its own and git could not read it: $lg_err)"
+			lg_why="the $lg_decl leg of $dir/project.yaml${lg_repo:+ ($lg_repo)}: git could not be read through it ($lg_what: $lg_err)"
 			legs_unreadable_count=$((legs_unreadable_count + 1))
 			# `<leg> TAB <why>`, one a line: the top block writes each leg's line
 			# once the poll has said whether any tree was withheld (N1), and
@@ -2278,15 +2292,29 @@ paired_note_unread() {   # <why>
 # were read: which paired trees are already this lane's, and the checkout each
 # recorded tree was filed under (`writer_checkout`). `<path> TAB <checkout>`, one
 # a line. A checkout with no `project.yaml` never asks it.
+#
+# A ROW IS A RECORD ONLY WHERE IT IS A WHOLE SCHEMA-1 ONE (#255): its schema,
+# `lane-trees`' thirteenth field, is `1` - the schema `lane-worktrees` reads too
+# (`_discover`). Every other row keeps no field but its id, its path where the
+# file named one, and its schema (`<none>`, an unknown one, `<unreadable>`, and
+# #252's `<malformed>`), so it is no record of whose a tree is: read as one, it
+# made the paired tree it names this lane's writer on the strength of a record
+# nobody could read, and the poll then tried to file a record of its own for
+# that tree - over a malformed one, which went SWAPPED in silence, or against
+# one of another schema, which `set-lane-tree` refuses and this handoff then
+# reported as a refused write. So such a row is NAMED, by its id, its schema and
+# any path it keeps, and the lane stays SWAPPING for it, exactly as for an
+# inventory that could not be read at all; it never reaches `paired_inv`.
 if [ "$legs_state" = known ]; then
 	pi_out=""; pi_rc=0
 	pi_out="$(LANES_NO_FETCH=1 "$LANES_EDIT" lane-trees "$lane" 2>/dev/null </dev/null)" || pi_rc=$?
 	case "$pi_rc" in
 	0)
-		paired_inv="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && $2 != "" { print $2 "\t" $10 }')"
-		pi_bad="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && $2 == "" { printf "%s ", $1 }')"
+		paired_inv="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && $2 != "" && $13 == "1" { print $2 "\t" $10 }')"
+		pi_bad="$(printf '%s\n' "$pi_out" | awk -F'\037' 'NF >= 2 && ($2 == "" || $13 != "1") {
+			printf "%s%s (schema %s%s)", (n++ ? ", " : ""), $1, ($13 == "" ? "<none>" : $13), ($2 == "" ? "" : ", path " $2) }')"
 		[ -z "$pi_bad" ] ||
-			paired_note_unread "this lane's worktree inventory holds record(s) that could not be read (${pi_bad% }), so a paired tree only they name could not be told from another lane's" ;;
+			paired_note_unread "this lane's worktree inventory holds record(s) that could not be read ($pi_bad), so a paired tree only they name could not be told from another lane's" ;;
 	8) : ;;
 	*) paired_note_unread "this lane's worktree inventory could not be read (\`$LANES_EDIT lane-trees $lane\` exited $pi_rc), so a paired tree only it names could not be told from another lane's" ;;
 	esac

lane-worktrees

@@ -2036,24 +2036,85 @@ def probe_leg(leg: Leg) -> None:
     """Is a checkout of its own there? An uninitialized leg is an empty
     directory of the assembly's, and git answers the ASSEMBLY there - which is
     no checkout of the leg's. A `.git` of its own that git cannot read is a
-    read that failed, never an absent leg."""
+    read that failed, never an absent leg.
+
+    SO IS A LEG DIRECTORY THIS ACCOUNT CANNOT SEARCH (#255, openRepoShape-2's
+    SHOULD on #246): whether a `.git` stands in it cannot be tested, and
+    `os.path.lexists` answers False for a path it cannot reach exactly as for
+    one that is not there - so the leg read `absent`, counted for nobody, and
+    `lane-end` could end a lane over an unpushed commit in it. The `.git` is
+    asked of `lstat` itself: only "no such file" (or "not a directory") is a
+    leg with no `.git`; any other failure is a read that failed, the same
+    `read_error` - `unreadable`, counted - in the words `lane-handoff` and
+    `lane-reconcile` use for it. A leg directory that is not there at all is
+    still `absent` (`leg.exists`)."""
     if not leg.exists:
         return
     proc = git(leg.path, "rev-parse", "--show-toplevel")
     if proc.returncode == 0 and real(proc.stdout.strip()) == leg.real:
         leg.checkout = True
-    elif os.path.lexists(os.path.join(leg.path, ".git")):
-        leg.read_error = ("it has a .git of its own and git could not read it: "
-                          + (_first_line(proc.stderr) if proc.returncode != 0 else
-                             f"git answers {proc.stdout.strip()} there"))
+        return
+    said = (_first_line(proc.stderr) if proc.returncode != 0 else
+            f"git answers {proc.stdout.strip()} there")
+    try:
+        os.lstat(os.path.join(leg.path, ".git"))
+    except (FileNotFoundError, NotADirectoryError):
+        return
+    except OSError:
+        leg.read_error = ("its directory cannot be searched, so whether it has a .git of its "
+                          "own could not be tested, and git could not read it: " + said)
+        return
+    leg.read_error = "it has a .git of its own and git could not read it: " + said
+
+
+def probe_uninitialized(leg: Leg) -> None:
+    """A dependency `git submodule status` lists as `-`: `read_error` is set
+    where its directory, or one above it, cannot be searched, and `checkout`
+    where it holds a checkout of its own that git can read (#255).
+    `probe_leg` alone cannot see a dependency beneath a directory this
+    account cannot search: `Leg.exists` is `os.path.isdir`, which answers
+    False for a path it cannot reach exactly as for one that is not there, so
+    it would read `absent`. The path is asked of `stat` itself: only "no such
+    file" (or "not a directory") is a dependency that is not there, which sets
+    nothing; any other failure is a read that failed. A directory that is
+    there is `probe_leg`'s: with no search it is unreadable through its
+    `lstat` of the `.git`; holding a checkout git can read - a clone made by
+    hand, which git does not count as initialized - it is a checkout; empty
+    and searchable, it is a dependency never initialized and sets nothing."""
+    try:
+        os.stat(leg.path)
+    except (FileNotFoundError, NotADirectoryError):
+        return
+    except OSError as err:
+        leg.read_error = ("a directory above it cannot be searched, so whether it is there "
+                          f"could not be tested: {err.strerror or err}")
+        return
+    probe_leg(leg)
 
 
 def assembly_checkouts(helper, assembly: str) -> tuple:
     """([Leg], declared, why-not) - every leg `<assembly>/project.yaml`
-    declares, and every dependency nested in one (the initialized submodules
-    `git submodule status --recursive` names in each leg), in that order;
-    `declared` is whether the manifest declares any leg at all, which is what
-    makes `<assembly>/worktrees` the paired root."""
+    declares, and every dependency nested in one (the submodules
+    `git submodule status --recursive` names in each leg: the initialized
+    ones, and a `-` line only as below), in that order; `declared` is whether
+    the manifest declares any leg at all, which is what makes
+    `<assembly>/worktrees` the paired root.
+
+    A DEPENDENCY GIT LISTS AS NOT INITIALIZED (A `-` LINE) IS NOT ALWAYS
+    NOTHING TO READ (#255, lane 3's review). Git lists a dependency whose
+    directory it cannot search - mode 000, or read and no search, or one
+    beneath a directory it cannot search - as `-` too, and exits 0, exactly as
+    for one never initialized. Skipped unread, it got no row, the sweep
+    exited 0, and `lane-end` ended the lane over that dependency's unpushed
+    commit. So a `-` line is probed (`probe_uninitialized`) and gets a row
+    where the probe is a read that failed, or where its directory holds a
+    checkout git can read - a clone made by hand, which git does not count as
+    initialized, and which holds the lane's work all the same: it is read like
+    an initialized one. A directory that is not there, or an empty one that
+    can be searched, is a dependency never initialized and still adds none.
+    The path on a `-` line is the whole of the rest of the line: git prints a
+    describe suffix only on a ` ` or `+` line (`print_status` in
+    builtin/submodule--helper.c), so none is trimmed from it."""
     rows, why = project_legs(helper, assembly)
     if why:
         return [], False, why
@@ -2069,17 +2130,25 @@ def assembly_checkouts(helper, assembly: str) -> tuple:
             return out, True, (f"the dependencies nested in the {decl} leg ({path}) could not be "
                                "listed (git submodule status failed): " + _first_line(sub.stderr))
         for line in sub.stdout.splitlines():
-            if not line.strip() or line.startswith("-"):
-                continue           # declared and not initialized: nothing to read
+            if not line.strip():
+                continue
             parts = line[1:].split(" ", 1)
             if len(parts) < 2:
                 continue
             rel = parts[1]
-            if rel.endswith(")") and " (" in rel:
+            if not line.startswith("-") and rel.endswith(")") and " (" in rel:
                 rel = rel[:rel.rindex(" (")]
             nested = Leg(os.path.normpath(os.path.join(path, rel)), f"{decl}/{rel}", "",
                          assembly, nested_in=decl)
-            probe_leg(nested)
+            if line.startswith("-"):
+                # Declared and not initialized: nothing to read - unless git
+                # could not search its directory, which it lists the same way,
+                # or a checkout of its own stands there (a hand clone).
+                probe_uninitialized(nested)
+                if not (nested.read_error or nested.checkout):
+                    continue
+            else:
+                probe_leg(nested)
             out.append(nested)
     return out, bool(rows), ""