From a93dfd882ad0035a7739f904a35feed9c315137e Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Wed, 21 Oct 2026 13:30:44 +0000 Subject: [PATCH 01/53] Amendment 23 S6.2: `lane-end --sweep-checkouts` lists the register checkouts CLOSED lanes left under `-lanes/`, and every clone beside W, in a dry run that removes nothing (#198, slice 4 stage 1) Stage 1 of slice 4: the flag, its refusals, the candidate list and the dry run. - `--sweep-checkouts` is its own act, answered before the lane derivation like `--retire-dormant`. It refuses a lane, every other act's flag, `--yes` with no `LANES_LANE`, `--yes` under `LANES_NO_FETCH=1` (S3 refuses a writer whose fetch did not run, rv1b's MUST 3), and a W that stands in a `*-lanes` directory (as `lanes_home_check` does: from a lane window every other lane's checkout would stand "beside W"). - A candidate is a clone of the register directly under `-lanes/`, at its own real path with a `.git` directory of its own, whose lane `lanes --closed --all` classes `closed`, or a former name's checkout beside its new one (S1). Never W, a LIVE lane's checkout, the runner's own, the checkout a caller's `LANES_REPO` names, a link, a non-clone or a hidden entry. - The clones beside W are listed and never candidates: this build removes nothing outside `-lanes/` (the brief's rule, kept against S6.2's wider one until lane openRepoTools-3 rules). - The dry run exits 3 when there is a candidate and 0 when there is none. `--yes` is refused in this commit; the holds and the removal are stage 2. Tests: tests/test_a23_sweep_checkouts.py, 16 cases on `tests/a23_box.py`'s RegisterBox, in #202's `ended` style. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit faac2e7b90cec371780ec785cb513e24f2b7b0b6) --- lane-end | 415 +++++++++++++++++++++++++++++- tests/test_a23_sweep_checkouts.py | 301 ++++++++++++++++++++++ 2 files changed, 711 insertions(+), 5 deletions(-) create mode 100644 tests/test_a23_sweep_checkouts.py diff --git a/lane-end b/lane-end index dc803e2..a2dac9a 100755 --- a/lane-end +++ b/lane-end @@ -36,6 +36,10 @@ # # the DORMANT rows of one repository, # # dry run by default; --yes is ONE # # commit (Amendment 19(c)) +# lane-end --sweep-checkouts [--yes] # the register checkouts under +# # -lanes/ that CLOSED lanes left, +# # dry run by default; --yes removes +# # those nothing holds (Amendment 23 S6.2) # lane-end [options] # the split form works too # lane-end [options] # = basename of the cwd's git root # @@ -166,9 +170,27 @@ # worktrees, branches and handoffs stay exactly where they # are. An agent's stdin is not a terminal, so `--yes` is a # word a PERSON types. +# --sweep-checkouts A FOURTH ACT, and it ends no lane: the sweep of register +# checkouts lane-collision-protocol Amendment 23 S6.2 +# leaves to this command. Each lane writes the register +# through its own clone, `-lanes//` (S1), and a +# lane that has CLOSED (Amendment 19) leaves its clone +# behind, as an old name leaves one beside its new name +# after a rename. Without `--yes` this is a DRY RUN that +# lists every one of them with what holds it, and every +# register clone BESIDE W (a per-profile clone) with what +# it holds; it fetches nothing and changes nothing. +# With `--yes`, run by a PERSON and naming the lane that +# runs it in `LANES_LANE`, it removes each clone under +# `-lanes/` that nothing holds, leaves first, and +# records what it removed in one `NOTED` line of that +# lane's log. A clone beside W is listed and never +# removed by this build, and nothing outside +# `-lanes/` is ever removed # --reason "" the words the sweep writes into each RETIRED line and # into each row's state cell. `--retire-dormant` only -# --yes make the sweep the one act. `--retire-dormant` only +# --yes make the sweep the act. `--retire-dormant` and +# `--sweep-checkouts` only # --dry-run print what would be written; write nothing # -h, --help # @@ -240,10 +262,17 @@ # 1 environment — no register, no writer, `lanes-edit.sh lane-objects` # failed (which is NOT the same as the lane having no object log), or the # close-out gate could not be read (no `lane-worktrees`, or an exit that -# is none of its three) +# is none of its three). `--sweep-checkouts`: no W, a read that failed +# (the listing of lanes, or a fetch `--yes` decides on), or a removal +# whose NOTED line could not be written # 2 refusal — bad arguments, no row (or two), something still in flight, or # the close-out gate: the lane's trees, branches, scratch, caches or -# residue still on disk, or the sweep refusing the lane +# residue still on disk, or the sweep refusing the lane. +# `--sweep-checkouts`: `--yes` with no `LANES_LANE`, or with +# `LANES_NO_FETCH=1`; a W that is itself a lane checkout; a runner lane +# the register does not know +# 3 `--sweep-checkouts`, the dry run: something under `-lanes/` is +# removable (0 is nothing to remove) # 8 nothing there. `--retire-dormant`: no lane of that repository is # recorded, or none of its rows is DORMANT. `--retire `: nothing # of any kind is live for the lane @@ -533,6 +562,9 @@ LANES_REG_DEFAULT=0 if [ -z "${LANES_FILE:-}" ] || [ -z "${LANES_REPO:-}" ]; then LANES_WS_ROOT="$(lanes_workspace_root || :)" fi +# The checkout a caller's own `LANES_REPO` names, before the default below +# fills it in: `--sweep-checkouts` never removes it (Amendment 23 S6.2). +LE_REPO_GIVEN="${LANES_REPO:-}" : "${LANES_REPO:=$LANES_WS_ROOT}" LANES_FILE="${LANES_FILE:-${LANES_REPO:+$LANES_REPO/lanes/LANES.md}}" @@ -581,7 +613,7 @@ usage() { sed -n -e '1,2d' -e '/^#/!q' -e 's/^# \{0,1\}//' -e p "$RESOLVED"; } # ------------------------------------------------------------------ arguments retire=0; force=0; dry_run=0; state_was=""; home_override=""; retire_target="" -retire_dormant=""; sweep_reason=""; sweep_yes=0; inventory_only=0 +retire_dormant=""; sweep_reason=""; sweep_yes=0; inventory_only=0; sweep_checkouts=0 positional=() # A TRANSCRIPT UUID AND A PID, TOLD APART FROM A LANE ARGUMENT BY SHAPE. @@ -651,6 +683,9 @@ while [ $# -gt 0 ]; do # which is not built — `lane-worktrees sweep --yes` is that act, on its own # — so the word is refused by name rather than read as an unknown option. --inventory-only) inventory_only=1; shift ;; + # AMENDMENT 23 S6.2 — THE REGISTER CHECKOUTS' SWEEP, its own act. It takes + # no value, so `--sweep-checkouts=` is an unknown option below. + --sweep-checkouts) sweep_checkouts=1; shift ;; --sweep | --sweep=*) die "--sweep is not built: retiring a lane's trees in the same run as its ending is not one act. \`lane-worktrees sweep --yes\` retires them first, on its own (it removes only what is on origin or in a bundle, never a live writer's tree), and then $prog ends the lane; or $prog --inventory-only ends the lane with its trees left where they are, in its inventory. Nothing was written." 2 ;; --dry-run) dry_run=1; shift ;; @@ -679,6 +714,7 @@ done # itself before it writes a byte. if [ -n "$retire_dormant" ]; then [ "$retire" = 0 ] || die "--retire and --retire-dormant are two different acts: --retire is a lane's own ending (or a live fork's), and --retire-dormant sweeps a repository's rows that have no lane left to end. Run one of them." 2 + [ "$sweep_checkouts" = 0 ] || die "--retire-dormant and --sweep-checkouts are two different sweeps: one retires a repository's dormant ROWS in the register, the other removes the register CHECKOUTS that closed lanes left on disk. Run one of them. Nothing was written." 2 [ "$force" = 0 ] || die "--force ends a lane that still holds something; it has no meaning for a sweep of rows whose lanes ended before Amendment 7. Nothing was written." 2 [ "$inventory_only" = 0 ] || die "--inventory-only is the door past ONE lane's close-out gate; the sweep of dormant rows ends no lane and asks no gate. Nothing was written." 2 [ -z "$state_was" ] || die "--state-was is for the state cell of a lane being ended; the sweep writes each row's cell itself, from the reason and the row's own last words. Nothing was written." 2 @@ -866,6 +902,375 @@ EOF exit $? fi +# ---- AMENDMENT 23 S6.2: `--sweep-checkouts`, THE REGISTER CHECKOUTS' SWEEP --- +# +# EVERY LANE WRITES THE REGISTER THROUGH A CLONE OF ITS OWN, `-lanes//` +# (S1), and S6.2 leaves the clones a CLOSED lane no longer writes through to +# this command: *"The sweep has Amendment 19's shape — a dry run, then a +# person's `--yes`"*. Like `--retire-dormant` it is not a lane argument and it +# ends no lane, so it is answered here, before the `` derivation below. +# +# WHAT IT LOOKS AT. Every entry directly under `-lanes/` but the hidden ones +# (S1's creation locks and `.partial` clones, the workstation's sidecar lock), +# and every register clone BESIDE W: a directory in W's parent with a `.git` +# directory of its own whose origin is W's — the per-profile clones S6.2 names. +# +# WHAT IT MAY REMOVE, AND NOTHING ELSE: a clone of the register directly under +# `-lanes/`, at its own real path, that is the checkout of a lane the +# register calls CLOSED (`lanes --closed`'s class, Amendment 19), or the old +# name's checkout beside its new one after a rename (S1: *"the writer writes +# through `-lanes//` and reports `-lanes//`, which S6's sweep +# lists"*). NEVER W, never a LIVE lane's checkout, never the checkout of the +# lane that runs the sweep or of the `LANES_REPO` its caller names, never a +# symbolic link and never anything a link leads to. A CLONE BESIDE W IS LISTED, +# WITH WHAT IT HOLDS, AND NEVER REMOVED BY THIS BUILD: the brief this slice was +# written under says nothing outside `-lanes/` is removed, where S6.2 says +# the per-profile clones are swept too, and the stricter rule is kept until +# lane openRepoTools-3 rules between them. +if [ "$sweep_checkouts" = 1 ]; then + [ "$retire" = 0 ] || die "--retire and --sweep-checkouts are two different acts: --retire ends a lane (or names a live fork's act), and --sweep-checkouts removes the register checkouts closed lanes left on disk, ending no lane. Run one of them. Nothing was written." 2 + [ "$force" = 0 ] || die "--force ends a lane that still holds something; it has no meaning for the sweep of register checkouts, which removes only what nothing holds. Nothing was removed." 2 + [ "$inventory_only" = 0 ] || die "--inventory-only is the door past ONE lane's close-out gate; the sweep of register checkouts ends no lane and asks no gate. Nothing was removed." 2 + [ -z "$state_was" ] || die "--state-was is for the state cell of a lane being ended; the sweep of register checkouts writes no cell. Nothing was removed." 2 + [ -z "$home_override" ] || die "--home is for a pre-cutover lane's own log line; the sweep of register checkouts writes no home into anything. Nothing was removed." 2 + [ -z "$sweep_reason" ] || die "--reason belongs to --retire-dormant: the sweep of register checkouts writes its own NOTED line, naming what it removed. Nothing was removed." 2 + if [ "${#positional[@]}" -gt 0 ]; then + die "--sweep-checkouts takes no lane and no repository ('${positional[0]}'): it looks at every register checkout under -lanes/, and the lane that RUNS it is named in LANES_LANE. Nothing was removed." 2 + fi + [ "$dry_run" = 0 ] || sweep_yes=0 + # `--yes` IS A WRITE, AND ITS WRITER IS A LANE. The NOTED line that records + # what was removed carries the sweeping lane's own transcript uuid (Amendment + # 7(b)) and goes through that lane's own checkout (S2: *"A write across lanes' + # rows ... goes through the checkout of the lane that runs it"*), so the lane + # is named by the person running it or nothing happens. And a writer whose + # fetch did not run is a writer S3 refuses (ratified choice 2): asked here, + # before the first removal, rather than met by the record after it. + if [ "$sweep_yes" = 1 ]; then + [ -n "${LANES_LANE:-}" ] || + die "--sweep-checkouts --yes removes directories and writes one NOTED line into the log of the lane that runs it, through that lane's own checkout (Amendment 23 S2, S6.2), so this act needs to know which lane is doing it. Re-run it from your lane: LANES_LANE= $prog --sweep-checkouts --yes. Nothing was removed." 2 + [ "${LANES_NO_FETCH:-}" != 1 ] || + die "--sweep-checkouts --yes fetches each checkout before it decides about it, and its NOTED line is a write, which Amendment 23 S3 refuses when its fetch did not run (ratified choice 2) — and LANES_NO_FETCH=1 is set. Unset it and re-run. Nothing was removed." 2 + case "$LANES_LANE" in + *[!A-Za-z0-9._-]* | [!A-Za-z0-9]*) die "LANES_LANE='$LANES_LANE' is not a lane name (letters, digits, . _ -). Nothing was removed." 2 ;; + esac + fi + [ -x "$LANES_EDIT" ] || die "no register writer at $LANES_EDIT — \`lanes-edit.sh\` is installed beside this command by \`openRepoTools --install\`. Run that, or set LANES_EDIT" 1 + + sc_tab="$(printf '\t')" + sc_branch="${LANES_BRANCH:-main}" + case "$sc_branch" in + '' | -* | *[!A-Za-z0-9._/-]*) die "LANES_BRANCH='$sc_branch' is not a branch name this sweep compares against. Nothing was removed." 2 ;; + esac + sc_utc="$(date -u +%Y%m%dT%H%M%SZ)" + + # A DIRECTORY'S REAL PATH, or nothing: `cd -P` resolves every link on the way. + sc_real() { (CDPATH='' cd -P -- "$1" 2>/dev/null && pwd -P) || :; } + sc_lc() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]'; } + # A repository's git common directory, real, or nothing. `--git-common-dir` + # answers relative to `-C`'s directory on the gits that answer relatively. + sc_git_common() { # + scg_d="$(git -C "$1" rev-parse --git-common-dir 2>/dev/null)" || return 0 + case "$scg_d" in /*) : ;; *) scg_d="$1/$scg_d" ;; esac + sc_real "$scg_d" + } + + # W, THE READERS' COPY — the checkout whose `-lanes/` this sweeps, found + # the way every helper finds it (Amendment 9(a)), by its real path. + sc_w="$(lanes_workspace_root 2>/dev/null || :)" + [ -n "$sc_w" ] || die "$(lanes_workspace_why)" 1 + sc_w_real="$(sc_real "$sc_w")" + [ -n "$sc_w_real" ] || die "W, $sc_w, cannot be entered, so neither it nor its lane checkouts can be read. Nothing was removed." 1 + sc_w="$sc_w_real" + sc_root="${sc_w%/*}" + [ -n "$sc_root" ] || die "W resolves to $sc_w, at the top of the filesystem, so the directory beside it is the whole of it. The sweep reads no such directory. Nothing was removed." 2 + # A W THAT IS ITSELF A LANE CHECKOUT IS REFUSED, AS `lanes-edit.sh`'S + # `lanes_home_check` REFUSES IT (the review of #282, MUST 3). Eagle's lane + # windows export `LANES_WORKSPACE_ROOT` at the lane's own checkout, and from + # there "the clones beside W" would be every OTHER lane's checkout. + case "${sc_root##*/}" in + ?*-lanes) + if [ -n "${LANES_WORKSPACE_ROOT:-}" ]; then + die "LANES_WORKSPACE_ROOT is $LANES_WORKSPACE_ROOT, so W resolves to $sc_w — but it stands in $sc_root/, where a workstation keeps its lane checkouts: it is a lane checkout, not W (Amendment 23 S1, S5), and the clones beside it are other lanes' checkouts. Unset LANES_WORKSPACE_ROOT in this window and re-run. Nothing was removed." 2 + fi + die "$(lanes_ws_yaml) names path: $sc_w as W — but it stands in $sc_root/, where a workstation keeps its lane checkouts: it is a lane checkout, not W (Amendment 23 S1, S5). Point that \`path:\` at the workspace checkout itself. Nothing was removed." 2 ;; + esac + sc_home="$sc_w-lanes" + sc_worigin="$(git -C "$sc_w" remote get-url origin 2>/dev/null || :)" + [ -n "$sc_worigin" ] || die "W ($sc_w) names no origin, so no clone can be told to be the register's. Nothing was removed." 1 + sc_wgcd="$(sc_git_common "$sc_w")" + [ -n "$sc_wgcd" ] || die "W's git directory could not be read ($sc_w), so W cannot be told from a clone of it. Nothing was removed." 1 + + # A CLONE OF THE REGISTER, BY S1'S OWN TEST: a directory (never a symbolic + # link) at its own real path, the root of its checkout, with a `.git` + # DIRECTORY of its own (not a linked worktree's file, not a link), whose + # origin is W's, and which is not W. 0 for one; 1 with SC_WHY saying what is + # there instead. + SC_WHY="" + sc_is_register_clone() { # + scr_p="$1"; SC_WHY="" + if [ -L "$scr_p" ]; then + SC_WHY="a symbolic link, which this sweep never follows"; return 1 + fi + if [ ! -d "$scr_p" ]; then SC_WHY="not a directory"; return 1; fi + scr_r="$(sc_real "$scr_p")" + if [ "$scr_r" != "$scr_p" ]; then + SC_WHY="a directory whose real path is ${scr_r:-unreadable}, not the path it is listed at"; return 1 + fi + if [ -L "$scr_p/.git" ] || [ ! -d "$scr_p/.git" ]; then + SC_WHY="no .git directory of its own (a linked worktree, a link, or no checkout at all)"; return 1 + fi + scr_top="$(git -C "$scr_p" rev-parse --show-toplevel 2>/dev/null || :)" + if [ "$scr_top" != "$scr_p" ]; then + SC_WHY="not the root of a checkout (git names ${scr_top:-none})"; return 1 + fi + scr_gcd="$(sc_git_common "$scr_p")" + if [ "$scr_gcd" = "$sc_wgcd" ] || [ "$scr_p" = "$sc_w" ]; then + SC_WHY="W itself, the readers' copy"; return 1 + fi + if [ "$scr_gcd" != "$scr_p/.git" ]; then + SC_WHY="a checkout whose git directory is ${scr_gcd:-unreadable}, not its own .git"; return 1 + fi + scr_o="$(git -C "$scr_p" remote get-url origin 2>/dev/null || :)" + if ! lanes_ws_same_repo "$scr_o" "$sc_worigin"; then + SC_WHY="a checkout of ${scr_o:-no origin at all}, not of the register's origin"; return 1 + fi + return 0 + } + + # THE LANES, AS THE REGISTER AND THE LOGS HAVE THEM: `lanes --closed --all`, + # the one read that classes a lane CLOSED (Amendment 19), so this sweep and + # every listing can never disagree about which lanes those are. Column 2 is + # the state, LIVE where a live session record here names the lane, and + # column 14 the class. A dry run reads what W last fetched, as the listing + # does by default; `--yes` fetches first. ITS NOTES ARE READ, as + # `--retire-dormant` reads them: a listing that could not read this + # workstation's session records cannot say which lanes are LIVE, and one + # that could not read the register cannot say which are CLOSED, and neither + # is an answer (Amendment 7(d)). + sc_lanes_args=(lanes --closed --all) + if [ "$sweep_yes" = 1 ]; then sc_lanes_args+=(--fetch); fi + LE_TMP="$(mktemp "${TMPDIR:-/tmp}/lane-end-sweep.XXXXXX" 2>/dev/null || printf '')" + [ -n "$LE_TMP" ] || + die "no temporary file could be made to capture the listing's own notes, so whether this workstation's session records could be read — which is what says a lane is LIVE — was never established. Fix \$TMPDIR and re-run. Nothing was removed." 1 + sc_rc=0 + sc_rows="$("$LANES_EDIT" "${sc_lanes_args[@]}" 2>"$LE_TMP")" || sc_rc=$? + sc_err="$(cat -- "$LE_TMP")" || + die "the capture of the listing's own notes could not be read back, so whether the lanes' state was read was never established. Nothing was removed." 1 + rm -f -- "$LE_TMP"; LE_TMP="" + if [ -n "$sc_err" ]; then printf '%s\n' "$sc_err" >&2; fi + case "$sc_err" in + *"session records could not be read"*) + die "this workstation's session records could not be read (the listing says so above), so which lanes are LIVE was never established, and a LIVE lane's checkout is never swept. Fix the read and re-run. Nothing was removed." 1 ;; + *"could not be read, so its rows are not in this listing"*) + die "the register could not be read (the listing says so above), so which lanes are CLOSED was never established. Fix the read and re-run. Nothing was removed." 1 ;; + esac + if [ "$sweep_yes" = 1 ]; then + case "$sc_err" in + *"NO FETCH WAS MADE"*) + die "--yes decides on a register fetched just now, and W's fetch was not made (the listing says why above). Nothing was removed." 1 ;; + esac + fi + case "$sc_rc" in + 0 | 8) : ;; + 2) die "the \`lanes-edit.sh\` on this workstation has no \`lanes\` subcommand, so it predates lane-collision-protocol Amendment 11 and cannot say which lanes are CLOSED. This workstation has not taken the install yet: openRepoTools --install" 2 ;; + *) die "$LANES_EDIT ${sc_lanes_args[*]} exited $sc_rc, so which lanes are CLOSED or LIVE was never established. Run it by hand to see what it says. Nothing was removed." 1 ;; + esac + # ` TAB TAB TAB `, one per lane. + sc_index="" + while IFS="$sc_tab" read -r i_lane i_state i_ws i_pf i_win i_sid i_dir i_obj i_age i_cmd i_home i_fk i_loc i_class i_rest; do + [ -n "${i_lane:-}" ] || continue + sc_index="${sc_index}$(sc_lc "$i_lane")${sc_tab}${i_lane}${sc_tab}${i_state:-UNKNOWN}${sc_tab}${i_class:-none} +" + done < — 0 with SC_L_LANE/STATE/CLASS, 1 when no lane has it + scl_want="$(sc_lc "$1")"; SC_L_LANE=""; SC_L_STATE=""; SC_L_CLASS="" + while IFS="$sc_tab" read -r scl_lc scl_lane scl_state scl_class; do + [ -n "$scl_lc" ] && [ "$scl_lc" = "$scl_want" ] || continue + SC_L_LANE="$scl_lane"; SC_L_STATE="$scl_state"; SC_L_CLASS="$scl_class" + return 0 + done </dev/null)" || sc_cl_rc=$? + if [ "$sc_cl_rc" != 0 ] || [ -z "$sc_runner" ]; then + die "LANES_LANE=$LANES_LANE could not be resolved to a lane (\`$LANES_EDIT canon-lane $LANES_LANE\` exited $sc_cl_rc), so whose checkout is never swept, and whose log the NOTED line goes into, is not known. Nothing was removed." 2 + fi + if [ "$sweep_yes" = 1 ] && ! sc_lookup "$sc_runner"; then + die "lane $sc_runner, which LANES_LANE names, has no row in the register and no log (\`lanes --all\` does not list it), so the NOTED line this act ends with has no lane to be written for. Name your own lane: LANES_LANE= $prog --sweep-checkouts --yes. Nothing was removed." 2 + fi + fi + # THE CHECKOUT A CALLER'S `LANES_REPO` NAMES, by its real toplevel. Eagle's + # lane windows export one at the lane's own checkout, and a write may go + # through it; it is never swept. + sc_xrepo="" + if [ -n "$LE_REPO_GIVEN" ]; then + sc_xrepo="$(sc_real "$LE_REPO_GIVEN")" + if [ -n "$sc_xrepo" ]; then + sc_xtop="$(git -C "$sc_xrepo" rev-parse --show-toplevel 2>/dev/null || :)" + if [ -n "$sc_xtop" ]; then sc_xrepo="$(sc_real "$sc_xtop")"; fi + fi + fi + + # ------------------------------------------------ the entries, one by one + # + # Each is a CANDIDATE, kept apart with what it is judged as, or a KEEP line + # that says why it is not one. Nothing below reads a clone's history: that + # is what holds it, and the holds are asked of candidates alone. + sc_cands=(); sc_cand_why=(); sc_cand_lane=(); sc_cand_new=() + sc_keep_lines="" + sc_keep() { # + sc_keep_lines="${sc_keep_lines} keep $1 $2 +" + } + sc_cand() { # [] + sc_cands+=("$1"); sc_cand_lane+=("$2"); sc_cand_why+=("$3"); sc_cand_new+=("${4-}") + } + # A lane's checkout, judged by the lane's state: CLOSED is a candidate, + # LIVE never is, and every other state is a lane somebody may pick up. + sc_judge_lane() { # + if [ -n "$sc_runner" ] && [ "$(sc_lc "$2")" = "$(sc_lc "$sc_runner")" ]; then + sc_keep "$1" "lane $2 runs this sweep: its checkout is the one its NOTED line goes through" + return 0 + fi + if ! sc_lookup "$2"; then + sc_keep "$1" "no lane $2 is in the register or the logs, so it is no CLOSED lane's" + return 0 + fi + if [ "$SC_L_STATE" = LIVE ]; then + sc_keep "$1" "lane $SC_L_LANE is LIVE: a live session record here names it" + return 0 + fi + if [ "$SC_L_CLASS" != closed ]; then + sc_keep "$1" "lane $SC_L_LANE is $SC_L_STATE, not CLOSED (Amendment 19)" + return 0 + fi + sc_cand "$1" "$SC_L_LANE" "lane $SC_L_LANE is CLOSED ($SC_L_STATE)" + } + sc_judge() { # + sj_p="$1"; sj_n="$2" + if ! sc_is_register_clone "$sj_p"; then + sc_keep "$sj_p" "$SC_WHY"; return 0 + fi + if [ -n "$sc_xrepo" ] && [ "$sj_p" = "$sc_xrepo" ]; then + sc_keep "$sj_p" "the checkout LANES_REPO names, which a write of this window may go through" + return 0 + fi + case "$sj_n" in + *[!A-Za-z0-9._-]* | [!A-Za-z0-9]*) + sc_keep "$sj_p" "its name is no lane name (letters, digits, . _ -)"; return 0 ;; + esac + if sc_lookup "$sj_n"; then + sc_judge_lane "$sj_p" "$SC_L_LANE"; return 0 + fi + # NOT A LANE BY ITS OWN NAME: perhaps a FORMER name (Amendment 16(e)). + sj_rc=0 + sj_c="$(LANES_NO_FETCH=1 "$LANES_EDIT" canon-lane "$sj_n" 2>/dev/null)" || sj_rc=$? + if [ "$sj_rc" != 0 ] || [ -z "$sj_c" ]; then + sc_keep "$sj_p" "whether $sj_n is a former lane name could not be read (\`canon-lane $sj_n\` exited $sj_rc)" + return 0 + fi + if [ "$(sc_lc "$sj_c")" = "$(sc_lc "$sj_n")" ]; then + sc_keep "$sj_p" "no lane $sj_n is in the register or the logs, so it is no CLOSED lane's" + return 0 + fi + # A FORMER NAME OF LANE `sj_c`. Beside the new name's own checkout it is a + # candidate whatever that lane's state, because nothing writes through it + # any more (S1); with no checkout under the new name it IS the lane's + # checkout until the next write renames it (S1), and is judged as one. + sj_new="$sc_home/$sj_c" + if [ -d "$sj_new" ] && [ ! -L "$sj_new" ] && [ ! "$sj_new" -ef "$sj_p" ]; then + sc_cand "$sj_p" "$sj_c" "a former name of lane $sj_c, beside that lane's own checkout $sj_new" "$sj_new" + return 0 + fi + sc_judge_lane "$sj_p" "$sj_c" + } + + # `-lanes/` ITSELF must be a directory at its own real path: a link there + # would lead the sweep out of it, and nothing in it is then looked at. + sc_home_why=""; sc_hidden=0 + if [ -L "$sc_home" ]; then + sc_home_why="$sc_home is a symbolic link, which this sweep never follows: nothing in it is looked at" + elif [ ! -e "$sc_home" ]; then + sc_home_why="there is no $sc_home/: no lane checkout has been made on this workstation" + elif [ ! -d "$sc_home" ]; then + sc_home_why="$sc_home is not a directory: nothing in it is looked at" + elif [ "$(sc_real "$sc_home")" != "$sc_home" ]; then + sc_home_why="$sc_home resolves through a link to $(sc_real "$sc_home"), which this sweep never follows: nothing in it is looked at" + elif ! ls -A -- "$sc_home" >/dev/null 2>&1; then + die "$sc_home cannot be listed, so what is in it is unknown — which is not an empty directory. Nothing was removed." 1 + else + for sc_e in "$sc_home"/.[!.]* "$sc_home"/..?*; do + if [ -e "$sc_e" ] || [ -L "$sc_e" ]; then sc_hidden=$((sc_hidden + 1)); fi + done + for sc_e in "$sc_home"/*; do + if [ ! -e "$sc_e" ] && [ ! -L "$sc_e" ]; then continue; fi + sc_judge "$sc_e" "${sc_e##*/}" + done + fi + + # THE CLONES BESIDE W: listed, never removed by this build (above). + sc_beside=() + if ls -A -- "$sc_root" >/dev/null 2>&1; then + for sc_e in "$sc_root"/*; do + if [ -L "$sc_e" ] || [ ! -d "$sc_e" ]; then continue; fi + if [ "$sc_e" = "$sc_w" ] || [ "$sc_e" = "$sc_home" ]; then continue; fi + if [ ! -d "$sc_e/.git" ] || [ -L "$sc_e/.git" ]; then continue; fi + if sc_is_register_clone "$sc_e"; then sc_beside+=("$sc_e"); fi + done + else + note "$sc_root, the directory W stands in, cannot be listed, so no clone beside W is listed. Nothing outside $sc_home/ is ever removed." + fi + + # ------------------------------------------------------------ the report + printf '%s --sweep-checkouts — %s (lane-collision-protocol Amendment 23 S6.2)\n' \ + "$prog" "$( [ "$sweep_yes" = 1 ] && printf 'THE ACT' || printf 'DRY RUN, nothing is removed' )" + printf ' W : %s\n' "$sc_w" + printf ' lane checkouts : %s/\n' "$sc_home" + printf ' runner : %s\n' "${sc_runner:-none named (LANES_LANE is unset)}" + if [ -n "$sc_home_why" ]; then printf ' (%s)\n' "$sc_home_why"; fi + if [ "$sc_hidden" -gt 0 ]; then + printf ' (%s hidden entr(ies) under %s/ - creation locks, partial clones, the sidecar lock - are never looked at)\n' "$sc_hidden" "$sc_home" + fi + printf '\n under %s/:\n' "$sc_home" + sc_i=0 + while [ "$sc_i" -lt "${#sc_cands[@]}" ]; do + printf ' candidate %s %s\n' "${sc_cands[$sc_i]}" "${sc_cand_why[$sc_i]}" + sc_i=$((sc_i + 1)) + done + printf '%s' "$sc_keep_lines" + if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ]; then printf ' (nothing)\n'; fi + printf '\n beside W (Amendment 23 S6.2), listed and never removed by this build:\n' + for sc_e in ${sc_beside[@]+"${sc_beside[@]}"}; do + printf ' beside %s\n' "$sc_e" + done + if [ "${#sc_beside[@]}" = 0 ]; then printf ' (none)\n'; fi + printf '\n %s candidate(s) under %s/; %s clone(s) beside W listed.\n' \ + "${#sc_cands[@]}" "$sc_home" "${#sc_beside[@]}" + + if [ "$sweep_yes" != 1 ]; then + if [ "${#sc_cands[@]}" -gt 0 ]; then + printf '\n DRY RUN — nothing was removed. Make it the act with:\n' + printf ' LANES_LANE= %s --sweep-checkouts --yes\n' "$prog" + exit 3 + fi + printf '\n DRY RUN — nothing was removed, and there is nothing to remove.\n' + exit 0 + fi + die "--sweep-checkouts --yes is not built in this commit (Amendment 23 act 2, slice 4, stage 1 of 3: the holds and the removal are stage 2). Nothing was removed." 2 +fi + # THE SWEEP'S OWN FLAGS BELONG TO THE SWEEP (Copilot round 3 on #93). They are # parsed above for every invocation, and without this `lane-end --yes` # ran the ORDINARY ending — a line that reads like a confirmed sweep doing @@ -873,7 +1278,7 @@ fi # reason on the floor. Neither is a typo this command may interpret. if [ -z "$retire_dormant" ]; then [ "$sweep_yes" = 0 ] || - die "--yes is the word that makes the SWEEP the act (Amendment 19(c)) and it belongs to --retire-dormant. Ending one lane needs no confirmation and takes none: $prog . Nothing was written." 2 + die "--yes is the word that makes a SWEEP the act and it belongs to --retire-dormant (Amendment 19(c)) or --sweep-checkouts (Amendment 23 S6.2). Ending one lane needs no confirmation and takes none: $prog . Nothing was written." 2 [ -z "$sweep_reason" ] || die "--reason says why a repository's DORMANT ROWS are being retired (Amendment 19(c)) and it belongs to --retire-dormant; ending one lane writes its own closing line and would drop these words. Nothing was written: $prog , or $prog --retire-dormant --reason \"$sweep_reason\"." 2 fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py new file mode 100644 index 0000000..71a4837 --- /dev/null +++ b/tests/test_a23_sweep_checkouts.py @@ -0,0 +1,301 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Amendment 23 S6.2: `lane-end --sweep-checkouts`, the sweep of the register +checkouts lanes leave behind. + +Every lane writes the register through a clone of its own, `-lanes//` +(S1). A lane that has CLOSED (Amendment 19) leaves its clone behind, and so +does an old name beside its new one after a rename. The sweep has Amendment +19's shape: a dry run that lists them and removes nothing, then a person's +`--yes`, which needs `LANES_LANE`. The per-profile clones BESIDE W are listed +with what they hold and never removed by this build: nothing outside +`-lanes/` is ever removed. + +EVERY RUN HERE IS THE REAL `lane-end` AND THE REAL `lanes-edit.sh` IN A +SANDBOX: `tests/a23_box.py`'s `RegisterBox` (a bare origin, W cloned from it, +the `workspace.yaml` naming W, a `$HOME` of its own), with lanes in the +register and the logs, their checkouts under `-lanes/`, and three +per-profile clones beside W chained by alternates. The fixture is #202's +`ended` style (`tests/test_precision_90a_96.py`): one function that runs the +command and hands back what it said. EVERY PATH THIS MODULE COULD REMOVE IS +UNDER THE TEST'S OWN BASETEMP, and the fixture asserts that before any run. +""" + +from __future__ import annotations + +import os +import shutil +import subprocess +from pathlib import Path + +import pytest + +from a23_box import REGISTER, RegisterBox, register_text +from conftest import REPO, WINDOWS_SKIP + +pytestmark = [pytest.mark.skipif(shutil.which("bash") is None, + reason="lane-end is a bash script"), + pytest.mark.skipif(shutil.which("git") is None, + reason="the register is a git repository"), + WINDOWS_SKIP] + +RUNNER = "repoS4-1" # IDLE: the lane that runs the sweep +CLOSED = "repoS4-2" # ENDED: its checkout is the sweep's +LIVE = "repoS4-3" # a live session record here names it +PAUSED = "repoS4-4" # PAUSED: a lane somebody may pick up +OLD, NEW = "repoS4-8", "repoS4-9" # renamed: OLD is a former name of NEW +UTC = "2026-10-20T10:00:00Z" +UUID = {lane: f"a23a{n}04d-5404-4000-8000-a23a{n}04d5404" + for n, lane in ((1, RUNNER), (2, CLOSED), (3, LIVE), (4, PAUSED), (9, NEW))} +PROFILE_CLONES = ("brett-wip-team01-laneA", "brett-wip-team02-laneA", "brett-wip-team03-laneA") + + +def _row(lane: str, state: str) -> str: + return (f"| `{lane}` | harness `{UUID[lane]}` | Eagle / test / tester | 2026-10-20T00:00Z " + f"| — | handoffs/repoS4/{lane}.md | {state} |") + + +def _log(lane: str, *verbs: str) -> str: + head = f"# lane {lane} — object log (lane-collision-protocol Amendment 7)\n" + return head + "".join( + f"{verb} — lane {lane}, session {UUID[lane]}@Eagle, {UTC}, lane:{lane}\n" + for verb in verbs) + + +class Sweep(RegisterBox): + """W, `-lanes/` with a checkout for each lane, and the clones beside W. + + `-lanes/` holds: the runner's, the CLOSED lane's, the LIVE lane's and + the PAUSED lane's checkouts; OLD's beside NEW's; a directory that is no + checkout, a symbolic link and a hidden creation lock. Beside W: three + per-profile clones, each borrowing from the one before it and the first + from W (c3 → c2 → c1 → W), and a clone of another repository.""" + + def __init__(self, root: Path) -> None: + lanes = (RUNNER, CLOSED, LIVE, PAUSED, NEW) + files = { + REGISTER: register_text([_row(RUNNER, "LIVE · running the sweep"), + _row(CLOSED, "ENDED · closed"), + _row(LIVE, "LIVE · working"), + _row(PAUSED, "PAUSED · swapped"), + _row(NEW, "LIVE · renamed from repoS4-8")]), + f"lanes/log/{RUNNER}.md": _log(RUNNER, "STARTED"), + f"lanes/log/{CLOSED}.md": _log(CLOSED, "STARTED", "ENDED"), + f"lanes/log/{LIVE}.md": _log(LIVE, "STARTED"), + f"lanes/log/{PAUSED}.md": _log(PAUSED, "STARTED", "PAUSED"), + f"lanes/log/{NEW}.md": _log(NEW, "STARTED"), + "lanes/aliases.tsv": f"{OLD}\t{NEW}\t{UTC}\n", + "handoffs/README.md": "# handoffs\n", + } + super().__init__(root, files) + self.lanes_root.mkdir() + for lane in (*lanes, OLD): + self.clone(self.lanes_root / lane, reference=self.w) + (self.lanes_root / "notes").mkdir() + (self.lanes_root / "notes" / "todo.txt").write_text("not a clone\n", encoding="utf-8") + (self.lanes_root / "a-link").symlink_to(self.checkout(CLOSED)) + (self.lanes_root / f".{CLOSED}.lock").mkdir() + projects = self.w.parent + previous = self.w + self.profile_clones = [] + for name in PROFILE_CLONES: + previous = self.clone(projects / name, reference=previous) + self.profile_clones.append(previous) + other = root / "other.git" + self.git("init", "-q", "--bare", "-b", "main", str(other)) + self.clone(projects / "otherrepo", origin=other) + # THE LIVE LANE: a session record naming its transcript, held by a + # live process (`tests/test_lanes_index.py`'s shape), killed by PID. + self.sleeper = subprocess.Popen(["sleep", "600"], cwd=str(root)) + sessions = self.home / ".claude" / "sessions" + sessions.mkdir(parents=True, exist_ok=True) + (sessions / "live.json").write_text( + '{"pid":%d,"sessionId":"%s","kind":"interactive","status":"busy"}\n' + % (self.sleeper.pid, UUID[LIVE]), encoding="utf-8") + + def close(self) -> None: + self.sleeper.kill() + self.sleeper.wait() + + def everything(self) -> dict: + """Every entry under `-lanes/` and beside W, with its HEAD, its + status and its alternates: what a run that removes nothing leaves.""" + out = {} + for parent in (self.lanes_root, self.w.parent): + for entry in sorted(parent.iterdir()): + key = str(entry) + if entry.is_symlink() or not (entry / ".git").is_dir(): + out[key] = "present" + continue + c = ["-C", key] + alt = entry / ".git" / "objects" / "info" / "alternates" + out[key] = (self.git(*c, "rev-parse", "HEAD", check=False).strip(), + self.git(*c, "status", "--porcelain", "--untracked-files=all", + check=False), + alt.read_text(encoding="utf-8") if alt.exists() else "") + return out + + +def _rows(text: str) -> dict: + """{path: (tag, why)} for the report's entry lines.""" + rows = {} + for line in text.splitlines(): + if not line.startswith(" ") or line.startswith(" "): + continue + tag, _, rest = line.strip().partition(" ") + path, _, why = rest.strip().partition(" ") + rows[path] = (tag, why) + return rows + + +@pytest.fixture +def sweep(tmp_path: Path): + """`sweep(*args, **env)` runs `lane-end --sweep-checkouts ` and + returns the process and its report's rows; `sweep.box` is the sandbox.""" + root = Path(os.path.realpath(tmp_path)) / "s" + box = Sweep(root) + # EVERY PATH A RUN COULD REMOVE IS UNDER THIS TEST'S OWN BASETEMP: W, its + # parent, `-lanes/` and the `$HOME` every helper reads are the box's. + base = Path(os.path.realpath(tmp_path)) + for path in (box.w, box.w.parent, box.lanes_root, box.home): + assert Path(os.path.realpath(path)).is_relative_to(base), path + assert box.env["HOME"] == str(box.home) + + def run(*args: str, **env: str): + full = dict(box.env, **env) + result = subprocess.run(["bash", str(REPO / "lane-end"), "--sweep-checkouts", *args], + env=full, cwd=str(box.root), capture_output=True, + text=True, timeout=300, check=False) + return result, _rows(result.stdout) + + run.box = box + yield run + box.close() + + +def _tagged(rows: dict, tag: str) -> set: + return {path for path, (t, _) in rows.items() if t == tag} + + +# ------------------------------------------------------------- the dry run + +def test_dry_run_lists_and_removes_nothing(sweep): + """THE DEFAULT IS THE DRY RUN: it names the CLOSED lane's checkout and the + former name's beside its new one as candidates, and removes, fetches and + locks nothing (Amendment 19(c)'s shape, S6.2). 3 says something is there + to remove.""" + box = sweep.box + before, head = box.everything(), box.origin_head() + result, rows = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert "DRY RUN" in result.stdout, result.stdout + assert _tagged(rows, "candidate") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert rows[str(box.checkout(OLD))][1].startswith(f"a former name of lane {NEW}"), rows + assert box.everything() == before, "a dry run changes nothing under -lanes/ or beside W" + assert box.origin_head() == head, "nothing reached origin" + assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "a dry run takes no lock" + assert f"LANES_LANE= lane-end --sweep-checkouts --yes" in result.stdout + + +def test_dry_run_with_nothing_to_remove_exits_0(sweep): + """0 is nothing to remove: with the CLOSED lane's checkout and the former + name's gone, every entry is kept.""" + box = sweep.box + for lane in (CLOSED, OLD): + box.checkout(lane).rename(box.root / f"aside-{lane}") + result, rows = sweep() + assert result.returncode == 0, (result.stdout, result.stderr) + assert not _tagged(rows, "candidate"), rows + assert "there is nothing to remove" in result.stdout, result.stdout + + +def test_yes_without_LANES_LANE_is_refused(sweep): + """`--yes` writes a NOTED line into the log of the lane that runs it, so a + run that names no lane is refused before anything is read or removed.""" + box = sweep.box + before, head = box.everything(), box.origin_head() + result, _ = sweep("--yes") + assert result.returncode == 2, (result.stdout, result.stderr) + assert "LANES_LANE= lane-end --sweep-checkouts --yes" in result.stderr + assert "Nothing was removed" in result.stderr, result.stderr + assert box.everything() == before and box.origin_head() == head + + +def test_yes_under_LANES_NO_FETCH_is_refused(sweep): + """A writer whose fetch did not run is a writer S3 refuses (ratified choice + 2), and `--yes` fetches each checkout before it decides: refused before + the first removal, not met by the NOTED line after it (rv1b's MUST 3).""" + box = sweep.box + before, head = box.everything(), box.origin_head() + result, _ = sweep("--yes", LANES_LANE=RUNNER, LANES_NO_FETCH="1") + assert result.returncode == 2, (result.stdout, result.stderr) + assert "LANES_NO_FETCH=1 is set" in result.stderr, result.stderr + assert box.everything() == before and box.origin_head() == head + + +@pytest.mark.parametrize("extra", [ + ("--retire",), ("--force",), ("--inventory-only",), ("--state-was", "x"), + ("--home", "opensoft/repoS4"), ("--reason", "why"), ("--retire-dormant", "repoS4"), + (RUNNER,), +], ids=["retire", "force", "inventory-only", "state-was", "home", "reason", + "retire-dormant", "a lane"]) +def test_the_sweep_takes_no_lane_and_no_other_act(sweep, extra): + """The sweep is its own act and ends no lane: every flag of another act, + and a lane, is refused with nothing done.""" + box = sweep.box + before = box.everything() + result, _ = sweep(*extra) + assert result.returncode == 2, (extra, result.stdout, result.stderr) + assert box.everything() == before + + +# ------------------------------------------------------- never a candidate + +def test_W_and_live_lane_never_candidates(sweep): + """W, the readers' copy, is never a candidate, and neither is a LIVE lane's + checkout, a PAUSED one, the runner's own, a directory that is no clone, a + symbolic link (never followed), a hidden entry, or the checkout a caller's + own `LANES_REPO` names.""" + box = sweep.box + result, rows = sweep(LANES_LANE=RUNNER, LANES_REPO=str(box.checkout(PAUSED))) + assert result.returncode == 3, (result.stdout, result.stderr) + assert str(box.w) not in rows, "W is not even listed: it is not under -lanes/" + keep = {path: why for path, (tag, why) in rows.items() if tag == "keep"} + assert "is LIVE" in keep[str(box.checkout(LIVE))], keep + assert "runs this sweep" in keep[str(box.checkout(RUNNER))], keep + assert "LANES_REPO names" in keep[str(box.checkout(PAUSED))], keep + assert "no .git directory" in keep[str(box.lanes_root / "notes")], keep + assert "symbolic link" in keep[str(box.lanes_root / "a-link")], keep + assert f"lane {NEW} is IDLE" in keep[str(box.checkout(NEW))], keep + assert not any(".lock" in path for path in rows), "hidden entries are never looked at" + assert "1 hidden entr(ies)" in result.stdout, result.stdout + assert _tagged(rows, "candidate") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))} + + +def test_a_paused_lane_is_kept_without_LANES_REPO(sweep): + """PAUSED is a lane somebody may pick up, not a CLOSED one.""" + box = sweep.box + result, rows = sweep() + assert rows[str(box.checkout(PAUSED))] == ("keep", f"lane {PAUSED} is PAUSED, not CLOSED (Amendment 19)") + + +def test_a_W_that_is_a_lane_checkout_is_refused(sweep): + """FROM A LANE WINDOW, `LANES_WORKSPACE_ROOT` names the lane's own checkout, + and every OTHER lane's checkout would stand "beside W". The sweep refuses, + as `lanes-edit.sh`'s `lanes_home_check` does, naming the variable.""" + box = sweep.box + before = box.everything() + result, _ = sweep(LANES_WORKSPACE_ROOT=str(box.checkout(RUNNER))) + assert result.returncode == 2, (result.stdout, result.stderr) + assert "Unset LANES_WORKSPACE_ROOT" in result.stderr, result.stderr + assert box.everything() == before + + +def test_clones_beside_W_are_listed_and_never_candidates(sweep): + """S6.2's per-profile clones are listed, and nothing outside `-lanes/` + is a candidate. A clone of another repository is not listed at all.""" + box = sweep.box + result, rows = sweep() + assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones}, rows + assert str(box.w.parent / "otherrepo") not in rows, rows + assert all(Path(p).parent == box.lanes_root for p in _tagged(rows, "candidate")), rows + assert "listed and never removed by this build" in result.stdout From 8fa703bd28587c467b0363b28120a24b0d8530bc Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Wed, 21 Oct 2026 14:21:16 +0000 Subject: [PATCH 02/53] Amendment 23 S6.2: `lane-end --sweep-checkouts --yes` removes a CLOSED lane's register checkout only when nothing holds it, deciding each one again under its own writers' lock after a fetch (#198, slice 4 stage 2) Stage 2 of slice 4: the holds and the removal. - The holds (S6.2 (a) to (c)), each reported with what it holds: - (a) a `+` under `git cherry` on any local branch or a detached HEAD (refs spelled in full), a merge commit origin lacks (which `git cherry` skips), any other local ref reaching what origin lacks, a stash, staged or modified files, index entries marked assume-unchanged or skip-worktree, untracked files anywhere, an operation in progress, unmerged entries, an `index.lock`, the checkout's own writers' lock, and no `origin/` to compare with. Ignored files hold nothing and are named. - (b) another repository (W, one beside it, or one under `-lanes/`) whose alternates file names its objects, and linked worktrees. - (c) a live process with its working directory inside (`/proc/*/cwd`, or `lsof`; with neither, "cannot tell" holds it). - `--yes`, one candidate at a time (rv1b's MUSTs 1-2): take its `lanes/.lanes-edit.lock` with `lanes-edit.sh`'s pid line, fetch it (`LANES_GIT`, `LANES_GIT_TIMEOUT`, an explicit refspec), ask the lane's state again (`lanes --fetch --closed --lane`), read every hold, what leans on it and the working directories again, and only then move it aside to `-lanes/..removing--` and remove that. A removal that does not finish says so by path and exits 1. - The dry run tags each candidate `remove` or `hold`, and lists what holds each clone beside W. Nothing outside `-lanes/` is removed. The NOTED line and the leaves-first order are stage 3. Tests: tests/test_a23_sweep_checkouts.py, 37 cases (16 of stage 1, now reading `remove`, and 21 new: the act, 14 holds, a dry-run-only hold, a live cwd, the re-check under the lock, a lane resumed after the listing, a failed fetch, and nothing beside W removed). Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 991d8c6c97ee443897b4e0fc4787f4275440ae4c) --- lane-end | 391 +++++++++++++++++++++++++++++- tests/test_a23_sweep_checkouts.py | 288 +++++++++++++++++++++- 2 files changed, 668 insertions(+), 11 deletions(-) diff --git a/lane-end b/lane-end index a2dac9a..0a759e5 100755 --- a/lane-end +++ b/lane-end @@ -592,7 +592,14 @@ LANE_WORKTREES="${LANE_WORKTREES:-$(lane_worktrees_default)}" LE_TMP="" -cleanup() { [ -n "${LE_TMP:-}" ] && rm -f -- "$LE_TMP"; return 0; } +# `SC_LOCK` is a register checkout's `lanes/.lanes-edit.lock` that +# `--sweep-checkouts --yes` took and holds, released on every way out. +SC_LOCK="" +cleanup() { + [ -n "${LE_TMP:-}" ] && rm -f -- "$LE_TMP" + if [ -n "${SC_LOCK:-}" ]; then rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || :; fi + return 0 +} # A SIGNAL ENDS THE RUN, AND `cleanup` RUNS ON THE WAY OUT. `cleanup` returns # rather than exits, so the one trap `cleanup EXIT INT TERM` ran it and then # RESUMED the script at the next command — harmless while nothing was left to @@ -1233,34 +1240,399 @@ SC_IDX note "$sc_root, the directory W stands in, cannot be listed, so no clone beside W is listed. Nothing outside $sc_home/ is ever removed." fi + # ------------------------------------------------------------ the holds + # + # S6.2: *"It removes one only where (a) nothing is unpublished — no `+` line + # under `git cherry origin/main` on any local branch, no stash, nothing + # staged or modified, no untracked file under `handoffs/` or `lanes/`; (b) no + # other clone's alternates file names it, so leaves go first; and (c) no + # live process has its working directory there. Any other is reported with + # what it holds, and left."* Each is read, and so is whatever git cannot + # prove of that list, because a removal is the one act nothing undoes: + # (a) every local branch AND a detached HEAD by content (`git cherry`, + # every ref spelled in full, slice 1b's B3), and a MERGE commit origin + # lacks, which `git cherry` skips and so cannot compare; any other + # local ref (a tag, a note) reaching what origin's branch lacks; a + # stash; staged or modified files, and index entries marked + # assume-unchanged or skip-worktree, which `git status` does not look + # at; untracked files ANYWHERE, not only under `handoffs/` and + # `lanes/`, because no tool can tell a person's work from scratch; an + # operation in progress, unmerged entries, a held `index.lock`, and + # the checkout's own writers' lock. No `origin/` to compare + # with is itself a hold: nothing can be proved published. IGNORED + # files do not hold — the register's own rules keep them out of every + # commit — and they are named, so a person sees them first; + # (b) another repository whose alternates file names its objects, and its + # own linked worktrees; + # (c) a process this user can see whose working directory is at or under + # it: `/proc/*/cwd` on Linux, `lsof` on macOS, and with neither it is + # held as "cannot tell". A process whose directory cannot be read + # (another user's, or one that ended meanwhile) is counted and said, + # and does not hold — `lsof` has the same reach, and (a) is what keeps + # the work. + SC_HOLD=""; SC_IGN="" + sc_hold() { SC_HOLD="${SC_HOLD:+$SC_HOLD; }$1"; } + sc_first() { # — the count and the first three, as `N (a b c)` + scf_n="$(printf '%s\n' "$1" | awk 'NF { n++ } END { print n + 0 }')" || scf_n="?" + scf_h="$(printf '%s\n' "$1" | awk 'NF && k < 3 { print; k++ }' | tr '\n' ' ')" || scf_h="" + printf '%s (%s)' "$scf_n" "${scf_h% }" + } + sc_holds() { # <1 when this run holds its writers' lock> + sh_p="$1"; sh_mine="$2"; SC_HOLD=""; SC_IGN="" + sh_gd="$sh_p/.git" + sh_o="refs/remotes/origin/$sc_branch" + if [ "$sh_mine" != 1 ] && { [ -e "$sh_p/lanes/.lanes-edit.lock" ] || [ -L "$sh_p/lanes/.lanes-edit.lock" ]; }; then + sc_hold "its writers' lock lanes/.lanes-edit.lock is there: a writer is in it, or one left it behind" + fi + for sh_op in rebase-merge rebase-apply MERGE_HEAD CHERRY_PICK_HEAD REVERT_HEAD BISECT_LOG; do + if [ -e "$sh_gd/$sh_op" ]; then sc_hold "an operation in progress ($sh_op)"; fi + done + if [ -e "$sh_gd/index.lock" ]; then sc_hold "a git process holds its index.lock"; fi + sh_x="$(git -C "$sh_p" ls-files -u 2>/dev/null)" || sh_x="unreadable" + if [ -n "$sh_x" ]; then sc_hold "unmerged entries"; fi + if ! git -C "$sh_p" rev-parse -q --verify "$sh_o^{commit}" >/dev/null 2>&1; then + sc_hold "it has no $sh_o, so nothing in it can be proved published" + else + sh_refs="$(git -C "$sh_p" for-each-ref --format='%(refname)' 2>/dev/null)" || { + sc_hold "its refs could not be listed"; sh_refs=""; } + if ! git -C "$sh_p" symbolic-ref -q HEAD >/dev/null 2>&1; then + sh_refs="HEAD +$sh_refs" + fi + while IFS= read -r sh_r; do + [ -n "$sh_r" ] || continue + case "$sh_r" in + refs/remotes/* | refs/stash) continue ;; + HEAD | refs/heads/*) + sh_name="${sh_r#refs/heads/}" + if [ "$sh_r" = HEAD ]; then sh_name="its detached HEAD"; else sh_name="branch $sh_name"; fi + sh_plus="$(git -C "$sh_p" cherry "$sh_o" "$sh_r" 2>/dev/null | awk '/^\+/ { n++ } END { print n + 0 }')" || sh_plus=x + case "$sh_plus" in + 0) : ;; + x) sc_hold "$sh_name could not be compared with origin by content" ;; + *) sc_hold "$sh_name has $sh_plus commit(s) origin lacks by content (git -C $sh_p cherry -v $sh_o $sh_r)" ;; + esac + sh_m="$(git -C "$sh_p" rev-list --merges -n 1 "$sh_r" --not "$sh_o" 2>/dev/null)" || sh_m=x + if [ -n "$sh_m" ]; then + sc_hold "$sh_name has a merge commit origin lacks, which git cherry cannot compare by content" + fi ;; + *) + sh_m="$(git -C "$sh_p" rev-list -n 1 "$sh_r" --not "$sh_o" 2>/dev/null)" || sh_m=x + if [ -n "$sh_m" ]; then sc_hold "$sh_r reaches commits origin's $sc_branch lacks"; fi ;; + esac + done </dev/null 2>&1; then + sh_n="$(git -C "$sh_p" stash list 2>/dev/null | awk 'NF { n++ } END { print n + 0 }')" || sh_n="?" + sc_hold "a stash of $sh_n entr(ies)" + fi + sh_x="$(GIT_OPTIONAL_LOCKS=0 git -C "$sh_p" status --porcelain --untracked-files=no 2>/dev/null)" || sh_x="?? unreadable" + case "$sh_x" in + '') : ;; + '?? unreadable') sc_hold "git status could not be read in it" ;; + *) sc_hold "staged or modified: $(sc_first "$(printf '%s\n' "$sh_x" | cut -c4-)")" ;; + esac + sh_x="$(git -C "$sh_p" ls-files -v 2>/dev/null | awk 'substr($0, 1, 1) ~ /[a-zS]/ { n++ } END { print n + 0 }')" || sh_x=x + case "$sh_x" in + 0) : ;; + x) sc_hold "its index could not be read" ;; + *) sc_hold "$sh_x index entr(ies) marked assume-unchanged or skip-worktree, which git status does not look at" ;; + esac + sh_x="$(git -C "$sh_p" ls-files --others --exclude-standard 2>/dev/null)" || sh_x="?unreadable" + case "$sh_x" in + '?unreadable') sc_hold "its untracked files could not be listed" ;; + *) + sh_x="$(printf '%s\n' "$sh_x" | awk 'NF && index($0, "lanes/.lanes-edit.lock/") != 1')" || + sh_x="(the list could not be read)" + if [ -n "$sh_x" ]; then sc_hold "untracked: $(sc_first "$sh_x")"; fi ;; + esac + if [ -d "$sh_gd/worktrees" ]; then + sh_x="$(ls -A -- "$sh_gd/worktrees" 2>/dev/null)" || sh_x="?" + if [ -n "$sh_x" ]; then sc_hold "linked worktree(s) of it: $(sc_first "$sh_x") (git -C $sh_p worktree list)"; fi + fi + sh_x="$(git -C "$sh_p" ls-files --others --ignored --exclude-standard --directory 2>/dev/null)" || sh_x="" + if [ -n "$sh_x" ]; then SC_IGN="ignored, which holds nothing: $(sc_first "$sh_x")"; fi + return 0 + } + + # (b) WHAT LEANS ON WHAT: ` TAB `, for W, every repository in W's own directory and + # every entry of `-lanes/`, hidden ones included. Read afresh before each + # removal, so a dependent kept earlier in the run still holds what it + # borrows from. + sc_dep_lines="" + sc_dep_scan() { + sc_dep_lines="" + for sds_e in "$sc_w" "$sc_root"/* "$sc_home"/* "$sc_home"/.[!.]* "$sc_home"/..?*; do + [ -d "$sds_e" ] || continue + if [ -f "$sds_e/.git/objects/info/alternates" ]; then + sds_a="$sds_e/.git/objects/info/alternates"; sds_o="$sds_e/.git/objects" + elif [ -f "$sds_e/objects/info/alternates" ]; then + sds_a="$sds_e/objects/info/alternates"; sds_o="$sds_e/objects" + else + continue + fi + while IFS= read -r sds_l || [ -n "$sds_l" ]; do + case "$sds_l" in '' | '#'*) continue ;; esac + case "$sds_l" in /*) sds_t="$sds_l" ;; *) sds_t="$sds_o/$sds_l" ;; esac + sds_r="$(sc_real "$sds_t")" + if [ -n "$sds_r" ]; then + sc_dep_lines="${sc_dep_lines}${sds_e}${sc_tab}${sds_r} +" + fi + done < "$sds_a" + done + } + sc_dependents() { # [] — what leans on it + scd_obj="$1/.git/objects"; scd_out="" + while IFS="$sc_tab" read -r scd_d scd_t; do + [ -n "$scd_d" ] && [ "$scd_t" = "$scd_obj" ] && [ "$scd_d" != "$1" ] || continue + case " ${2-} " in *" $scd_d "*) continue ;; esac + scd_out="${scd_out:+$scd_out }$scd_d" + done < TAB `. + sc_cwds=""; sc_cwd_mode=""; sc_cwd_unread=0 + sc_cwd_scan() { + sc_cwds=""; sc_cwd_unread=0 + if [ -d /proc/self ] && readlink /proc/self/cwd >/dev/null 2>&1; then + sc_cwd_mode=proc + for scw_d in /proc/[0-9]*; do + scw_l="$(readlink "$scw_d/cwd" 2>/dev/null)" || { + if [ -d "$scw_d" ]; then sc_cwd_unread=$((sc_cwd_unread + 1)); fi + continue; } + sc_cwds="${sc_cwds}${scw_d#/proc/}${sc_tab}${scw_l% (deleted)} +" + done + elif command -v lsof >/dev/null 2>&1; then + sc_cwd_mode=lsof + scw_pid="" + scw_out="$(lsof -a -d cwd -Fpn 2>/dev/null)" || : + while IFS= read -r scw_l; do + case "$scw_l" in + p*) scw_pid="${scw_l#p}" ;; + n*) sc_cwds="${sc_cwds}${scw_pid}${sc_tab}${scw_l#n} +" ;; + esac + done < — the pids whose working directory is at or under it + scc_out="" + while IFS="$sc_tab" read -r scc_pid scc_cwd; do + [ -n "$scc_pid" ] || continue + case "$scc_cwd" in "$1" | "$1"/*) scc_out="${scc_out:+$scc_out }$scc_pid" ;; esac + done < <1 when this run holds its lock> [] + sc_holds "$1" "$2" + sjh_deps="$(sc_dependents "$1" "${3-}")" + if [ -n "$sjh_deps" ]; then sc_hold "leaned on by $sjh_deps (its alternates file names this clone's objects)"; fi + case "$sc_cwd_mode" in + none) sc_hold "neither /proc nor lsof is here, so whether a process works in it cannot be told" ;; + *) + sjh_pids="$(sc_cwd_inside "$1")" + if [ -n "$sjh_pids" ]; then sc_hold "a live process works in it (pid $sjh_pids)"; fi ;; + esac + return 0 + } + + # NEVER A PATH THAT IS NOT DIRECTLY UNDER `-lanes/`, by its real parent, + # asked again at the moment of each act on it. + sc_under_home() { # + suh_n="${1##*/}" + case "$suh_n" in '' | . | ..) return 1 ;; esac + [ "${1%/*}" = "$sc_home" ] && [ "$(sc_real "${1%/*}")" = "$sc_home" ] + } + + # THE LOCK'S PID LINE, in `lanes-edit.sh`'s own form, ` + # `, so a writer of that checkout in this container waits on a live + # holder and one in another container ages it by the directory's mtime. + sc_ident="none unknown" + if [ "$sweep_yes" = 1 ]; then + sc_wsl="$("$LANES_EDIT" workstation 2>/dev/null || :)" + sc_host="$(printf '%s\n' "$sc_wsl" | cut -f3)" || sc_host="" + sc_cont="$(printf '%s\n' "$sc_wsl" | cut -f7)" || sc_cont="" + case "$sc_host" in '' | *[!A-Za-z0-9._-]*) sc_host=unknown ;; esac + case "$sc_cont" in '' | *[!A-Za-z0-9._-]*) sc_cont=none ;; esac + sc_ident="$sc_cont $sc_host" + fi + sc_lock_take() { # — 0 holding it (SC_LOCK), 1 with SC_WHY + slt_d="$1/lanes/.lanes-edit.lock" + if [ -L "$1/lanes" ] || [ ! -d "$1/lanes" ]; then + SC_WHY="it has no lanes/ directory of its own, so its writers' lock cannot be taken"; return 1 + fi + if ! mkdir -- "$slt_d" 2>/dev/null; then + SC_WHY="its writers' lock lanes/.lanes-edit.lock is held: a writer is in it, or one left it behind"; return 1 + fi + SC_LOCK="$slt_d" + printf '%s %s\n' "$$" "$sc_ident" > "$slt_d/pid" 2>/dev/null || : + return 0 + } + sc_lock_drop() { + if [ -n "$SC_LOCK" ]; then rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || :; fi + SC_LOCK="" + } + # `--yes` FETCHES EACH CHECKOUT FIRST, bounded as `lanes-edit.sh`'s network + # calls are (`LANES_GIT`, `LANES_GIT_TIMEOUT`), with an explicit refspec so + # the tracking ref moves whatever the clone's `remote.origin.fetch` says. + sc_fetch() { # + sf_git="${LANES_GIT:-git}"; sf_to="${LANES_GIT_TIMEOUT:-60}" + sf_args=(-c gc.auto=0 -c maintenance.auto=false -C "$1" fetch -q origin + "+refs/heads/$sc_branch:refs/remotes/origin/$sc_branch") + case "$sf_to" in + '' | 0 | *[!0-9]*) "$sf_git" "${sf_args[@]}" >/dev/null 2>&1; return $? ;; + esac + if command -v timeout >/dev/null 2>&1; then + timeout -k 5 "$sf_to" "$sf_git" "${sf_args[@]}" >/dev/null 2>&1; return $? + fi + "$sf_git" "${sf_args[@]}" >/dev/null 2>&1 + } + # THE LANE'S STATE, ASKED AGAIN UNDER THE CHECKOUT'S LOCK after a fetch of + # W (rv1b's MUSTs 1-2: a decision made from register state is made after + # the fetch, under the lock). 0 when it is still what made it a candidate. + sc_still() { # + ss_rc=0 + ss_row="$("$LANES_EDIT" lanes --fetch --closed --lane "$2" 2>/dev/null)" || ss_rc=$? + if [ "$ss_rc" != 0 ] || [ -z "$ss_row" ]; then + SC_WHY="lane $2's state could not be read again under its checkout's lock (lanes exited $ss_rc)"; return 1 + fi + IFS="$sc_tab" read -r ss_lane ss_state ss_ws ss_pf ss_win ss_sid ss_dir ss_obj ss_age ss_cmd ss_home ss_fk ss_loc ss_class ss_rest </dev/null)" || ss_c="" + if [ "$(sc_lc "$ss_c")" != "$(sc_lc "$2")" ] || [ ! -d "$3" ] || [ -L "$3" ] || [ "$3" -ef "$1" ]; then + SC_WHY="it is no longer a former name of lane $2 beside that lane's own checkout"; return 1 + fi + return 0 + fi + if [ "${ss_state:-}" = LIVE ]; then SC_WHY="lane $2 is LIVE now"; return 1; fi + if [ "${ss_class:-}" != closed ]; then SC_WHY="lane $2 is ${ss_state:-in no state} now, not CLOSED"; return 1; fi + return 0 + } + # ------------------------------------------------------------ the report printf '%s --sweep-checkouts — %s (lane-collision-protocol Amendment 23 S6.2)\n' \ "$prog" "$( [ "$sweep_yes" = 1 ] && printf 'THE ACT' || printf 'DRY RUN, nothing is removed' )" printf ' W : %s\n' "$sc_w" printf ' lane checkouts : %s/\n' "$sc_home" printf ' runner : %s\n' "${sc_runner:-none named (LANES_LANE is unset)}" + if [ "$sweep_yes" = 1 ]; then + printf ' origin : each checkout is fetched under its own lock before it is decided\n' + else + printf ' origin : each checkout'"'"'s origin/%s as it last fetched it: a dry run fetches nothing\n' "$sc_branch" + fi if [ -n "$sc_home_why" ]; then printf ' (%s)\n' "$sc_home_why"; fi if [ "$sc_hidden" -gt 0 ]; then printf ' (%s hidden entr(ies) under %s/ - creation locks, partial clones, the sidecar lock - are never looked at)\n' "$sc_hidden" "$sc_home" fi + sc_cwd_scan + if [ "$sc_cwd_unread" -gt 0 ]; then + printf ' (%s process(es) whose working directory could not be read - another user'"'"'s, or ended while read - do not hold anything)\n' "$sc_cwd_unread" + fi + sc_dep_scan + printf '\n under %s/:\n' "$sc_home" + sc_gone=""; sc_n_removed=0; sc_n_held=0; sc_n_removable=0; sc_partial="" sc_i=0 while [ "$sc_i" -lt "${#sc_cands[@]}" ]; do - printf ' candidate %s %s\n' "${sc_cands[$sc_i]}" "${sc_cand_why[$sc_i]}" + sc_c="${sc_cands[$sc_i]}"; sc_why="${sc_cand_why[$sc_i]}" + sc_l="${sc_cand_lane[$sc_i]}"; sc_new="${sc_cand_new[$sc_i]}" sc_i=$((sc_i + 1)) + if ! sc_under_home "$sc_c"; then + printf ' hold %s %s; it is not directly under %s/, and nothing else is ever removed\n' "$sc_c" "$sc_why" "$sc_home" + sc_n_held=$((sc_n_held + 1)); continue + fi + if [ "$sweep_yes" != 1 ]; then + sc_judge_holds "$sc_c" 0 "$sc_gone" + if [ -n "$SC_HOLD" ]; then + printf ' hold %s %s; holds: %s\n' "$sc_c" "$sc_why" "$SC_HOLD" + sc_n_held=$((sc_n_held + 1)) + else + printf ' remove %s %s%s\n' "$sc_c" "$sc_why" "${SC_IGN:+; $SC_IGN}" + sc_gone="${sc_gone:+$sc_gone }$sc_c"; sc_n_removable=$((sc_n_removable + 1)) + fi + continue + fi + # THE ACT, ONE CHECKOUT AT A TIME, every decision made again under its + # own writers' lock after a fetch: the lock first, so no writer of it is + # in it; then its fetch, the lane's state asked again, every hold, what + # leans on it and the working directories read again; then, and only + # then, it is moved aside under `-lanes/` — so its S1 path is absent at + # once and the next write there clones afresh — and the aside removed. + if ! sc_lock_take "$sc_c"; then + printf ' hold %s %s; holds: %s\n' "$sc_c" "$sc_why" "$SC_WHY" + sc_n_held=$((sc_n_held + 1)); continue + fi + sc_act_hold="" + if ! sc_fetch "$sc_c"; then + sc_act_hold="its fetch of origin $sc_branch failed or timed out, and nothing is decided on an old origin/$sc_branch" + elif ! sc_still "$sc_c" "$sc_l" "$sc_new"; then + sc_act_hold="$SC_WHY" + else + sc_dep_scan; sc_cwd_scan + sc_judge_holds "$sc_c" 1 + sc_act_hold="$SC_HOLD" + fi + if [ -n "$sc_act_hold" ]; then + sc_lock_drop + printf ' hold %s %s; holds: %s\n' "$sc_c" "$sc_why" "$sc_act_hold" + sc_n_held=$((sc_n_held + 1)); continue + fi + sc_aside="$sc_home/.${sc_c##*/}.removing-$sc_utc-$$" + if ! sc_under_home "$sc_c" || [ -e "$sc_aside" ] || [ -L "$sc_aside" ] || ! mv -- "$sc_c" "$sc_aside" 2>/dev/null; then + sc_lock_drop + printf ' hold %s %s; holds: it could not be moved aside to %s\n' "$sc_c" "$sc_why" "$sc_aside" + sc_n_held=$((sc_n_held + 1)); continue + fi + SC_LOCK="" # it went with the checkout + if sc_under_home "$sc_aside"; then rm -rf -- "$sc_aside" 2>/dev/null || :; fi + if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then + printf ' PARTLY %s %s; moved aside to %s, which could not all be removed\n' "$sc_c" "$sc_why" "$sc_aside" + sc_partial="${sc_partial:+$sc_partial }$sc_aside" + else + printf ' removed %s %s\n' "$sc_c" "$sc_why" + fi + sc_gone="${sc_gone:+$sc_gone }$sc_c"; sc_n_removed=$((sc_n_removed + 1)) done printf '%s' "$sc_keep_lines" if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ]; then printf ' (nothing)\n'; fi + printf '\n beside W (Amendment 23 S6.2), listed and never removed by this build:\n' for sc_e in ${sc_beside[@]+"${sc_beside[@]}"}; do - printf ' beside %s\n' "$sc_e" + if [ "$sweep_yes" = 1 ]; then + printf ' beside %s left\n' "$sc_e" + continue + fi + sc_judge_holds "$sc_e" 0 "$sc_gone" + if [ -n "$SC_HOLD" ]; then + printf ' beside %s holds: %s\n' "$sc_e" "$SC_HOLD" + else + printf ' beside %s nothing holds it%s\n' "$sc_e" "${SC_IGN:+; $SC_IGN}" + fi done if [ "${#sc_beside[@]}" = 0 ]; then printf ' (none)\n'; fi - printf '\n %s candidate(s) under %s/; %s clone(s) beside W listed.\n' \ - "${#sc_cands[@]}" "$sc_home" "${#sc_beside[@]}" if [ "$sweep_yes" != 1 ]; then - if [ "${#sc_cands[@]}" -gt 0 ]; then + printf '\n %s removable, %s held under %s/; %s clone(s) beside W listed.\n' \ + "$sc_n_removable" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" + if [ "$sc_n_removable" -gt 0 ]; then printf '\n DRY RUN — nothing was removed. Make it the act with:\n' printf ' LANES_LANE= %s --sweep-checkouts --yes\n' "$prog" exit 3 @@ -1268,7 +1640,12 @@ SC_IDX printf '\n DRY RUN — nothing was removed, and there is nothing to remove.\n' exit 0 fi - die "--sweep-checkouts --yes is not built in this commit (Amendment 23 act 2, slice 4, stage 1 of 3: the holds and the removal are stage 2). Nothing was removed." 2 + printf '\n %s removed, %s held under %s/; %s clone(s) beside W left.\n' \ + "$sc_n_removed" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" + if [ -n "$sc_partial" ]; then + die "the removal of $sc_partial did not finish (above): what is left is no checkout, under a hidden name, so the next write of that lane clones afresh. Remove it by hand once you have looked at why." 1 + fi + exit 0 fi # THE SWEEP'S OWN FLAGS BELONG TO THE SWEEP (Copilot round 3 on #93). They are diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 71a4837..5a23535 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -188,7 +188,7 @@ def test_dry_run_lists_and_removes_nothing(sweep): result, rows = sweep() assert result.returncode == 3, (result.stdout, result.stderr) assert "DRY RUN" in result.stdout, result.stdout - assert _tagged(rows, "candidate") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert _tagged(rows, "remove") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows assert rows[str(box.checkout(OLD))][1].startswith(f"a former name of lane {NEW}"), rows assert box.everything() == before, "a dry run changes nothing under -lanes/ or beside W" assert box.origin_head() == head, "nothing reached origin" @@ -204,7 +204,7 @@ def test_dry_run_with_nothing_to_remove_exits_0(sweep): box.checkout(lane).rename(box.root / f"aside-{lane}") result, rows = sweep() assert result.returncode == 0, (result.stdout, result.stderr) - assert not _tagged(rows, "candidate"), rows + assert not _tagged(rows, "remove"), rows assert "there is nothing to remove" in result.stdout, result.stdout @@ -268,7 +268,7 @@ def test_W_and_live_lane_never_candidates(sweep): assert f"lane {NEW} is IDLE" in keep[str(box.checkout(NEW))], keep assert not any(".lock" in path for path in rows), "hidden entries are never looked at" assert "1 hidden entr(ies)" in result.stdout, result.stdout - assert _tagged(rows, "candidate") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))} + assert _tagged(rows, "remove") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))} def test_a_paused_lane_is_kept_without_LANES_REPO(sweep): @@ -297,5 +297,285 @@ def test_clones_beside_W_are_listed_and_never_candidates(sweep): result, rows = sweep() assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones}, rows assert str(box.w.parent / "otherrepo") not in rows, rows - assert all(Path(p).parent == box.lanes_root for p in _tagged(rows, "candidate")), rows + judged = _tagged(rows, "remove") | _tagged(rows, "hold") + assert judged and all(Path(p).parent == box.lanes_root for p in judged), rows assert "listed and never removed by this build" in result.stdout + + +# ------------------------------------------------------- the act, and holds + +def _assert_under(base: Path, paths) -> None: + """Every path the run removed is under the test's own basetemp.""" + for path in paths: + assert Path(os.path.realpath(path)).is_relative_to(base), path + + +def _no_aside(box: Sweep) -> None: + assert not list(box.lanes_root.glob(".*.removing-*")), "no half-removed aside is left" + + +def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): + """`--yes`, run by a lane, removes the CLOSED lane's checkout and the former + name's beside its new one, and leaves every other entry — the LIVE lane's, + the PAUSED one's, its own, a non-clone, a link — and every clone beside W + exactly as they were.""" + box = sweep.box + before = box.everything() + gone = {str(box.checkout(CLOSED)), str(box.checkout(OLD))} + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == gone, rows + _assert_under(Path(os.path.realpath(tmp_path)), gone) + for path in gone: + assert not os.path.lexists(path), path + after = box.everything() + assert {k: v for k, v in before.items() if k not in gone} == after, \ + "nothing but the two removed entries changed, under -lanes/ or beside W" + assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones} + _no_aside(box) + assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "every lock it took is released" + + +def _commit_unpushed(box: Sweep, c: Path) -> None: + box.write(c, "lanes/extra.md", "a record never pushed\n") + box.commit(c, "an unpushed record", "lanes/extra.md") + + +def _stash(box: Sweep, c: Path) -> None: + (c / "handoffs" / "README.md").write_text("# handoffs\nstashed\n", encoding="utf-8") + box.git("-C", str(c), "stash", "-q") + + +def _staged(box: Sweep, c: Path) -> None: + box.write(c, "handoffs/staged.md", "staged, never committed\n") + box.git("-C", str(c), "add", "--", "handoffs/staged.md") + + +def _modified(box: Sweep, c: Path) -> None: + (c / "handoffs" / "README.md").write_text("# handoffs\nedited by hand\n", encoding="utf-8") + + +def _untracked_handoff(box: Sweep, c: Path) -> None: + box.write(c, "handoffs/repoS4/draft.md", "a handoff written, never committed\n") + + +def _untracked_elsewhere(box: Sweep, c: Path) -> None: + box.write(c, "notes.txt", "a person's note in the root\n") + + +def _side_branch(box: Sweep, c: Path) -> None: + box.git("-C", str(c), "switch", "-q", "-c", "side") + _commit_unpushed(box, c) + box.git("-C", str(c), "switch", "-q", "main") + + +def _detached(box: Sweep, c: Path) -> None: + box.git("-C", str(c), "switch", "-q", "--detach") + _commit_unpushed(box, c) + + +def _merge_only(box: Sweep, c: Path) -> None: + """A merge commit origin lacks, whose parents origin has both, carrying + content of its own. `git cherry` skips merges, so it lists no `+` line at + all here, and only the merge check can hold it: B, an empty commit, is + published to origin's main; main goes back to B's parent; and B is merged + in with `--no-ff` and a file only the merge carries.""" + c_ = ["-C", str(c)] + box.git(*c_, "commit", "-q", "--allow-empty", "-m", "B, published") + box.git(*c_, "push", "-q", "origin", "main") + box.git(*c_, "reset", "-q", "--keep", "HEAD~1") + box.git(*c_, "merge", "-q", "--no-ff", "--no-commit", "refs/remotes/origin/main") + box.write(c, "lanes/evil.md", "content only the merge carries\n") + box.git(*c_, "add", "--", "lanes/evil.md") + box.git(*c_, "commit", "-q", "-m", "merge B, with content") + cherry = box.git(*c_, "cherry", "refs/remotes/origin/main", "refs/heads/main") + assert not [line for line in cherry.splitlines() if line.startswith("+")], cherry + assert box.git(*c_, "rev-list", "--merges", "refs/heads/main", "--not", + "refs/remotes/origin/main").strip(), "the merge is the one commit origin lacks" + + +def _tag(box: Sweep, c: Path) -> None: + box.git("-C", str(c), "commit", "-q", "--allow-empty", "-m", "tagged, never pushed") + box.git("-C", str(c), "tag", "kept") + box.git("-C", str(c), "reset", "-q", "--keep", "HEAD~1") + + +def _lock(box: Sweep, c: Path) -> None: + (c / "lanes" / ".lanes-edit.lock").mkdir() + (c / "lanes" / ".lanes-edit.lock" / "pid").write_text("1 none unknown\n", encoding="utf-8") + + +def _worktree(box: Sweep, c: Path) -> None: + """A linked worktree of the checkout, made by hand in the shape `git + worktree add` leaves (this suite runs no `git worktree add`): its admin + directory under `.git/worktrees/`, and the worktree's own `.git` file.""" + linked = box.root / "wt-of-closed" + linked.mkdir() + admin = c / ".git" / "worktrees" / "wt-of-closed" + admin.mkdir(parents=True) + head = box.git("-C", str(c), "rev-parse", "HEAD").strip() + (admin / "HEAD").write_text(head + "\n", encoding="utf-8") + (admin / "commondir").write_text("../..\n", encoding="utf-8") + (admin / "gitdir").write_text(f"{linked / '.git'}\n", encoding="utf-8") + (linked / ".git").write_text(f"gitdir: {admin}\n", encoding="utf-8") + assert str(linked) in box.git("-C", str(c), "worktree", "list"), "git sees the linked worktree" + + +def _assume_unchanged(box: Sweep, c: Path) -> None: + box.git("-C", str(c), "update-index", "--assume-unchanged", "handoffs/README.md") + (c / "handoffs" / "README.md").write_text("# handoffs\nhidden from status\n", encoding="utf-8") + + +def _merging(box: Sweep, c: Path) -> None: + head = box.git("-C", str(c), "rev-parse", "HEAD").strip() + (c / ".git" / "MERGE_HEAD").write_text(head + "\n", encoding="utf-8") + + +def _no_origin_main(box: Sweep, c: Path) -> None: + box.git("-C", str(c), "update-ref", "-d", "refs/remotes/origin/main") + + +HOLDS = [ + pytest.param(_commit_unpushed, "commit(s) origin lacks by content", id="a + on main"), + pytest.param(_side_branch, "branch side has 1 commit(s) origin lacks", id="a + on a branch"), + pytest.param(_detached, "its detached HEAD has 1 commit(s)", id="a + on a detached HEAD"), + pytest.param(_merge_only, "a merge commit origin lacks", id="a merge only"), + pytest.param(_tag, "refs/tags/kept reaches commits", id="a tag"), + pytest.param(_stash, "a stash of 1 entr(ies)", id="a stash"), + pytest.param(_staged, "staged or modified: 1 (handoffs/staged.md)", id="staged"), + pytest.param(_modified, "staged or modified: 1 (handoffs/README.md)", id="modified"), + pytest.param(_untracked_handoff, "untracked: 1 (handoffs/repoS4/draft.md)", id="untracked under handoffs"), + pytest.param(_untracked_elsewhere, "untracked: 1 (notes.txt)", id="untracked elsewhere"), + pytest.param(_assume_unchanged, "marked assume-unchanged or skip-worktree", id="assume-unchanged"), + pytest.param(_merging, "an operation in progress (MERGE_HEAD)", id="a merge in progress"), + pytest.param(_lock, "lanes/.lanes-edit.lock", id="its writers' lock"), + pytest.param(_worktree, "linked worktree(s) of it", id="a linked worktree"), +] + + +@pytest.mark.parametrize("make,said", HOLDS) +def test_holds(sweep, make, said): + """S6.2 (a): each hold is reported with what it holds, and the checkout is + left — by the dry run, and by `--yes`, which decides under its lock after + a fetch and releases that lock as it leaves.""" + box = sweep.box + c = box.checkout(CLOSED) + make(box, c) + before = box.everything()[str(c)] + result, rows = sweep() + assert rows[str(c)][0] == "hold", rows[str(c)] + assert said in rows[str(c)][1], rows[str(c)] + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert said in rows[str(c)][1], rows[str(c)] + assert c.is_dir() and box.everything()[str(c)] == before, "the held checkout is as it was" + if make is not _lock: + assert not (c / "lanes" / ".lanes-edit.lock").exists(), "the sweep's own lock is released" + _no_aside(box) + + +def test_no_origin_main_holds_the_dry_run(sweep): + """A checkout with no `refs/remotes/origin/main` has nothing its branches + can be compared with, so the dry run, which fetches nothing, holds it. + `--yes` fetches it first with an explicit refspec, which makes that ref + again, and decides on what the fetch brought: here nothing holds it.""" + box = sweep.box + c = box.checkout(CLOSED) + _no_origin_main(box, c) + _, rows = sweep() + assert rows[str(c)][0] == "hold", rows[str(c)] + assert "it has no refs/remotes/origin/main" in rows[str(c)][1], rows[str(c)] + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows[str(c)] + assert not os.path.lexists(c) + _no_aside(box) + + +def test_a_live_cwd_holds(sweep): + """S6.2 (c): a live process working inside the checkout holds it. The + process is a `sleep` started there, killed by its PID afterwards.""" + box = sweep.box + c = box.checkout(CLOSED) + sleeper = subprocess.Popen(["sleep", "600"], cwd=str(c / "lanes")) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"a live process works in it (pid {sleeper.pid})" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + finally: + sleeper.kill() + sleeper.wait() + + +def _hook_on_fetch_of(box: Sweep, target: Path, body: str) -> None: + """A `LANES_GIT` hook that runs `body` when the sweep fetches `target`.""" + box.use_net(hook=( + f'case " $* " in *" -C {target} fetch "*) {body} ;; esac\n' + "exit 0\n")) + + +def test_recheck_before_removal(sweep): + """THE HOLDS ARE READ AGAIN UNDER THE LOCK, AFTER THE FETCH: a handoff + written into the checkout after the listing and before its removal holds + it, whatever the listing said.""" + box = sweep.box + c = box.checkout(CLOSED) + _, rows = sweep() + assert rows[str(c)][0] == "remove", rows[str(c)] + _hook_on_fetch_of(box, c, f"printf late > {c}/handoffs/late.md") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert "untracked: 1 (handoffs/late.md)" in rows[str(c)][1], rows[str(c)] + assert (c / "handoffs" / "late.md").is_file() + assert "fetch" in box.net_ops(), "the candidate was fetched before it was decided" + + +def test_a_lane_resumed_after_the_listing_is_kept(sweep): + """THE LANE'S STATE IS ASKED AGAIN UNDER THE LOCK, AFTER A FETCH (rv1b's + MUSTs 1-2): a CLOSED lane that RESUMED on origin after the listing read it + is kept.""" + box = sweep.box + c = box.checkout(CLOSED) + resumed = (f"RESUMED — lane {CLOSED}, session {UUID[CLOSED]}@Eagle, " + f"2026-10-21T09:00:00Z, lane:{CLOSED}\n") + peer = box.seed + _hook_on_fetch_of(box, c, ( + f"git -C {peer} pull -q --ff-only origin main && " + f"printf '%s' '{resumed}' >> {peer}/lanes/log/{CLOSED}.md && " + f"git -C {peer} commit -q -am resumed && git -C {peer} push -q origin main")) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"lane {CLOSED} is IDLE now, not CLOSED" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + +def test_a_fetch_that_fails_holds(sweep): + """`--yes` decides on a fetched origin: a candidate whose fetch fails is + held, and its lock released.""" + box = sweep.box + c = box.checkout(CLOSED) + box.use_net(hook=f'case " $* " in *" -C {c} fetch "*) exit 1 ;; esac\nexit 0\n') + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert "its fetch of origin main failed" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() and not (c / "lanes" / ".lanes-edit.lock").exists() + + +def test_nothing_outside_W_lanes_is_removed(sweep): + """The clones beside W are listed with what holds them, and `--yes` removes + none of them: nothing outside `-lanes/` is ever removed.""" + box = sweep.box + before = {str(c): box.everything()[str(c)] for c in box.profile_clones} + _, rows = sweep() + assert rows[str(box.profile_clones[0])] == ("beside", f"holds: leaned on by {box.profile_clones[1]} (its alternates file names this clone's objects)"), rows + assert rows[str(box.profile_clones[2])][1].startswith("nothing holds it"), rows + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert {str(c): box.everything()[str(c)] for c in box.profile_clones} == before + assert all(rows[str(c)] == ("beside", "left") for c in box.profile_clones), rows From b1eac0d6ba9c48a6f01de40429c4bffa3b062f83 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Wed, 21 Oct 2026 15:05:56 +0000 Subject: [PATCH 03/53] Amendment 23 S6.2: `lane-end --sweep-checkouts --yes` removes a chain of closed lanes' checkouts leaves first, keeps every clone a held one borrows from, and records what it removed in one NOTED line through the runner's own checkout (#198, slice 4 stage 3) Stage 3 of slice 4: the dependents order and the NOTED line. - Leaves first (S6.2 (b), D21): the candidates are taken in dependency order, every clone before the clones it borrows objects from, by passes that each place every candidate no unplaced candidate leans on, in listed order. A cycle places nothing more, and what is left goes last, held by what leans on it. The order is a plan: `--yes` still reads what leans on each candidate again just before its removal, so a held dependent keeps its whole chain. - What leans on a clone (D18): a repository whose alternates file names an object directory AT OR UNDER the clone, not only its own `.git/objects`, because removing the clone removes any store kept inside it. - The NOTED line (D9, D15, D20): after the removals, and only when something was removed, ONE `lanes-edit.sh log NOTED lane:` line in the runner's own log, written through the runner's own checkout under S3 and S4, in a subshell with `LANES_NO_FETCH` unset. Its text is built only from names held to letters, digits, `.`, `_` and `-`, and names no host path. A line the writer refuses exits 1 with the exact command that writes it by hand. - The session field (D19): `--yes` takes `LANES_SESSION`, or the lane's last recorded session, and refuses (2) before anything is removed when that is not a transcript uuid. Tests: tests/test_a23_sweep_checkouts.py, 47 cases (37 of stages 1-2, one of them now checking the runner's NOTED commit, and 10 new: leaves first in one run, a held link of the chain three ways, the NOTED line through the runner's checkout, nothing removed writes nothing, a refused NOTED line exits 1 with a working by-hand command, a non-uuid session refused, a store kept inside a clone held, and a directory sharing W's git directory kept as W). The CHAIN lanes repoS4-5, 6 and 7 are ENDED rows and logs in the fixture; their checkouts are made by the cases that need them. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 2140189e2c7850475a1135a9ac7b07bc2219fcca) --- lane-end | 130 ++++++++++++++-- tests/test_a23_sweep_checkouts.py | 247 +++++++++++++++++++++++++++++- 2 files changed, 355 insertions(+), 22 deletions(-) diff --git a/lane-end b/lane-end index 0a759e5..456367f 100755 --- a/lane-end +++ b/lane-end @@ -270,7 +270,8 @@ # residue still on disk, or the sweep refusing the lane. # `--sweep-checkouts`: `--yes` with no `LANES_LANE`, or with # `LANES_NO_FETCH=1`; a W that is itself a lane checkout; a runner lane -# the register does not know +# the register does not know, or one with no transcript uuid for the +# session field of its NOTED line # 3 `--sweep-checkouts`, the dry run: something under `-lanes/` is # removable (0 is nothing to remove) # 8 nothing there. `--retire-dormant`: no lane of that repository is @@ -1118,6 +1119,27 @@ SC_IDX die "lane $sc_runner, which LANES_LANE names, has no row in the register and no log (\`lanes --all\` does not list it), so the NOTED line this act ends with has no lane to be written for. Name your own lane: LANES_LANE= $prog --sweep-checkouts --yes. Nothing was removed." 2 fi fi + # THE NOTED LINE'S SESSION FIELD, found before anything is removed. It is + # the runner's transcript uuid and only that (Amendment 7(b)): the caller's + # `LANES_SESSION`, or the lane's last recorded session. A line the writer + # would refuse for want of one is refused here instead, while nothing is + # gone, rather than met after the removals it was to record. The writer + # asks again when the line is written, and its answer is the decision. + sc_uuid="" + if [ "$sweep_yes" = 1 ]; then + sc_uuid="${LANES_SESSION:-}" + if [ -z "$sc_uuid" ]; then + sc_uuid="$(LANES_NO_FETCH=1 "$LANES_EDIT" last-session "$sc_runner" 2>/dev/null)" || sc_uuid="" + fi + if ! is_fork_uuid "$sc_uuid"; then + if [ -n "${LANES_SESSION:-}" ]; then + sc_uwhy="LANES_SESSION is '$LANES_SESSION', which is not a transcript uuid" + else + sc_uwhy="LANES_SESSION is unset, and \`lanes-edit.sh last-session $sc_runner\` names none" + fi + die "lane $sc_runner has no transcript uuid for the session field of the NOTED line this act ends with (Amendment 7(b)): $sc_uwhy. Name your session: LANES_SESSION=\"\$CLAUDE_CODE_SESSION_ID\" LANES_LANE=$sc_runner $prog --sweep-checkouts --yes. Nothing was removed." 2 + fi + fi # THE CHECKOUT A CALLER'S `LANES_REPO` NAMES, by its real toplevel. Eagle's # lane windows export one at the lane's own checkout, and a write may go # through it; it is never swept. @@ -1385,10 +1407,14 @@ SH_REFS done < "$sds_a" done } + # A REPOSITORY LEANS ON A CLONE when its alternates file names an object + # directory AT OR UNDER that clone — its own `.git/objects`, or a store kept + # anywhere inside it — because removing the clone removes that store too. sc_dependents() { # [] — what leans on it - scd_obj="$1/.git/objects"; scd_out="" + scd_out="" while IFS="$sc_tab" read -r scd_d scd_t; do - [ -n "$scd_d" ] && [ "$scd_t" = "$scd_obj" ] && [ "$scd_d" != "$1" ] || continue + [ -n "$scd_d" ] && [ "$scd_d" != "$1" ] || continue + case "$scd_t" in "$1" | "$1"/*) : ;; *) continue ;; esac case " ${2-} " in *" $scd_d "*) continue ;; esac scd_out="${scd_out:+$scd_out }$scd_d" done < — 0 when a candidate not yet placed leans on it + slu_c="${sc_cands[$1]}" + while IFS="$sc_tab" read -r slu_d slu_t; do + [ -n "$slu_d" ] && [ "$slu_d" != "$slu_c" ] || continue + case "$slu_t" in "$slu_c" | "$slu_c"/*) : ;; *) continue ;; esac + slu_j=0 + while [ "$slu_j" -lt "${#sc_cands[@]}" ]; do + if [ "${sc_placed[$slu_j]}" = 0 ] && [ "${sc_cands[$slu_j]}" = "$slu_d" ]; then return 0; fi + slu_j=$((slu_j + 1)) + done + done < + printf ' hold %s %s; holds: %s\n' "$1" "$2" "$3" + sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${1##*/}" + } + for sc_i in ${sc_order[@]+"${sc_order[@]}"}; do sc_c="${sc_cands[$sc_i]}"; sc_why="${sc_cand_why[$sc_i]}" sc_l="${sc_cand_lane[$sc_i]}"; sc_new="${sc_cand_new[$sc_i]}" - sc_i=$((sc_i + 1)) if ! sc_under_home "$sc_c"; then printf ' hold %s %s; it is not directly under %s/, and nothing else is ever removed\n' "$sc_c" "$sc_why" "$sc_home" - sc_n_held=$((sc_n_held + 1)); continue + sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${sc_c##*/}"; continue fi if [ "$sweep_yes" != 1 ]; then sc_judge_holds "$sc_c" 0 "$sc_gone" if [ -n "$SC_HOLD" ]; then - printf ' hold %s %s; holds: %s\n' "$sc_c" "$sc_why" "$SC_HOLD" - sc_n_held=$((sc_n_held + 1)) + sc_say_held "$sc_c" "$sc_why" "$SC_HOLD" else printf ' remove %s %s%s\n' "$sc_c" "$sc_why" "${SC_IGN:+; $SC_IGN}" sc_gone="${sc_gone:+$sc_gone }$sc_c"; sc_n_removable=$((sc_n_removable + 1)) @@ -1577,8 +1651,7 @@ SS_ROW # then, it is moved aside under `-lanes/` — so its S1 path is absent at # once and the next write there clones afresh — and the aside removed. if ! sc_lock_take "$sc_c"; then - printf ' hold %s %s; holds: %s\n' "$sc_c" "$sc_why" "$SC_WHY" - sc_n_held=$((sc_n_held + 1)); continue + sc_say_held "$sc_c" "$sc_why" "$SC_WHY"; continue fi sc_act_hold="" if ! sc_fetch "$sc_c"; then @@ -1592,22 +1665,22 @@ SS_ROW fi if [ -n "$sc_act_hold" ]; then sc_lock_drop - printf ' hold %s %s; holds: %s\n' "$sc_c" "$sc_why" "$sc_act_hold" - sc_n_held=$((sc_n_held + 1)); continue + sc_say_held "$sc_c" "$sc_why" "$sc_act_hold"; continue fi sc_aside="$sc_home/.${sc_c##*/}.removing-$sc_utc-$$" if ! sc_under_home "$sc_c" || [ -e "$sc_aside" ] || [ -L "$sc_aside" ] || ! mv -- "$sc_c" "$sc_aside" 2>/dev/null; then sc_lock_drop - printf ' hold %s %s; holds: it could not be moved aside to %s\n' "$sc_c" "$sc_why" "$sc_aside" - sc_n_held=$((sc_n_held + 1)); continue + sc_say_held "$sc_c" "$sc_why" "it could not be moved aside to $sc_aside"; continue fi SC_LOCK="" # it went with the checkout if sc_under_home "$sc_aside"; then rm -rf -- "$sc_aside" 2>/dev/null || :; fi if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then printf ' PARTLY %s %s; moved aside to %s, which could not all be removed\n' "$sc_c" "$sc_why" "$sc_aside" sc_partial="${sc_partial:+$sc_partial }$sc_aside" + sc_partial_names="${sc_partial_names:+$sc_partial_names }${sc_aside##*/}" else printf ' removed %s %s\n' "$sc_c" "$sc_why" + sc_removed_names="${sc_removed_names:+$sc_removed_names }${sc_c##*/}" fi sc_gone="${sc_gone:+$sc_gone }$sc_c"; sc_n_removed=$((sc_n_removed + 1)) done @@ -1642,6 +1715,33 @@ SS_ROW fi printf '\n %s removed, %s held under %s/; %s clone(s) beside W left.\n' \ "$sc_n_removed" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" + # THE NOTED LINE (S6.2; S2 lists the sweep among the writers): ONE line in + # the runner's own log, naming what was removed and what was held, written + # by `lanes-edit.sh log` and so through the runner's own checkout under S3 + # and S4. It is written with `LANES_NO_FETCH` UNSET, because S3 refuses a + # writer whose fetch did not run (ratified choice 2; rv1b's MUST 3 on + # `lane-worktrees`). Nothing removed is nothing to record, and nothing is + # written. The paths are spelled from the projects root, `-lanes/`, + # never from the host's own root, and every name in the line is one this + # sweep already holds to letters, digits, `.`, `_` and `-`, so the free + # text's own rules (no ` — `, no newline, no leading arrow) hold for it. + if [ "$sc_n_removed" -gt 0 ]; then + sc_rel="${sc_home##*/}" + case "$sc_rel" in '' | *[!A-Za-z0-9._-]*) sc_rel="-lanes" ;; esac + sc_text="sweep-checkouts (Amendment 23 S6.2): removed $sc_n_removed register checkout(s) under $sc_rel/ that held nothing unpublished" + if [ -n "$sc_removed_names" ]; then sc_text="$sc_text: $sc_removed_names"; fi + if [ -n "$sc_partial_names" ]; then sc_text="$sc_text; moved aside and not all removed: $sc_partial_names"; fi + if [ -n "$sc_held_names" ]; then sc_text="$sc_text; held and left: $sc_held_names"; fi + sc_text="$sc_text; ${#sc_beside[@]} clone(s) beside W listed and left" + sc_noted_rc=0 + ( unset LANES_NO_FETCH + LANES_LANE="$sc_runner" LANES_SESSION="$sc_uuid" "$LANES_EDIT" log NOTED "lane:$sc_runner" "$sc_text" ) >&2 || + sc_noted_rc=$? + if [ "$sc_noted_rc" != 0 ]; then + die "the NOTED line recording this sweep was NOT written (\`lanes-edit.sh log\` exited $sc_noted_rc, above), so lane $sc_runner's log does not say what was removed. Nothing was lost: each checkout removed held nothing unpublished. Write the line by hand: LANES_LANE=$(printf '%q' "$sc_runner") LANES_SESSION=$sc_uuid lanes-edit.sh log NOTED $(printf '%q' "lane:$sc_runner") $(printf '%q' "$sc_text")" 1 + fi + printf ' NOTED in lane %s'"'"'s log: %s\n' "$sc_runner" "$sc_text" + fi if [ -n "$sc_partial" ]; then die "the removal of $sc_partial did not finish (above): what is left is no checkout, under a hidden name, so the next write of that lane clones afresh. Remove it by hand once you have looked at why." 1 fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 5a23535..530fa46 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -43,9 +43,11 @@ LIVE = "repoS4-3" # a live session record here names it PAUSED = "repoS4-4" # PAUSED: a lane somebody may pick up OLD, NEW = "repoS4-8", "repoS4-9" # renamed: OLD is a former name of NEW +CHAIN = ("repoS4-5", "repoS4-6", "repoS4-7") # ENDED: checkouts made by `_chain` UTC = "2026-10-20T10:00:00Z" UUID = {lane: f"a23a{n}04d-5404-4000-8000-a23a{n}04d5404" - for n, lane in ((1, RUNNER), (2, CLOSED), (3, LIVE), (4, PAUSED), (9, NEW))} + for n, lane in ((1, RUNNER), (2, CLOSED), (3, LIVE), (4, PAUSED), + (5, CHAIN[0]), (6, CHAIN[1]), (7, CHAIN[2]), (9, NEW))} PROFILE_CLONES = ("brett-wip-team01-laneA", "brett-wip-team02-laneA", "brett-wip-team03-laneA") @@ -66,7 +68,9 @@ class Sweep(RegisterBox): `-lanes/` holds: the runner's, the CLOSED lane's, the LIVE lane's and the PAUSED lane's checkouts; OLD's beside NEW's; a directory that is no - checkout, a symbolic link and a hidden creation lock. Beside W: three + checkout, a symbolic link and a hidden creation lock. The CHAIN lanes are + ENDED in the register and the logs, and have no checkout until `_chain` + makes them. Beside W: three per-profile clones, each borrowing from the one before it and the first from W (c3 → c2 → c1 → W), and a clone of another repository.""" @@ -77,12 +81,14 @@ def __init__(self, root: Path) -> None: _row(CLOSED, "ENDED · closed"), _row(LIVE, "LIVE · working"), _row(PAUSED, "PAUSED · swapped"), - _row(NEW, "LIVE · renamed from repoS4-8")]), + _row(NEW, "LIVE · renamed from repoS4-8"), + *(_row(lane, "ENDED · closed") for lane in CHAIN)]), f"lanes/log/{RUNNER}.md": _log(RUNNER, "STARTED"), f"lanes/log/{CLOSED}.md": _log(CLOSED, "STARTED", "ENDED"), f"lanes/log/{LIVE}.md": _log(LIVE, "STARTED"), f"lanes/log/{PAUSED}.md": _log(PAUSED, "STARTED", "PAUSED"), f"lanes/log/{NEW}.md": _log(NEW, "STARTED"), + **{f"lanes/log/{lane}.md": _log(lane, "STARTED", "ENDED") for lane in CHAIN}, "lanes/aliases.tsv": f"{OLD}\t{NEW}\t{UTC}\n", "handoffs/README.md": "# handoffs\n", } @@ -193,7 +199,7 @@ def test_dry_run_lists_and_removes_nothing(sweep): assert box.everything() == before, "a dry run changes nothing under -lanes/ or beside W" assert box.origin_head() == head, "nothing reached origin" assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "a dry run takes no lock" - assert f"LANES_LANE= lane-end --sweep-checkouts --yes" in result.stdout + assert "LANES_LANE= lane-end --sweep-checkouts --yes" in result.stdout def test_dry_run_with_nothing_to_remove_exits_0(sweep): @@ -317,10 +323,11 @@ def _no_aside(box: Sweep) -> None: def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): """`--yes`, run by a lane, removes the CLOSED lane's checkout and the former name's beside its new one, and leaves every other entry — the LIVE lane's, - the PAUSED one's, its own, a non-clone, a link — and every clone beside W - exactly as they were.""" + the PAUSED one's, a non-clone, a link — and every clone beside W exactly + as they were. Its own checkout moves only by the NOTED commit it pushed.""" box = sweep.box before = box.everything() + runner = str(box.checkout(RUNNER)) gone = {str(box.checkout(CLOSED)), str(box.checkout(OLD))} result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 0, (result.stdout, result.stderr) @@ -329,8 +336,12 @@ def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): for path in gone: assert not os.path.lexists(path), path after = box.everything() - assert {k: v for k, v in before.items() if k not in gone} == after, \ + assert {k: v for k, v in before.items() if k not in gone and k != runner} == \ + {k: v for k, v in after.items() if k != runner}, \ "nothing but the two removed entries changed, under -lanes/ or beside W" + assert box.git("-C", runner, "rev-parse", "HEAD").strip() == box.origin_head(), \ + "the runner's checkout moved only to the NOTED commit it pushed" + assert after[runner][1:] == before[runner][1:], "and holds nothing new" assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones} _no_aside(box) assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "every lock it took is released" @@ -579,3 +590,225 @@ def test_nothing_outside_W_lanes_is_removed(sweep): assert result.returncode == 0, (result.stdout, result.stderr) assert {str(c): box.everything()[str(c)] for c in box.profile_clones} == before assert all(rows[str(c)] == ("beside", "left") for c in box.profile_clones), rows + + +# ------------------------------------------------- leaves first, and chains + +def _chain(box: Sweep) -> list: + """Three CLOSED lanes' checkouts under `-lanes/`, each borrowing objects + from the one before it: 7 from 6, 6 from 5, and 5 from W.""" + out, previous = [], box.w + for lane in CHAIN: + previous = box.clone(box.checkout(lane), reference=previous) + out.append(previous) + return out + + +def _entries(text: str, paths) -> list: + """`(tag, path)` for each report line naming one of `paths`, in the order + the report gives them.""" + want = {str(p) for p in paths} + out = [] + for line in text.splitlines(): + if not line.startswith(" ") or line.startswith(" "): + continue + tag, _, rest = line.strip().partition(" ") + path = rest.strip().partition(" ")[0] + if path in want: + out.append((tag, path)) + return out + + +def test_leaves_first_in_one_run(sweep, tmp_path): + """S6.2 (b): *"so leaves go first"*. A chain of CLOSED lanes' checkouts, + each borrowing objects from the one before it (7 → 6 → 5 → W), is all + removable in ONE run: the dry run lists 7, 6, 5 as removable in that + order, though the directory lists them 5, 6, 7, and `--yes` removes them + in that order. W, which 5 borrows from, is never a candidate.""" + box = sweep.box + chain = _chain(box) + leaves_first = [str(c) for c in reversed(chain)] + result, _ = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert _entries(result.stdout, chain) == [("remove", p) for p in leaves_first], result.stdout + result, _ = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert _entries(result.stdout, chain) == [("removed", p) for p in leaves_first], result.stdout + _assert_under(Path(os.path.realpath(tmp_path)), chain) + for c in chain: + assert not os.path.lexists(c), c + box.git("-C", str(box.w), "fsck", "--connectivity-only") # W, which 5 borrowed from, is whole + _no_aside(box) + + +@pytest.mark.parametrize("held,expect", [ + (2, {2: "commit(s) origin lacks", 1: "leaned on by {2}", 0: "leaned on by {1}"}), + (1, {2: "removed", 1: "commit(s) origin lacks", 0: "leaned on by {1}"}), + (0, {2: "removed", 1: "removed", 0: "commit(s) origin lacks"}), +], ids=["the leaf held", "the middle held", "the root held"]) +def test_held_parent_keeps_its_dependents_chain(sweep, held, expect): + """S6.2 (b): *"A clone still depended on by a held clone stays."* One + link of the chain 7 → 6 → 5 holds an unpublished commit. Everything that + link borrows from, down to W, is held by it — in the dry run and under + `--yes`, which reads what leans on each candidate again just before its + removal — while every link that borrows from it, held by nothing, goes.""" + box = sweep.box + chain = _chain(box) + _commit_unpushed(box, chain[held]) + for args, env, gone in ((), {}, "remove"), (("--yes",), {"LANES_LANE": RUNNER}, "removed"): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + for i, said in expect.items(): + c = str(chain[i]) + if said == "removed": + assert rows[c][0] == gone, (c, rows[c]) + else: + assert rows[c][0] == "hold", (c, rows[c]) + assert said.format(*chain) in rows[c][1], (c, rows[c]) + for i, said in expect.items(): + assert os.path.lexists(chain[i]) == (said != "removed"), chain[i] + + +# ----------------------------------------------------------- the NOTED line + +def test_noted_line_written_through_runner_checkout(sweep): + """`--yes` ends with ONE NOTED line in the runner's own log, naming what it + removed, written as the runner's session through the RUNNER'S OWN + checkout (S2) under S3 and S4: the commit on origin is that checkout's + HEAD, W's own HEAD does not move, and that checkout fetched origin before + it pushed (no `LANES_NO_FETCH`; rv1b's MUST 3).""" + box = sweep.box + box.use_net() + runner = box.checkout(RUNNER) + w_head = box.git("-C", str(box.w), "rev-parse", "HEAD").strip() + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + noted = [line for line in box.origin_file(f"lanes/log/{RUNNER}.md").splitlines() + if line.startswith("NOTED")] + assert len(noted) == 1, noted + assert f"lane {RUNNER}, session {UUID[RUNNER]}@" in noted[0], noted + assert noted[0].endswith( + "sweep-checkouts (Amendment 23 S6.2): removed 2 register checkout(s) under " + f"brett-wip-lanes/ that held nothing unpublished: {CLOSED} {OLD}; " + "3 clone(s) beside W listed and left"), noted + assert str(box.root) not in noted[0], "no host path is written into the log" + assert box.git("-C", str(runner), "rev-parse", "HEAD").strip() == box.origin_head(), \ + "the NOTED commit is the runner's checkout's own" + assert box.git("-C", str(box.w), "rev-parse", "HEAD").strip() == w_head, "W's HEAD did not move" + lines = box.net_log.read_text(encoding="utf-8").splitlines() + mine = [i for i, line in enumerate(lines) if f"-C {runner} " in line] + fetches = [i for i in mine if lines[i].startswith("fetch ")] + pushes = [i for i in mine if lines[i].startswith("push ")] + assert fetches and pushes and fetches[0] < pushes[0], lines + assert "NOTED in lane" in result.stdout, result.stdout + + +def test_nothing_removed_writes_nothing(sweep): + """Nothing removed is nothing to record: with the CLOSED lane's checkout + held and the former name's gone, `--yes` removes nothing, writes no line + and makes no commit.""" + box = sweep.box + box.checkout(OLD).rename(box.root / f"aside-{OLD}") + _commit_unpushed(box, box.checkout(CLOSED)) + head = box.origin_head() + runner_head = box.git("-C", str(box.checkout(RUNNER)), "rev-parse", "HEAD").strip() + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert not _tagged(rows, "removed"), rows + assert box.origin_head() == head and "NOTED" not in box.origin_file(f"lanes/log/{RUNNER}.md") + assert box.git("-C", str(box.checkout(RUNNER)), "rev-parse", "HEAD").strip() == runner_head + assert "NOTED in lane" not in result.stdout + + +REFUSING_LOG = """#!/usr/bin/env bash +if [ "${1-}" = log ]; then + printf 'FAKE: the register refused this line\\n' >&2 + exit 2 +fi +exec bash %s "$@" +""" + + +def test_noted_line_not_written_exits_1_with_the_command(sweep): + """A NOTED line the writer refuses after the removals is not a quiet + success: exit 1, and the exact command that writes it by hand, which + then writes it.""" + box = sweep.box + wrapper = box.fakebin / "lanes-edit-refusing-log" + wrapper.write_text(REFUSING_LOG % (REPO / "lanes-edit.sh"), encoding="utf-8") + wrapper.chmod(0o755) + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=str(wrapper)) + assert result.returncode == 1, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert "the NOTED line recording this sweep was NOT written" in result.stderr, result.stderr + assert "Nothing was lost" in result.stderr + by_hand = result.stderr.split("Write the line by hand: ", 1)[1].strip() + assert by_hand.startswith(f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), by_hand + assert "NOTED" not in box.origin_file(f"lanes/log/{RUNNER}.md") + done = subprocess.run(["bash", "-c", by_hand], cwd=str(box.root), capture_output=True, text=True, + env=dict(box.env, PATH=f"{REPO}{os.pathsep}{box.env['PATH']}"), + timeout=180, check=False) + assert done.returncode == 0, (done.stdout, done.stderr) + assert "sweep-checkouts (Amendment 23 S6.2): removed 2" in box.origin_file(f"lanes/log/{RUNNER}.md") + + +def test_yes_without_a_session_uuid_is_refused(sweep): + """The NOTED line's session field is a transcript uuid and only that + (Amendment 7(b)): a `LANES_SESSION` that is none is refused before + anything is removed, not met by the line after the removals.""" + box = sweep.box + before, head = box.everything(), box.origin_head() + result, _ = sweep("--yes", LANES_LANE=RUNNER, LANES_SESSION="unknown") + assert result.returncode == 2, (result.stdout, result.stderr) + assert "has no transcript uuid for the session field" in result.stderr, result.stderr + assert "Nothing was removed" in result.stderr + assert box.everything() == before and box.origin_head() == head + + +def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): + """D18: a repository LEANS ON A CLONE when its alternates file names an + object directory anywhere AT OR UNDER that clone, not only the clone's own + `.git/objects`: removing the clone removes that store too. Here a store + kept inside the CLOSED lane's `.git` is borrowed by a repository beside W, + and the CLOSED lane's checkout is held by it, in the dry run and under + `--yes`, and that repository stays whole.""" + box = sweep.box + c = box.checkout(CLOSED) + store = c / ".git" / "kept-store" + box.git("init", "-q", "--bare", str(store)) + borrower = box.w.parent / "borrower" + box.git("init", "-q", "-b", "main", str(borrower)) + box.write(borrower, "f.txt", "kept\n") + box.commit(borrower, "a commit whose objects go to the store", "f.txt") + objects = borrower / ".git" / "objects" + for entry in objects.iterdir(): + if entry.name not in ("info", "pack"): + shutil.copytree(entry, store / "objects" / entry.name, dirs_exist_ok=True) + (objects / "info" / "alternates").write_text(f"{store / 'objects'}\n", encoding="utf-8") + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"leaned on by {borrower} " in rows[str(c)][1], rows[str(c)] + assert c.is_dir() and store.is_dir() + box.git("-C", str(borrower), "fsck", "--connectivity-only") + + +def test_a_directory_sharing_Ws_git_directory_is_kept_as_W(sweep): + """W IS NEVER A CANDIDATE, by its git directory as well as by its path: a + directory under `-lanes/`, named for a CLOSED lane, whose own `.git` + names W's as its common directory (a linked worktree's admin files copied + by hand) IS W's repository, and is kept as W, by the dry run and `--yes`.""" + box = sweep.box + d = box.checkout(CHAIN[0]) + (d / ".git").mkdir(parents=True) + (d / ".git" / "commondir").write_text(f"{box.w / '.git'}\n", encoding="utf-8") + (d / ".git" / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8") + assert box.git("-C", str(d), "rev-parse", "--git-common-dir").strip() == str(box.w / ".git") + w_before = box.w_state() + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(d)] == ("keep", "W itself, the readers' copy"), rows[str(d)] + assert d.is_dir() and box.w_state() == w_before From 3f18f87d3c1691376885198fe90c8797fb2ecd1f Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Wed, 21 Oct 2026 22:14:44 +0000 Subject: [PATCH 04/53] Amendment 23 S6.2: `lane-end --sweep-checkouts --yes` no longer decides on a register whose fetch of W failed, at the listing or at the re-ask under each checkout's lock (rv4's MUST 1, #198, slice 4) `lanes --fetch` exits 0 on `log_sync`'s `fell-back` (ls-remote answered, the fetch of W then failed or timed out) and reads the stale ref; it says so only in its notes, with the phrase "reading the logs as they stand locally". The sweep matched only "NO FETCH WAS MADE" at the listing and sent the re-ask's stderr to /dev/null, so `--yes` took a stale register for "fetched just now" and removed the checkout of a lane that had RESUMED on origin. - The listing (`--yes`): the refusal matches the shared phrase, so a failed fetch of W refuses with 1 and "Nothing was removed", as a fetch never made does. - `sc_still`: the re-ask's stderr is captured (into `LE_TMP`, which `cleanup` removes on every exit) and the checkout is held, with a reason that says the state was not read fresh, when it says "reading the logs as they stand locally", "session records could not be read" or "could not be read, so its rows are not in this listing". A capture that cannot be made or read back holds too. - Tests (`tests/test_a23_sweep_checkouts.py`), each red on `cede054` and green here: a W fetch that fails at the listing refuses `--yes` and changes nothing (rv4's P5); a lane that RESUMES at its checkout's fetch while W's fetch fails from then on is kept (P1); session records that become unreadable at the re-check keep the checkout. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Sonnet 5.5 (cherry picked from commit b896230ddb1aab37138889d048c33582bc4ec713) --- lane-end | 38 +++++++++++++++-- tests/test_a23_sweep_checkouts.py | 70 +++++++++++++++++++++++++++++++ 2 files changed, 105 insertions(+), 3 deletions(-) diff --git a/lane-end b/lane-end index 456367f..9e66d56 100755 --- a/lane-end +++ b/lane-end @@ -1073,9 +1073,17 @@ if [ "$sweep_checkouts" = 1 ]; then die "the register could not be read (the listing says so above), so which lanes are CLOSED was never established. Fix the read and re-run. Nothing was removed." 1 ;; esac if [ "$sweep_yes" = 1 ]; then + # `lanes --fetch` says "reading the logs as they stand locally" on EVERY + # way a fetch of W did not bring the register in: "NO FETCH WAS MADE" + # where none was tried, and `log_sync`'s `fell-back` where `ls-remote` + # answered and the fetch then failed or timed out, which reads the stale + # ref and exits 0 (rv4's MUST 1). Matching the first alone let a failed + # fetch pass as "fetched just now". The phrase is `lanes-edit.sh`'s own, + # for callers that match it (`lanes`' comment), and `sc_still` matches it + # again under each checkout's lock. case "$sc_err" in - *"NO FETCH WAS MADE"*) - die "--yes decides on a register fetched just now, and W's fetch was not made (the listing says why above). Nothing was removed." 1 ;; + *"reading the logs as they stand locally"*) + die "--yes decides on a register fetched just now, and W's fetch did not complete: none was made, or it failed or timed out and the logs were read as they stand locally (the listing says why above). Nothing was removed." 1 ;; esac fi case "$sc_rc" in @@ -1532,9 +1540,33 @@ SCC # THE LANE'S STATE, ASKED AGAIN UNDER THE CHECKOUT'S LOCK after a fetch of # W (rv1b's MUSTs 1-2: a decision made from register state is made after # the fetch, under the lock). 0 when it is still what made it a candidate. + # + # THE RE-ASK'S STDERR IS READ, as the listing's is (rv4's MUST 1): `lanes + # --fetch` exits 0 on a W fetch that failed after `ls-remote` answered + # (`fell-back`) and on records it could not read, and says so only in its + # notes. A row read from a stale ref is not a state "after a fetch", so a + # re-ask whose notes say any of these holds the checkout. sc_still() { # ss_rc=0 - ss_row="$("$LANES_EDIT" lanes --fetch --closed --lane "$2" 2>/dev/null)" || ss_rc=$? + LE_TMP="$(mktemp "${TMPDIR:-/tmp}/lane-end-sweep.XXXXXX" 2>/dev/null || printf '')" + if [ -z "$LE_TMP" ]; then + SC_WHY="no temporary file could be made to capture what lane $2's state read said, so whether it was read fresh under its checkout's lock was never established"; return 1 + fi + ss_row="$("$LANES_EDIT" lanes --fetch --closed --lane "$2" 2>"$LE_TMP")" || ss_rc=$? + ss_err="$(cat -- "$LE_TMP" 2>/dev/null)" || ss_err="could not be read, so its rows are not in this listing (the capture of the notes)" + rm -f -- "$LE_TMP"; LE_TMP="" + ss_why="" + case "$ss_err" in + *"reading the logs as they stand locally"*) + ss_why="W's fetch did not complete, so the logs were read as they stand locally" ;; + *"session records could not be read"*) + ss_why="this workstation's session records could not be read, so whether it is LIVE is not established" ;; + *"could not be read, so its rows are not in this listing"*) + ss_why="the register could not be read, so whether it is CLOSED is not established" ;; + esac + if [ -n "$ss_why" ]; then + SC_WHY="lane $2's state was not read fresh under its checkout's lock: $ss_why"; return 1 + fi if [ "$ss_rc" != 0 ] || [ -z "$ss_row" ]; then SC_WHY="lane $2's state could not be read again under its checkout's lock (lanes exited $ss_rc)"; return 1 fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 530fa46..bec2705 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -578,6 +578,76 @@ def test_a_fetch_that_fails_holds(sweep): assert c.is_dir() and not (c / "lanes" / ".lanes-edit.lock").exists() +def test_a_W_fetch_that_fails_at_the_listing_refuses_yes(sweep): + """`--yes` decides on a register fetched just now (D14). W's fetch FAILS + here while `ls-remote` succeeds, which `log_sync` says as `fell-back`: it + reads the stale ref and exits 0, and the listing's own note is the phrase + "reading the logs as they stand locally" without "NO FETCH WAS MADE" (rv4's + MUST 1, P5). Refused before the first removal, with nothing written.""" + box = sweep.box + before, head = box.everything(), box.origin_head() + box.use_net(hook=f'case " $* " in *" -C {box.w} fetch "*) exit 1 ;; esac\nexit 0\n') + result, _ = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 1, (result.stdout, result.stderr) + assert "reading the logs as they stand locally" in result.stderr, result.stderr + assert "W's fetch did not complete" in result.stderr, result.stderr + assert "Nothing was removed" in result.stderr, result.stderr + assert box.everything() == before + assert box.origin_head() == head + assert "push" not in box.net_ops() + + +def test_a_lane_resumed_while_W_cannot_fetch_is_kept(sweep): + """THE RE-CHECK UNDER THE LOCK IS A FRESH READ OR NONE (rv4's MUST 1, P1). + W fetches at the listing; the CLOSED lane then RESUMES on origin at its + own checkout's fetch, and W's fetch fails from then on, so the re-ask in + `sc_still` reads the stale ref, in which the lane is still CLOSED. That is + not a decision made after a fetch: the checkout is kept, and the reason + says why.""" + box = sweep.box + c = box.checkout(CLOSED) + marker = box.root / "resumed.marker" + resumed = (f"RESUMED — lane {CLOSED}, session {UUID[CLOSED]}@Eagle, " + f"2026-10-21T09:00:00Z, lane:{CLOSED}\n") + peer = box.seed + box.use_net(hook=( + f'case " $* " in\n' + f' *" -C {c} fetch "*) git -C {peer} pull -q --ff-only origin main && ' + f"printf '%s' '{resumed}' >> {peer}/lanes/log/{CLOSED}.md && " + f"git -C {peer} commit -q -am resumed && git -C {peer} push -q origin main && " + f'touch {marker} ;;\n' + f' *" -C {box.w} fetch "*) if [ -e {marker} ]; then exit 1; fi ;;\n' + f'esac\nexit 0\n')) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert marker.exists(), box.net_log.read_text(encoding="utf-8") + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert "was not read fresh" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root reads a directory whose mode is 000") +def test_session_records_unreadable_at_the_recheck_keep_the_checkout(sweep): + """The re-ask under the lock says "session records could not be read" on + stderr and exits 0, as the listing does; the listing's refusal is exit 1. + Here the records are readable at the listing and are not at the re-check, + so which lanes are LIVE is no longer established, and a LIVE lane's + checkout is never swept: the CLOSED lane's checkout is kept.""" + box = sweep.box + c = box.checkout(CLOSED) + sessions = box.home / ".claude" / "sessions" + _hook_on_fetch_of(box, c, f"chmod 000 {sessions}") + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + finally: + sessions.chmod(0o700) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert "was not read fresh" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + def test_nothing_outside_W_lanes_is_removed(sweep): """The clones beside W are listed with what holds them, and `--yes` removes none of them: nothing outside `-lanes/` is ever removed.""" From b923cbacc43e0981dd61ffb80f23473750fda48d Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:24:30 +0000 Subject: [PATCH 05/53] Amendment 23 S6.2: a re-ask under a checkout's lock whose captured notes cannot be read back holds that checkout by itself, with its own reason, and a register unreadable at the re-check gets a test of its own (rv4-r2's SHOULD 1, #198, slice 4) `sc_still` captured the re-ask's notes in `LE_TMP` and, where `cat` could not read the file back, put a stand-in sentence in their place that was written to match the "could not be read, so its rows are not in this listing" clause. So the checkout was held only through that clause, the reason it printed was false ("the register could not be read"), and a reword of the clause would have let the path fail open with no test to notice. - `sc_still`: a failed `cat` of the capture sets its own reason ("its capture could not be read back") and returns 1 at once, as the `mktemp` branch above it does, and as the listing refuses on both. - Tests (`tests/test_a23_sweep_checkouts.py`): - the capture goes mode 000 during the re-ask's own fetch of W (rv4-r2's probe): held, with the capture's reason and not the register's. Red at `b896230` (it named the register), green here; - the register cannot be read at the re-check: origin takes a commit that changes it, the re-ask's fetch of W brings it in, and its blob is then moved out of W's objects, so the tree names a register `git show` cannot read. Held, "the register could not be read". It passes at `b896230` too, since the clause was there; it is the test that now reaches the clause, which the capture's stand-in reached before. With the clause deleted, both the CLOSED lane's checkout and the former name's are removed. - `_push_from_peer`, a hook command in which the peer that seeded origin appends a line to a file and pushes it. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 2e58431d2483147ebd9e74ea712892f95f8a380c) --- lane-end | 10 ++++- tests/test_a23_sweep_checkouts.py | 71 +++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+), 2 deletions(-) diff --git a/lane-end b/lane-end index 9e66d56..bee8f0f 100755 --- a/lane-end +++ b/lane-end @@ -1545,7 +1545,9 @@ SCC # --fetch` exits 0 on a W fetch that failed after `ls-remote` answered # (`fell-back`) and on records it could not read, and says so only in its # notes. A row read from a stale ref is not a state "after a fetch", so a - # re-ask whose notes say any of these holds the checkout. + # re-ask whose notes say any of these holds the checkout. A capture that + # cannot be made, or cannot be read back, holds it by itself, with that as + # its reason (rv4-r2's SHOULD 1), as the listing refuses on both. sc_still() { # ss_rc=0 LE_TMP="$(mktemp "${TMPDIR:-/tmp}/lane-end-sweep.XXXXXX" 2>/dev/null || printf '')" @@ -1553,8 +1555,12 @@ SCC SC_WHY="no temporary file could be made to capture what lane $2's state read said, so whether it was read fresh under its checkout's lock was never established"; return 1 fi ss_row="$("$LANES_EDIT" lanes --fetch --closed --lane "$2" 2>"$LE_TMP")" || ss_rc=$? - ss_err="$(cat -- "$LE_TMP" 2>/dev/null)" || ss_err="could not be read, so its rows are not in this listing (the capture of the notes)" + ss_cat=0 + ss_err="$(cat -- "$LE_TMP" 2>/dev/null)" || ss_cat=$? rm -f -- "$LE_TMP"; LE_TMP="" + if [ "$ss_cat" != 0 ]; then + SC_WHY="what lane $2's state read said was captured, and its capture could not be read back, so whether it was read fresh under its checkout's lock was never established"; return 1 + fi ss_why="" case "$ss_err" in *"reading the logs as they stand locally"*) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index bec2705..8043bfe 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -528,6 +528,15 @@ def _hook_on_fetch_of(box: Sweep, target: Path, body: str) -> None: "exit 0\n")) +def _push_from_peer(box: Sweep, path: str, text: str, message: str) -> str: + """A hook's shell command: the peer that seeded origin appends `text` to + `path` and pushes it, as another session's write lands on origin.""" + peer = box.seed + return (f"git -C {peer} pull -q --ff-only origin main && " + f"printf '%s' '{text}' >> {peer}/{path} && " + f"git -C {peer} commit -q -am {message} && git -C {peer} push -q origin main") + + def test_recheck_before_removal(sweep): """THE HOLDS ARE READ AGAIN UNDER THE LOCK, AFTER THE FETCH: a handoff written into the checkout after the listing and before its removal holds @@ -648,6 +657,68 @@ def test_session_records_unreadable_at_the_recheck_keep_the_checkout(sweep): assert c.is_dir() +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root reads a file whose mode is 000") +def test_a_capture_that_cannot_be_read_back_holds_by_itself(sweep): + """THE RE-ASK'S NOTES ARE READ BACK, OR THE CHECKOUT IS HELD FOR THAT, with + its own reason (rv4-r2's SHOULD 1). The file `sc_still` captures the + re-ask's notes in goes mode 000 during the re-ask's own fetch of W, so + `cat` cannot read it back. The checkout is held because the capture could + not be read back, not by a stand-in sentence that matches another note's + clause and then names that note's reason.""" + box = sweep.box + c = box.checkout(CLOSED) + tmpdir = Path(box.env["TMPDIR"]) + marker = box.root / "fetched.marker" + box.use_net(hook=( + 'case " $* " in\n' + f' *" -C {c} fetch "*) touch {marker} ;;\n' + f' *" -C {box.w} fetch "*) if [ -e {marker} ]; then ' + f'chmod 000 {tmpdir}/lane-end-sweep.* 2>/dev/null || :; fi ;;\n' + 'esac\nexit 0\n')) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + finally: + for p in tmpdir.glob("lane-end-sweep.*"): + p.chmod(0o600) + assert marker.exists(), box.net_log.read_text(encoding="utf-8") + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert "its capture could not be read back" in rows[str(c)][1], rows[str(c)] + assert "the register could not be read" not in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + +def test_a_register_unreadable_at_the_recheck_keeps_the_checkout(sweep): + """The re-ask under the lock says the register "could not be read, so its + rows are not in this listing" and exits 0, as the listing does; the + listing's refusal is exit 1. Here the register reads at the listing; then + origin takes a commit that changes it, and the re-ask's own fetch of W + brings that commit in, after which its register blob is moved out of W's + objects. The tree still names the register, so it is published and `git + show` cannot read it: whether the lane is CLOSED is no longer established, + and the checkout is kept.""" + box = sweep.box + c = box.checkout(CLOSED) + marker = box.root / "pushed.marker" + aside = box.root / "register-blob.aside" + touched = _push_from_peer(box, "lanes/LANES.md", "\n", "touched") + box.use_net(hook=( + 'case " $* " in\n' + f' *" -C {c} fetch "*) if [ ! -e {marker} ]; then {touched} && touch {marker}; fi ;;\n' + f' *" -C {box.w} fetch "*) if [ -e {marker} ]; then shift; git "$@" || exit $?; ' + f'b="$(git -C {box.w} rev-parse refs/remotes/origin/main:lanes/LANES.md)" && ' + f'mv -- "{box.w}/.git/objects/${{b:0:2}}/${{b:2}}" {aside} 2>/dev/null; exit 97; fi ;;\n' + 'esac\nexit 0\n')) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert aside.is_file(), box.net_log.read_text(encoding="utf-8") + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert ("was not read fresh under its checkout's lock: the register could not be read" + in rows[str(c)][1]), rows[str(c)] + assert c.is_dir() + + def test_nothing_outside_W_lanes_is_removed(sweep): """The clones beside W are listed with what holds them, and `--yes` removes none of them: nothing outside `-lanes/` is ever removed.""" From ea6a07d35b169696c1a7243805a510df664a11b3 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:25:16 +0000 Subject: [PATCH 06/53] Amendment 23 S6.2 tests: the two cases in which a lane RESUMES on origin at its checkout's fetch share one hook command, `_resumed_on_origin`, instead of each spelling the peer's pull, append, commit and push (rv4-r2's NIT 2, #198, slice 4) The re-check case from rv1b's MUSTs 1-2 and the one from rv4's MUST 1 (P1) each carried the same five-step shell body, which Sonar's duplication check may flag. `_resumed_on_origin(box, lane)` builds it on `_push_from_peer`, and both cases call it. No case changes what it asserts. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit e152a89daa1ab9824646165c23bb42083a99e2d9) --- tests/test_a23_sweep_checkouts.py | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 8043bfe..dab603d 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -537,6 +537,14 @@ def _push_from_peer(box: Sweep, path: str, text: str, message: str) -> str: f"git -C {peer} commit -q -am {message} && git -C {peer} push -q origin main") +def _resumed_on_origin(box: Sweep, lane: str) -> str: + """A hook's shell command: `lane` RESUMES on origin, so it is no longer + CLOSED there.""" + line = (f"RESUMED — lane {lane}, session {UUID[lane]}@Eagle, " + f"2026-10-21T09:00:00Z, lane:{lane}\n") + return _push_from_peer(box, f"lanes/log/{lane}.md", line, "resumed") + + def test_recheck_before_removal(sweep): """THE HOLDS ARE READ AGAIN UNDER THE LOCK, AFTER THE FETCH: a handoff written into the checkout after the listing and before its removal holds @@ -560,13 +568,7 @@ def test_a_lane_resumed_after_the_listing_is_kept(sweep): is kept.""" box = sweep.box c = box.checkout(CLOSED) - resumed = (f"RESUMED — lane {CLOSED}, session {UUID[CLOSED]}@Eagle, " - f"2026-10-21T09:00:00Z, lane:{CLOSED}\n") - peer = box.seed - _hook_on_fetch_of(box, c, ( - f"git -C {peer} pull -q --ff-only origin main && " - f"printf '%s' '{resumed}' >> {peer}/lanes/log/{CLOSED}.md && " - f"git -C {peer} commit -q -am resumed && git -C {peer} push -q origin main")) + _hook_on_fetch_of(box, c, _resumed_on_origin(box, CLOSED)) result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 0, (result.stdout, result.stderr) assert rows[str(c)][0] == "hold", rows[str(c)] @@ -616,17 +618,11 @@ def test_a_lane_resumed_while_W_cannot_fetch_is_kept(sweep): box = sweep.box c = box.checkout(CLOSED) marker = box.root / "resumed.marker" - resumed = (f"RESUMED — lane {CLOSED}, session {UUID[CLOSED]}@Eagle, " - f"2026-10-21T09:00:00Z, lane:{CLOSED}\n") - peer = box.seed box.use_net(hook=( - f'case " $* " in\n' - f' *" -C {c} fetch "*) git -C {peer} pull -q --ff-only origin main && ' - f"printf '%s' '{resumed}' >> {peer}/lanes/log/{CLOSED}.md && " - f"git -C {peer} commit -q -am resumed && git -C {peer} push -q origin main && " - f'touch {marker} ;;\n' + 'case " $* " in\n' + f' *" -C {c} fetch "*) {_resumed_on_origin(box, CLOSED)} && touch {marker} ;;\n' f' *" -C {box.w} fetch "*) if [ -e {marker} ]; then exit 1; fi ;;\n' - f'esac\nexit 0\n')) + 'esac\nexit 0\n')) result, rows = sweep("--yes", LANES_LANE=RUNNER) assert marker.exists(), box.net_log.read_text(encoding="utf-8") assert result.returncode == 0, (result.stdout, result.stderr) From 36f08a1d2ca136fd7b173bcaa84c57363c57d01f Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:27:27 +0000 Subject: [PATCH 07/53] Amendment 23 S6.2 tests: the path on which W's `ls-remote` fails and NO FETCH IS MADE is pinned at the listing and at the re-ask under a checkout's lock, as the fetch that fails after `ls-remote` already was (rv4-r2's NIT 1, #198, slice 4) The module tested only `log_sync`'s `fell-back` (the fetch fails after `ls-remote` answers). `lanes --fetch` says the other way, an `ls-remote` that fails, as "NO FETCH WAS MADE ... reading the logs as they stand locally", and nothing here made it. - The listing's refusal case is parametrized over the failing operation: the fetch (`fell-back`), and `ls-remote` (no fetch). Each refuses `--yes` with 1 and changes nothing. - A new case: W's `ls-remote` fails from the candidate's own fetch on, so the re-ask makes no fetch, and the checkout is held as "not read fresh ... W's fetch did not complete". - With the listing's refusal disabled both listing cases fail, and with `sc_still`'s fetch clause disabled the new case fails (and so does P1). Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit dfec3c5c03f2a170e2aa2e45f42a736826de8be5) --- tests/test_a23_sweep_checkouts.py | 43 ++++++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index dab603d..9b4328a 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -589,17 +589,24 @@ def test_a_fetch_that_fails_holds(sweep): assert c.is_dir() and not (c / "lanes" / ".lanes-edit.lock").exists() -def test_a_W_fetch_that_fails_at_the_listing_refuses_yes(sweep): - """`--yes` decides on a register fetched just now (D14). W's fetch FAILS - here while `ls-remote` succeeds, which `log_sync` says as `fell-back`: it - reads the stale ref and exits 0, and the listing's own note is the phrase - "reading the logs as they stand locally" without "NO FETCH WAS MADE" (rv4's - MUST 1, P5). Refused before the first removal, with nothing written.""" +@pytest.mark.parametrize("op,note", [ + ("fetch", "fetch failed — reading the logs as they stand locally"), + ("ls-remote", "NO FETCH WAS MADE"), +], ids=["the fetch fails after ls-remote (fell-back)", "ls-remote fails (no fetch)"]) +def test_a_W_fetch_that_fails_at_the_listing_refuses_yes(sweep, op, note): + """`--yes` decides on a register fetched just now (D14). Either way W's + fetch can fail refuses it before the first removal, with nothing written: + the FETCH fails while `ls-remote` succeeds, which `log_sync` says as + `fell-back` — it reads the stale ref and exits 0, and the listing's own + note has the phrase "reading the logs as they stand locally" without "NO + FETCH WAS MADE" (rv4's MUST 1, P5) — or `ls-remote` fails, and no fetch is + made at all (rv4-r2's NIT 1).""" box = sweep.box before, head = box.everything(), box.origin_head() - box.use_net(hook=f'case " $* " in *" -C {box.w} fetch "*) exit 1 ;; esac\nexit 0\n') + box.use_net(hook=f'case " $* " in *" -C {box.w} {op} "*) exit 1 ;; esac\nexit 0\n') result, _ = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 1, (result.stdout, result.stderr) + assert note in result.stderr, result.stderr assert "reading the logs as they stand locally" in result.stderr, result.stderr assert "W's fetch did not complete" in result.stderr, result.stderr assert "Nothing was removed" in result.stderr, result.stderr @@ -631,6 +638,28 @@ def test_a_lane_resumed_while_W_cannot_fetch_is_kept(sweep): assert c.is_dir() +def test_a_W_ls_remote_that_fails_at_the_recheck_holds(sweep): + """THE RE-ASK MAKES NO FETCH AT ALL WHEN W'S `ls-remote` FAILS (rv4-r2's + NIT 1): W fetches at the listing, and its `ls-remote` fails from the + candidate's own fetch on, so the re-ask's note is "NO FETCH WAS MADE ... + reading the logs as they stand locally". The checkout is kept.""" + box = sweep.box + c = box.checkout(CLOSED) + marker = box.root / "fetched.marker" + box.use_net(hook=( + 'case " $* " in\n' + f' *" -C {c} fetch "*) touch {marker} ;;\n' + f' *" -C {box.w} ls-remote "*) if [ -e {marker} ]; then exit 1; fi ;;\n' + 'esac\nexit 0\n')) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert marker.exists(), box.net_log.read_text(encoding="utf-8") + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert ("was not read fresh under its checkout's lock: W's fetch did not complete" + in rows[str(c)][1]), rows[str(c)] + assert c.is_dir() + + @pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, reason="root reads a directory whose mode is 000") def test_session_records_unreadable_at_the_recheck_keep_the_checkout(sweep): From efc6ff7eeb5fc2f367dfc5fe78fb2f0f13b5f178 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:36:14 +0000 Subject: [PATCH 08/53] Amendment 23 S6.2: a NOTED write that exits 3 (pushed, not confirmed) or 6 (its commit kept) is no longer reported as "NOT written" with a by-hand command that would record the sweep twice; both are handed the publish act first (rv4's SHOULD 2, #198, slice 4) `lanes-edit.sh log`'s 3 means the line was committed and pushed and the read-back could not confirm it: it may be on origin. Its 6 means, in most of its causes, that the commit is kept in the checkout the write went through and only origin lacks it (a push refused with origin unmoved, the replay's bound), and in the rest that `git add` or `commit` failed with nothing committed. The sweep said "NOT written" for every non-zero exit and handed out `log NOTED ...` by hand, which then wrote a second line, or met S3's refusal of a checkout holding a commit origin lacks. - 3: the sweep says the line may be on origin already and is not to be written again, and prints the publish act (`LANES_LANE= lanes-edit.sh commit "..."` with nothing staged), which sends the commit only where origin lacks it by content and says so where it does not. - 6: the publish act first; the by-hand line only where the publish act finds nothing to send, which is the `git add`/`commit` failure. - Any other refusal: as before, "NOT written" and the line by hand. - Each remedy is printed on a line of its own. The exit is still 1. - Tests (`tests/test_a23_sweep_checkouts.py`), both red at `dfec3c5` ("was NOT written") and green here: - rv4's P2: the real `log` runs and its wrapper exits 3; origin holds the line once, the sweep hands out the publish act alone, and running it leaves origin holding the line once; - every push is refused with origin unmoved, so `log` exits 6 with its commit kept on HEAD; the sweep hands out the publish act and then the by-hand line; with pushes allowed again the publish act sends the kept commit, and origin holds the line once. - The refusing-log case now uses the shared `_wrapped_log` and `_by_hand` helpers; it asserts what it did. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit a4cbf1a0d08d16cdee213c0b4a5b05477d5222ca) --- lane-end | 27 +++++++- tests/test_a23_sweep_checkouts.py | 104 +++++++++++++++++++++++++----- 2 files changed, 115 insertions(+), 16 deletions(-) diff --git a/lane-end b/lane-end index bee8f0f..98a8970 100755 --- a/lane-end +++ b/lane-end @@ -1775,8 +1775,33 @@ SLU ( unset LANES_NO_FETCH LANES_LANE="$sc_runner" LANES_SESSION="$sc_uuid" "$LANES_EDIT" log NOTED "lane:$sc_runner" "$sc_text" ) >&2 || sc_noted_rc=$? + # A WRITE THAT DID NOT CONFIRM IS NOT A WRITE THAT DID NOT HAPPEN (rv4's + # SHOULD 2). `log`'s 3 is `pushed, not confirmed`: the line may be on + # origin already. Its 6 is, in most of its causes, the commit KEPT in the + # checkout the write went through and not on origin, and in the rest a + # `git add` or `commit` that failed, with nothing committed. Writing the + # line again after either would record the sweep twice, or meet S3's + # refusal of a checkout holding a commit origin lacks. So both are handed + # THE PUBLISH ACT first (Amendment 23 S4, #194): `commit` with nothing + # staged sends what origin lacks by content and says so where it lacks + # nothing, so it is safe to run whichever happened. Only a 6 whose publish + # act finds nothing to send, and every other refusal, gets the line by hand. if [ "$sc_noted_rc" != 0 ]; then - die "the NOTED line recording this sweep was NOT written (\`lanes-edit.sh log\` exited $sc_noted_rc, above), so lane $sc_runner's log does not say what was removed. Nothing was lost: each checkout removed held nothing unpublished. Write the line by hand: LANES_LANE=$(printf '%q' "$sc_runner") LANES_SESSION=$sc_uuid lanes-edit.sh log NOTED $(printf '%q' "lane:$sc_runner") $(printf '%q' "$sc_text")" 1 + sc_by_hand="LANES_LANE=$(printf '%q' "$sc_runner") LANES_SESSION=$sc_uuid lanes-edit.sh log NOTED $(printf '%q' "lane:$sc_runner") $(printf '%q' "$sc_text")" + sc_publish="LANES_LANE=$(printf '%q' "$sc_runner") lanes-edit.sh commit $(printf '%q' "the NOTED line of lane-end --sweep-checkouts")" + case "$sc_noted_rc" in + 3) + note "the NOTED line recording this sweep was committed and pushed, and the read-back could not confirm it on origin (\`lanes-edit.sh log\` exited 3, above), so it may be on origin already, and it is not to be written again. Nothing was lost: each checkout removed held nothing unpublished. The publish act sends its commit only where origin lacks it by content, and says so where origin has it:" + note " $sc_publish" + exit 1 ;; + 6) + note "the NOTED line recording this sweep was not published (\`lanes-edit.sh log\` exited 6, above). Nothing was lost: each checkout removed held nothing unpublished. Where that says its commit is kept, the line is committed and only origin lacks it, so it is not to be written again: the publish act sends what origin lacks by content:" + note " $sc_publish" + note "Only where the publish act finds nothing to send (git could not add or commit the line, above) is it written by hand:" + note " $sc_by_hand" + exit 1 ;; + esac + die "the NOTED line recording this sweep was NOT written (\`lanes-edit.sh log\` exited $sc_noted_rc, above), so lane $sc_runner's log does not say what was removed. Nothing was lost: each checkout removed held nothing unpublished. Write the line by hand: $sc_by_hand" 1 fi printf ' NOTED in lane %s'"'"'s log: %s\n' "$sc_runner" "$sc_text" fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 9b4328a..d91be25 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -887,13 +887,34 @@ def test_nothing_removed_writes_nothing(sweep): assert "NOTED in lane" not in result.stdout -REFUSING_LOG = """#!/usr/bin/env bash -if [ "${1-}" = log ]; then - printf 'FAKE: the register refused this line\\n' >&2 - exit 2 -fi -exec bash %s "$@" -""" +def _wrapped_log(box: Sweep, name: str, log_body: str) -> str: + """A `LANES_EDIT` that runs `log_body` for `log` and the real + `lanes-edit.sh`, which `$REAL` names, for every other verb.""" + wrapper = box.fakebin / name + wrapper.write_text(f'#!/usr/bin/env bash\nREAL={REPO / "lanes-edit.sh"}\n' + f'if [ "${{1-}}" = log ]; then\n{log_body}fi\nexec bash "$REAL" "$@"\n', + encoding="utf-8") + wrapper.chmod(0o755) + return str(wrapper) + + +def _noted_on_origin(box: Sweep) -> list: + return [line for line in box.origin_file(f"lanes/log/{RUNNER}.md").splitlines() + if line.startswith("NOTED")] + + +def _remedies(stderr: str) -> list: + """The commands a failed NOTED write hands out on lines of their own.""" + return [line.split(":", 1)[1].strip() for line in stderr.splitlines() + if line.startswith("lane-end: LANES_LANE=")] + + +def _by_hand(box: Sweep, command: str) -> subprocess.CompletedProcess: + """Runs a command the sweep printed, as a person would, with this + worktree's `lanes-edit.sh` first on PATH.""" + return subprocess.run(["bash", "-c", command], cwd=str(box.root), capture_output=True, + text=True, env=dict(box.env, PATH=f"{REPO}{os.pathsep}{box.env['PATH']}"), + timeout=180, check=False) def test_noted_line_not_written_exits_1_with_the_command(sweep): @@ -901,24 +922,77 @@ def test_noted_line_not_written_exits_1_with_the_command(sweep): success: exit 1, and the exact command that writes it by hand, which then writes it.""" box = sweep.box - wrapper = box.fakebin / "lanes-edit-refusing-log" - wrapper.write_text(REFUSING_LOG % (REPO / "lanes-edit.sh"), encoding="utf-8") - wrapper.chmod(0o755) - result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=str(wrapper)) + refusing = _wrapped_log(box, "lanes-edit-refusing-log", ( + " printf 'FAKE: the register refused this line\\n' >&2\n" + " exit 2\n")) + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=refusing) assert result.returncode == 1, (result.stdout, result.stderr) assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows assert "the NOTED line recording this sweep was NOT written" in result.stderr, result.stderr assert "Nothing was lost" in result.stderr by_hand = result.stderr.split("Write the line by hand: ", 1)[1].strip() assert by_hand.startswith(f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), by_hand - assert "NOTED" not in box.origin_file(f"lanes/log/{RUNNER}.md") - done = subprocess.run(["bash", "-c", by_hand], cwd=str(box.root), capture_output=True, text=True, - env=dict(box.env, PATH=f"{REPO}{os.pathsep}{box.env['PATH']}"), - timeout=180, check=False) + assert not _noted_on_origin(box) + done = _by_hand(box, by_hand) assert done.returncode == 0, (done.stdout, done.stderr) assert "sweep-checkouts (Amendment 23 S6.2): removed 2" in box.origin_file(f"lanes/log/{RUNNER}.md") +def test_noted_line_unconfirmed_is_never_written_again(sweep): + """A NOTED write that exits 3, `pushed, not confirmed`, MAY BE ON ORIGIN + (rv4's SHOULD 2, P2). Here the line did land and the read-back is what + failed. The sweep exits 1 without saying "NOT written", and hands out + one command, the publish act, which sends the commit only where origin + lacks it by content: run, it finds nothing to send, and origin still + holds the line once.""" + box = sweep.box + exit3 = _wrapped_log(box, "lanes-edit-unconfirmed-log", ( + ' bash "$REAL" "$@"; rc=$?\n' + " if [ \"$rc\" = 0 ]; then printf 'FAKE: pushed, not confirmed\\n' >&2; exit 3; fi\n" + ' exit "$rc"\n')) + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=exit3) + assert result.returncode == 1, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert len(_noted_on_origin(box)) == 1 + assert "was NOT written" not in result.stderr, result.stderr + assert "Write the line by hand" not in result.stderr, result.stderr + assert "may be on origin already" in result.stderr, result.stderr + remedies = _remedies(result.stderr) + assert len(remedies) == 1, remedies + assert remedies[0].startswith(f"LANES_LANE={RUNNER} lanes-edit.sh commit "), remedies + done = _by_hand(box, remedies[0]) + assert done.returncode == 0, (done.stdout, done.stderr) + assert len(_noted_on_origin(box)) == 1, "the publish act wrote no second line" + + +def test_noted_line_kept_unpublished_is_published_not_written_again(sweep): + """A NOTED write that exits 6 with its commit KEPT in the checkout it was + made in (rv4's SHOULD 2): every push is refused here with origin unmoved. + The sweep exits 1 without saying "NOT written", and hands out the publish + act first, and the by-hand line only for a publish act that finds nothing + to send. With pushes allowed again, the publish act sends the kept commit, + and origin holds the line once.""" + box = sweep.box + gate = box.root / "push.refused" + gate.write_text("", encoding="utf-8") + box.use_net(hook=f'case "$1" in push) if [ -e {gate} ]; then exit 1; fi ;; esac\nexit 0\n') + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 1, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert not _noted_on_origin(box) + assert "was NOT written" not in result.stderr, result.stderr + assert "kept on HEAD" in result.stderr, result.stderr + remedies = _remedies(result.stderr) + assert len(remedies) == 2, remedies + publish, by_hand = remedies + assert publish.startswith(f"LANES_LANE={RUNNER} lanes-edit.sh commit "), remedies + assert by_hand.startswith(f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), remedies + gate.rename(box.root / "push.allowed") + done = _by_hand(box, publish) + assert done.returncode == 0, (done.stdout, done.stderr) + assert len(_noted_on_origin(box)) == 1, _noted_on_origin(box) + + def test_yes_without_a_session_uuid_is_refused(sweep): """The NOTED line's session field is a transcript uuid and only that (Amendment 7(b)): a `LANES_SESSION` that is none is refused before From d1dc971f4045836fd7aac7ec3054de0334e17bd9 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:39:20 +0000 Subject: [PATCH 09/53] Amendment 23 S6.2 tests: the cwd scan and the former-name re-ask under each checkout's lock, and the exit for a removal that did not finish, each get a case, so none of rv4's four one-line mutants survives (rv4's SHOULD 1, #198, slice 4) rv4 found four one-line mutants of `lane-end` that no case killed. Each is a part of D14 (the decision made again under the lock) or an exit: - R1, the cwd scan not repeated under the lock: a process starts working in the CLOSED lane's checkout at that checkout's own fetch (rv4's P4), and the checkout is held, naming its pid. Under R1 it is removed. - R2, the listing's refusal on a W fetch that did not complete: killed since `dfec3c5` by both cases of the parametrized listing test (the fetch fails after `ls-remote`, and `ls-remote` fails). - R3, the former-name re-ask never holding: the listing finds repoS4-8's checkout beside its new name's, and at its own fetch the new name's checkout goes. The former name's checkout is then that lane's own again (S1), and is held as "no longer a former name". Under R3 it is removed. - R4, no exit 1 for a removal that did not finish: the CLOSED lane's checkout holds an ignored, read-only directory, so its aside cannot all be removed. It is named PARTLY, the run exits 1 naming the aside, and the aside stays. Under R4 the run exits 0. Skipped as root. Each was run once against its case in a separate clone (`c7-mut.py`): all three KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 9a7e8c88b544a0e3a0cdab17fc29f8444524de13) --- tests/test_a23_sweep_checkouts.py | 74 +++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index d91be25..0c46d75 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -24,6 +24,7 @@ import os import shutil +import signal import subprocess from pathlib import Path @@ -521,6 +522,32 @@ def test_a_live_cwd_holds(sweep): sleeper.wait() +def test_a_cwd_that_appears_after_the_listing_holds(sweep): + """S6.2 (c) IS READ AGAIN UNDER THE LOCK, AFTER THE FETCH (rv4's SHOULD 1, + R1, P4): a process that starts working in the checkout after the listing, + here at the checkout's own fetch, holds it. The process is a `sleep`, + killed by its PID afterwards.""" + box = sweep.box + c = box.checkout(CLOSED) + pidfile = box.root / "late.pid" + _hook_on_fetch_of(box, c, ( + f'( cd {c}/handoffs && exec sleep 600 ) /dev/null 2>&1 & ' + f'printf "%s" "$!" > {pidfile}')) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + late = pidfile.read_text(encoding="utf-8") + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert f"a live process works in it (pid {late})" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + finally: + if pidfile.exists(): + try: + os.kill(int(pidfile.read_text(encoding="utf-8")), signal.SIGKILL) + except (ProcessLookupError, ValueError): + pass + + def _hook_on_fetch_of(box: Sweep, target: Path, body: str) -> None: """A `LANES_GIT` hook that runs `body` when the sweep fetches `target`.""" box.use_net(hook=( @@ -576,6 +603,24 @@ def test_a_lane_resumed_after_the_listing_is_kept(sweep): assert c.is_dir() +def test_a_former_name_whose_new_checkout_goes_after_the_listing_is_kept(sweep): + """A FORMER NAME IS ASKED AGAIN UNDER THE LOCK (rv4's SHOULD 1, R3). The + listing finds repoS4-8's checkout beside its new name's; at its own fetch + the new name's checkout goes, so the former name's checkout IS that + lane's own again until its next write renames it (S1), and is kept.""" + box = sweep.box + old, new = box.checkout(OLD), box.checkout(NEW) + _, rows = sweep() + assert rows[str(old)][0] == "remove", rows[str(old)] + _hook_on_fetch_of(box, old, f"mv -- {new} {box.root / 'aside-new'}") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(old)][0] == "hold", rows[str(old)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (f"it is no longer a former name of lane {NEW} beside that lane's own checkout" + in rows[str(old)][1]), rows[str(old)] + assert old.is_dir() + + def test_a_fetch_that_fails_holds(sweep): """`--yes` decides on a fetched origin: a candidate whose fetch fails is held, and its lock released.""" @@ -835,6 +880,35 @@ def test_held_parent_keeps_its_dependents_chain(sweep, held, expect): assert os.path.lexists(chain[i]) == (said != "removed"), chain[i] +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root removes what a read-only directory holds") +def test_a_removal_that_does_not_finish_exits_1(sweep): + """A REMOVAL THAT DOES NOT FINISH IS SAID, BY PATH, AND EXITS 1 (rv4's + SHOULD 1, R4). The CLOSED lane's checkout holds an IGNORED directory + (ignored files hold nothing) that is read-only, so the checkout is moved + aside and the removal of the aside stops partway: it is named PARTLY, + the run exits 1, and the aside stays, hidden, under `-lanes/`.""" + box = sweep.box + c = box.checkout(CLOSED) + (c / ".git" / "info" / "exclude").write_text("scratch/\n", encoding="utf-8") + box.write(c, "scratch/kept.txt", "an ignored file in a read-only directory\n") + (c / "scratch").chmod(0o555) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + asides = list(box.lanes_root.glob(f".{CLOSED}.removing-*")) + assert rows[str(c)][0] == "PARTLY", rows[str(c)] + assert result.returncode == 1, (result.stdout, result.stderr) + assert len(asides) == 1, asides + assert f"the removal of {asides[0]} did not finish" in result.stderr, result.stderr + assert not os.path.lexists(c) + assert (asides[0] / "scratch" / "kept.txt").is_file() + finally: + for aside in box.lanes_root.glob(f".{CLOSED}.removing-*"): + (aside / "scratch").chmod(0o755) + if (c / "scratch").is_dir(): + (c / "scratch").chmod(0o755) + + # ----------------------------------------------------------- the NOTED line def test_noted_line_written_through_runner_checkout(sweep): From f07aa026c28c6dba3a01bad118578f5335ba08ac Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:40:11 +0000 Subject: [PATCH 10/53] Amendment 23 S6.2 tests: the ten composite asserts are split into one assert per condition, so a failure names the condition that failed (rv4's SHOULD 3, Sonar S9073, #198, slice 4) The defect slice 1b split out in `3c6b240`. `assert a and b` reports only that the whole was false; each condition is now its own assert, with the message the composite carried where it had one. The refusal cases (nothing under `-lanes/` changed, origin unmoved), the beside-W listing, every hold, the fetch that fails, the NOTED line's fetch-before-push order, nothing removed writing nothing, the store kept inside a clone, and the directory sharing W's git directory. No case changes what it asserts. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 1345897d7bb9dcc64475ebc8616e3390607fb7cc) --- tests/test_a23_sweep_checkouts.py | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 0c46d75..fe92fe6 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -224,7 +224,8 @@ def test_yes_without_LANES_LANE_is_refused(sweep): assert result.returncode == 2, (result.stdout, result.stderr) assert "LANES_LANE= lane-end --sweep-checkouts --yes" in result.stderr assert "Nothing was removed" in result.stderr, result.stderr - assert box.everything() == before and box.origin_head() == head + assert box.everything() == before + assert box.origin_head() == head def test_yes_under_LANES_NO_FETCH_is_refused(sweep): @@ -236,7 +237,8 @@ def test_yes_under_LANES_NO_FETCH_is_refused(sweep): result, _ = sweep("--yes", LANES_LANE=RUNNER, LANES_NO_FETCH="1") assert result.returncode == 2, (result.stdout, result.stderr) assert "LANES_NO_FETCH=1 is set" in result.stderr, result.stderr - assert box.everything() == before and box.origin_head() == head + assert box.everything() == before + assert box.origin_head() == head @pytest.mark.parametrize("extra", [ @@ -305,7 +307,8 @@ def test_clones_beside_W_are_listed_and_never_candidates(sweep): assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones}, rows assert str(box.w.parent / "otherrepo") not in rows, rows judged = _tagged(rows, "remove") | _tagged(rows, "hold") - assert judged and all(Path(p).parent == box.lanes_root for p in judged), rows + assert judged, rows + assert all(Path(p).parent == box.lanes_root for p in judged), rows assert "listed and never removed by this build" in result.stdout @@ -481,7 +484,8 @@ def test_holds(sweep, make, said): assert result.returncode == 0, (result.stdout, result.stderr) assert rows[str(c)][0] == "hold", rows[str(c)] assert said in rows[str(c)][1], rows[str(c)] - assert c.is_dir() and box.everything()[str(c)] == before, "the held checkout is as it was" + assert c.is_dir() + assert box.everything()[str(c)] == before, "the held checkout is as it was" if make is not _lock: assert not (c / "lanes" / ".lanes-edit.lock").exists(), "the sweep's own lock is released" _no_aside(box) @@ -631,7 +635,8 @@ def test_a_fetch_that_fails_holds(sweep): assert result.returncode == 0, (result.stdout, result.stderr) assert rows[str(c)][0] == "hold", rows[str(c)] assert "its fetch of origin main failed" in rows[str(c)][1], rows[str(c)] - assert c.is_dir() and not (c / "lanes" / ".lanes-edit.lock").exists() + assert c.is_dir() + assert not (c / "lanes" / ".lanes-edit.lock").exists(), "its lock is released" @pytest.mark.parametrize("op,note", [ @@ -940,7 +945,9 @@ def test_noted_line_written_through_runner_checkout(sweep): mine = [i for i, line in enumerate(lines) if f"-C {runner} " in line] fetches = [i for i in mine if lines[i].startswith("fetch ")] pushes = [i for i in mine if lines[i].startswith("push ")] - assert fetches and pushes and fetches[0] < pushes[0], lines + assert fetches, lines + assert pushes, lines + assert fetches[0] < pushes[0], lines assert "NOTED in lane" in result.stdout, result.stdout @@ -956,7 +963,8 @@ def test_nothing_removed_writes_nothing(sweep): result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 0, (result.stdout, result.stderr) assert not _tagged(rows, "removed"), rows - assert box.origin_head() == head and "NOTED" not in box.origin_file(f"lanes/log/{RUNNER}.md") + assert box.origin_head() == head + assert "NOTED" not in box.origin_file(f"lanes/log/{RUNNER}.md") assert box.git("-C", str(box.checkout(RUNNER)), "rev-parse", "HEAD").strip() == runner_head assert "NOTED in lane" not in result.stdout @@ -1077,7 +1085,8 @@ def test_yes_without_a_session_uuid_is_refused(sweep): assert result.returncode == 2, (result.stdout, result.stderr) assert "has no transcript uuid for the session field" in result.stderr, result.stderr assert "Nothing was removed" in result.stderr - assert box.everything() == before and box.origin_head() == head + assert box.everything() == before + assert box.origin_head() == head def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): @@ -1105,7 +1114,8 @@ def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): assert result.returncode in (0, 3), (result.stdout, result.stderr) assert rows[str(c)][0] == "hold", rows[str(c)] assert f"leaned on by {borrower} " in rows[str(c)][1], rows[str(c)] - assert c.is_dir() and store.is_dir() + assert c.is_dir() + assert store.is_dir() box.git("-C", str(borrower), "fsck", "--connectivity-only") @@ -1125,4 +1135,5 @@ def test_a_directory_sharing_Ws_git_directory_is_kept_as_W(sweep): result, rows = sweep(*args, **env) assert result.returncode in (0, 3), (result.stdout, result.stderr) assert rows[str(d)] == ("keep", "W itself, the readers' copy"), rows[str(d)] - assert d.is_dir() and box.w_state() == w_before + assert d.is_dir() + assert box.w_state() == w_before From 5b285a2908707edcd0ed3176ed6b28a2c757b9e5 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:48:50 +0000 Subject: [PATCH 11/53] Amendment 23 S6.2: a former name's checkout is kept while its lane is LIVE, at the listing and in the re-ask under its lock, since a window started before the rename may still write through it (rv4's SHOULD 4, D27, #198, slice 4) Since 10-19 the launcher exports `LANES_REPO=-lanes/`, and a window of a lane started before its rename still names the old path. The sweep took a former name's checkout beside its new one as a candidate whatever the lane's state, and only its own caller's `LANES_REPO` was kept, so a LIVE window's writes failed after the sweep until it was relaunched. Nothing was lost (the holds pass), but the window broke. - `sc_judge`: a former name's checkout beside its new one is kept while that lane is LIVE ("a former name of lane X, which is LIVE: a window of it started before the rename may still write through this path, which its LANES_REPO names"). - `sc_still`: the former-name re-ask holds it when the lane reads LIVE now. - D27: LIVE, and not `/proc//environ`. On this workstation a window of the lane is a live session record naming it, which is what LIVE is, and the listing and the sweep already share that read. `environ` is the environment at exec only, cannot be read for another user's processes, and does not exist on macOS. A renamed lane's old checkout now stays while the lane is LIVE; the next sweep after that removes it. - Tests, both red at `1345897` (the former name's checkout is `remove`, and removed) and green here: repoS4-9 LIVE (a live session record names its transcript) keeps repoS4-8's checkout in the dry run and under `--yes`; repoS4-9 coming LIVE at repoS4-8's own fetch holds it as "LIVE now". Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 65594885b445802224064c5e2dcba54bca4425a9) --- lane-end | 18 ++++++++++-- tests/test_a23_sweep_checkouts.py | 47 +++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/lane-end b/lane-end index 98a8970..8edcf23 100755 --- a/lane-end +++ b/lane-end @@ -1223,11 +1223,20 @@ SC_IDX return 0 fi # A FORMER NAME OF LANE `sj_c`. Beside the new name's own checkout it is a - # candidate whatever that lane's state, because nothing writes through it - # any more (S1); with no checkout under the new name it IS the lane's - # checkout until the next write renames it (S1), and is judged as one. + # candidate, because no write of the lane's goes through it any more (S1), + # UNLESS THE LANE IS LIVE (rv4's SHOULD 4, D27): since 10-19 the launcher + # exports `LANES_REPO=-lanes/`, and a window of the lane started + # before the rename still names the old path, so removing it would fail + # that window's writes until it is relaunched. A window of the lane on + # this workstation is a live session record naming it, which is what + # LIVE is. With no checkout under the new name it IS the lane's checkout + # until the next write renames it (S1), and is judged as one. sj_new="$sc_home/$sj_c" if [ -d "$sj_new" ] && [ ! -L "$sj_new" ] && [ ! "$sj_new" -ef "$sj_p" ]; then + if sc_lookup "$sj_c" && [ "$SC_L_STATE" = LIVE ]; then + sc_keep "$sj_p" "a former name of lane $SC_L_LANE, which is LIVE: a window of it started before the rename may still write through this path, which its LANES_REPO names" + return 0 + fi sc_cand "$sj_p" "$sj_c" "a former name of lane $sj_c, beside that lane's own checkout $sj_new" "$sj_new" return 0 fi @@ -1584,6 +1593,9 @@ SS_ROW if [ "$(sc_lc "$ss_c")" != "$(sc_lc "$2")" ] || [ ! -d "$3" ] || [ -L "$3" ] || [ "$3" -ef "$1" ]; then SC_WHY="it is no longer a former name of lane $2 beside that lane's own checkout"; return 1 fi + if [ "${ss_state:-}" = LIVE ]; then + SC_WHY="lane $2 is LIVE now, and a window of it started before the rename may still write through this path (D27)"; return 1 + fi return 0 fi if [ "${ss_state:-}" = LIVE ]; then SC_WHY="lane $2 is LIVE now"; return 1; fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index fe92fe6..055414e 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -625,6 +625,53 @@ def test_a_former_name_whose_new_checkout_goes_after_the_listing_is_kept(sweep): assert old.is_dir() +def _live_record(box: Sweep, lane: str) -> tuple: + """A session record here naming `lane`'s transcript, held by the box's own + live `sleep`, which makes `lane` LIVE: its path and its text.""" + return (box.home / ".claude" / "sessions" / f"{lane}.json", + '{"pid":%d,"sessionId":"%s","kind":"interactive","status":"busy"}' + % (box.sleeper.pid, UUID[lane])) + + +def _live_session_of(box: Sweep, lane: str) -> str: + """A hook's shell command that writes `_live_record`.""" + record, body = _live_record(box, lane) + return f"printf '%s\\n' '{body}' > {record}" + + +def test_a_former_name_of_a_live_lane_is_kept(sweep): + """A FORMER NAME'S CHECKOUT IS KEPT WHILE ITS LANE IS LIVE (rv4's SHOULD 4, + D27). Since 10-19 the launcher exports `LANES_REPO=-lanes/`, and + a window of the lane started before the rename still names the old path, + so removing it makes that window's writes fail until it is relaunched. + Here repoS4-9 is LIVE: a live session record names its transcript. Its + former name's checkout is kept by the dry run and by `--yes`.""" + box = sweep.box + old = box.checkout(OLD) + record, body = _live_record(box, NEW) + record.write_text(body + "\n", encoding="utf-8") + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert rows[str(old)][0] == "keep", rows[str(old)] + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert f"a former name of lane {NEW}, which is LIVE" in rows[str(old)][1], rows[str(old)] + assert old.is_dir() + + +def test_a_former_name_whose_lane_goes_live_after_the_listing_is_kept(sweep): + """THE SAME RULE, ASKED AGAIN UNDER THE LOCK (D27): repoS4-9 comes LIVE at + its former name's own fetch, after the listing found that checkout a + candidate. It is kept.""" + box = sweep.box + old = box.checkout(OLD) + _hook_on_fetch_of(box, old, _live_session_of(box, NEW)) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(old)][0] == "hold", rows[str(old)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert f"lane {NEW} is LIVE now" in rows[str(old)][1], rows[str(old)] + assert old.is_dir() + + def test_a_fetch_that_fails_holds(sweep): """`--yes` decides on a fetched origin: a candidate whose fetch fails is held, and its lock released.""" From f233cc665a72d2c4a446ee08c116fdd140c9ce55 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:51:51 +0000 Subject: [PATCH 12/53] Amendment 23 S6.2: a `.git` file naming a clone's git directory holds it, the act names ignored files and counts a partial removal apart, and under each lock the sweep keeps its lock fresh, leaves one taken over, and asks again what made the checkout a candidate (rv4's NITs 1-4, #198, slice 4) - NIT 1: `sc_dep_scan` reads the `.git` FILES in the directories it reads alternates files in (W, the clones beside it, `-lanes/`), and a clone whose git directory, or one inside it, such a file names is held: "its git directory is named by the .git file of X, which works in it" (rv4's P3). `sc_dependents` takes the list to read as an argument. - NIT 2: the act's `removed` and `PARTLY` lines name the ignored files, as the dry run's `remove` line does, so an ignored nested repository that goes with a checkout is on the record. - NIT 3: a removal that did not finish is counted apart: "N removed, M moved aside and not all removed, K held", and the NOTED line's "removed N" counts only finished removals. A run whose only act is a partial removal still writes its NOTED line. - NIT 4, under each checkout's lock: - the lock's age is refreshed (`touch -c`, which never makes a lock) before the fetch and before the re-ask, as `git_net`'s heartbeat does; - `sc_is_register_clone` is asked again after the re-ask, so a checkout that stopped being a clone of the register's origin is held; - the lock's pid line is read again just before the move, and a line naming another holder (a takeover) holds the checkout and leaves that lock to its holder; `sc_lock_drop` and `cleanup` leave such a lock, as `mutex_release` does. - `LANES_GIT_TIMEOUT=0` still means no bound, as it does for `lanes-edit.sh`'s own network calls (D16): left on purpose. - EXIT 1's text names a removal that did not finish and a NOTED line not confirmed. - Tests, each red at `6559488` and green here: a `.git` file beside W naming the CLOSED lane's `.git` (held, dry run and `--yes`); an ignored file named on the `removed` line; the partial-removal case now also pins "1 removed, 1 moved aside and not all removed" and the NOTED line's counts; a checkout whose origin is re-pointed at its own fetch (held); a lock whose pid line another writer replaced at the fetch (held, and that pid line left as it is). Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 10468ac67729a573ec0712eb7ca0cd6628b6abfb) --- lane-end | 97 +++++++++++++++++++++++++------ tests/test_a23_sweep_checkouts.py | 75 ++++++++++++++++++++++++ 2 files changed, 153 insertions(+), 19 deletions(-) diff --git a/lane-end b/lane-end index 8edcf23..1a74d24 100755 --- a/lane-end +++ b/lane-end @@ -263,8 +263,9 @@ # failed (which is NOT the same as the lane having no object log), or the # close-out gate could not be read (no `lane-worktrees`, or an exit that # is none of its three). `--sweep-checkouts`: no W, a read that failed -# (the listing of lanes, or a fetch `--yes` decides on), or a removal -# whose NOTED line could not be written +# (the listing of lanes, or a fetch `--yes` decides on), a removal +# that did not finish, or removals whose NOTED line was not written or +# not confirmed on origin (the commands to finish it are printed) # 2 refusal — bad arguments, no row (or two), something still in flight, or # the close-out gate: the lane's trees, branches, scratch, caches or # residue still on disk, or the sweep refusing the lane. @@ -598,7 +599,15 @@ LE_TMP="" SC_LOCK="" cleanup() { [ -n "${LE_TMP:-}" ] && rm -f -- "$LE_TMP" - if [ -n "${SC_LOCK:-}" ]; then rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || :; fi + # A LOCK WHOSE PID LINE NAMES ANOTHER HOLDER IS LEFT AS IT IS, as + # `lanes-edit.sh`'s `mutex_release` leaves it (rv4's NIT 4): it was taken + # over while this run held it, and it is that writer's now. + if [ -n "${SC_LOCK:-}" ]; then + cl_l=""; IFS= read -r cl_l < "$SC_LOCK/pid" 2>/dev/null || : + if [ -z "$cl_l" ] || [ "$cl_l" = "$$ ${sc_ident:-}" ]; then + rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || : + fi + fi return 0 } # A SIGNAL ENDS THE RUN, AND `cleanup` RUNS ON THE WAY OUT. `cleanup` returns @@ -1401,11 +1410,28 @@ SH_REFS # every entry of `-lanes/`, hidden ones included. Read afresh before each # removal, so a dependent kept earlier in the run still holds what it # borrows from. - sc_dep_lines="" + # + # AND A `.git` FILE IS READ THE SAME WAY (rv4's NIT 1): a directory whose + # `.git` file names a clone's git directory, or one inside it, works in it, + # and removing the clone takes that away. `sc_gitfile_lines` holds ` TAB `. A linked + # worktree is one, and is also held through `.git/worktrees/`. + sc_dep_lines=""; sc_gitfile_lines="" sc_dep_scan() { - sc_dep_lines="" + sc_dep_lines=""; sc_gitfile_lines="" for sds_e in "$sc_w" "$sc_root"/* "$sc_home"/* "$sc_home"/.[!.]* "$sc_home"/..?*; do [ -d "$sds_e" ] || continue + if [ -f "$sds_e/.git" ] && [ ! -L "$sds_e/.git" ]; then + sds_g="$(sed -n -e '1s/^gitdir: //p' -- "$sds_e/.git" 2>/dev/null)" || sds_g="" + case "$sds_g" in '') : ;; /*) : ;; *) sds_g="$sds_e/$sds_g" ;; esac + if [ -n "$sds_g" ]; then + sds_r="$(sc_real "$sds_g")" + if [ -n "$sds_r" ]; then + sc_gitfile_lines="${sc_gitfile_lines}${sds_e}${sc_tab}${sds_r} +" + fi + fi + fi if [ -f "$sds_e/.git/objects/info/alternates" ]; then sds_a="$sds_e/.git/objects/info/alternates"; sds_o="$sds_e/.git/objects" elif [ -f "$sds_e/objects/info/alternates" ]; then @@ -1427,15 +1453,15 @@ SH_REFS # A REPOSITORY LEANS ON A CLONE when its alternates file names an object # directory AT OR UNDER that clone — its own `.git/objects`, or a store kept # anywhere inside it — because removing the clone removes that store too. - sc_dependents() { # [] — what leans on it - scd_out="" + sc_dependents() { # [] [] — what leans on it + scd_out=""; scd_lines="${3-$sc_dep_lines}" while IFS="$sc_tab" read -r scd_d scd_t; do [ -n "$scd_d" ] && [ "$scd_d" != "$1" ] || continue case "$scd_t" in "$1" | "$1"/*) : ;; *) continue ;; esac case " ${2-} " in *" $scd_d "*) continue ;; esac scd_out="${scd_out:+$scd_out }$scd_d" done < "$slt_d/pid" 2>/dev/null || : return 0 } + # THE LOCK STAYS THE SWEEP'S, OR IT IS LEFT AS IT IS (rv4's NIT 4). A holder + # in another container is judged by the lock's age alone, so the sweep + # refreshes that age before each slow step, as `git_net`'s heartbeat does + # (`touch -c`, which never makes a lock). Its pid line is read again just + # before the move, and a lock whose line names another holder was taken + # over: the checkout is held, and that lock is left to its holder, as + # `mutex_release` leaves one. + sc_lock_beat() { + if [ -n "$SC_LOCK" ]; then touch -c -- "$SC_LOCK" 2>/dev/null || :; fi + } + sc_lock_holder() { # the lock's pid line, or nothing where it cannot be read + slh_l=""; IFS= read -r slh_l < "$SC_LOCK/pid" 2>/dev/null || : + printf '%s' "$slh_l" + } sc_lock_drop() { - if [ -n "$SC_LOCK" ]; then rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || :; fi + if [ -n "$SC_LOCK" ]; then + sld_l="$(sc_lock_holder)" + if [ -z "$sld_l" ] || [ "$sld_l" = "$$ $sc_ident" ]; then + rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || : + fi + fi SC_LOCK="" } # `--yes` FETCHES EACH CHECKOUT FIRST, bounded as `lanes-edit.sh`'s network @@ -1671,7 +1718,7 @@ SLU done printf '\n under %s/:\n' "$sc_home" - sc_gone=""; sc_n_removed=0; sc_n_held=0; sc_n_removable=0; sc_partial="" + sc_gone=""; sc_n_removed=0; sc_n_partial=0; sc_n_held=0; sc_n_removable=0; sc_partial="" sc_removed_names=""; sc_held_names=""; sc_partial_names="" sc_say_held() { # printf ' hold %s %s; holds: %s\n' "$1" "$2" "$3" @@ -1703,15 +1750,25 @@ SLU if ! sc_lock_take "$sc_c"; then sc_say_held "$sc_c" "$sc_why" "$SC_WHY"; continue fi - sc_act_hold="" + sc_act_hold=""; sc_act_ign="" + sc_lock_beat if ! sc_fetch "$sc_c"; then sc_act_hold="its fetch of origin $sc_branch failed or timed out, and nothing is decided on an old origin/$sc_branch" - elif ! sc_still "$sc_c" "$sc_l" "$sc_new"; then + elif ! { sc_lock_beat; sc_still "$sc_c" "$sc_l" "$sc_new"; }; then sc_act_hold="$SC_WHY" + elif ! sc_is_register_clone "$sc_c"; then + sc_act_hold="it is no longer what made it a candidate: $SC_WHY" else sc_dep_scan; sc_cwd_scan sc_judge_holds "$sc_c" 1 - sc_act_hold="$SC_HOLD" + sc_act_hold="$SC_HOLD"; sc_act_ign="$SC_IGN" + fi + if [ -z "$sc_act_hold" ]; then + sc_holder="$(sc_lock_holder)" + if [ "$sc_holder" != "$$ $sc_ident" ]; then + sc_act_hold="its writers' lock lanes/.lanes-edit.lock is no longer this sweep's (${sc_holder:-its pid line cannot be read}): a writer took it over, and it is left to that writer" + SC_LOCK="" + fi fi if [ -n "$sc_act_hold" ]; then sc_lock_drop @@ -1725,14 +1782,16 @@ SLU SC_LOCK="" # it went with the checkout if sc_under_home "$sc_aside"; then rm -rf -- "$sc_aside" 2>/dev/null || :; fi if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then - printf ' PARTLY %s %s; moved aside to %s, which could not all be removed\n' "$sc_c" "$sc_why" "$sc_aside" + printf ' PARTLY %s %s; moved aside to %s, which could not all be removed%s\n' "$sc_c" "$sc_why" "$sc_aside" "${sc_act_ign:+; $sc_act_ign}" sc_partial="${sc_partial:+$sc_partial }$sc_aside" sc_partial_names="${sc_partial_names:+$sc_partial_names }${sc_aside##*/}" + sc_n_partial=$((sc_n_partial + 1)) else - printf ' removed %s %s\n' "$sc_c" "$sc_why" + printf ' removed %s %s%s\n' "$sc_c" "$sc_why" "${sc_act_ign:+; $sc_act_ign}" sc_removed_names="${sc_removed_names:+$sc_removed_names }${sc_c##*/}" + sc_n_removed=$((sc_n_removed + 1)) fi - sc_gone="${sc_gone:+$sc_gone }$sc_c"; sc_n_removed=$((sc_n_removed + 1)) + sc_gone="${sc_gone:+$sc_gone }$sc_c" done printf '%s' "$sc_keep_lines" if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ]; then printf ' (nothing)\n'; fi @@ -1763,8 +1822,8 @@ SLU printf '\n DRY RUN — nothing was removed, and there is nothing to remove.\n' exit 0 fi - printf '\n %s removed, %s held under %s/; %s clone(s) beside W left.\n' \ - "$sc_n_removed" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" + printf '\n %s removed, %s moved aside and not all removed, %s held under %s/; %s clone(s) beside W left.\n' \ + "$sc_n_removed" "$sc_n_partial" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" # THE NOTED LINE (S6.2; S2 lists the sweep among the writers): ONE line in # the runner's own log, naming what was removed and what was held, written # by `lanes-edit.sh log` and so through the runner's own checkout under S3 @@ -1775,7 +1834,7 @@ SLU # never from the host's own root, and every name in the line is one this # sweep already holds to letters, digits, `.`, `_` and `-`, so the free # text's own rules (no ` — `, no newline, no leading arrow) hold for it. - if [ "$sc_n_removed" -gt 0 ]; then + if [ $((sc_n_removed + sc_n_partial)) -gt 0 ]; then sc_rel="${sc_home##*/}" case "$sc_rel" in '' | *[!A-Za-z0-9._-]*) sc_rel="-lanes" ;; esac sc_text="sweep-checkouts (Amendment 23 S6.2): removed $sc_n_removed register checkout(s) under $sc_rel/ that held nothing unpublished" diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 055414e..2c254ea 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -351,6 +351,19 @@ def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "every lock it took is released" +def test_ignored_files_are_named_at_the_act(sweep): + """IGNORED FILES HOLD NOTHING, AND THE ACT NAMES THEM as the dry run does + (rv4's NIT 2), so what went with a removed checkout is on its line.""" + box = sweep.box + c = box.checkout(CLOSED) + (c / ".git" / "info" / "exclude").write_text("*.log\n", encoding="utf-8") + box.write(c, "scratch.log", "ignored\n") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "removed", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert "ignored, which holds nothing: 1 (scratch.log)" in rows[str(c)][1], rows[str(c)] + + def _commit_unpushed(box: Sweep, c: Path) -> None: box.write(c, "lanes/extra.md", "a record never pushed\n") box.commit(c, "an unpushed record", "lanes/extra.md") @@ -686,6 +699,40 @@ def test_a_fetch_that_fails_holds(sweep): assert not (c / "lanes" / ".lanes-edit.lock").exists(), "its lock is released" +def test_a_checkout_whose_origin_changes_after_the_listing_is_held(sweep): + """WHAT MADE IT A REGISTER CHECKOUT IS ASKED AGAIN UNDER THE LOCK (rv4's + NIT 4): at its own fetch the CLOSED lane's checkout is pointed at another + repository (W's path here, which that fetch then reads), so it is no + longer a clone of the register's origin, and it is held.""" + box = sweep.box + c = box.checkout(CLOSED) + _hook_on_fetch_of(box, c, f"git -C {c} remote set-url origin {box.w}") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert f"a checkout of {box.w}, not of the register's origin" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + +def test_a_lock_taken_over_under_the_sweep_holds_and_is_left(sweep): + """THE LOCK IS THE SWEEP'S UNTIL THE MOVE, OR NOTHING IS MOVED (rv4's NIT + 4): at the checkout's fetch another writer's pid line replaces the + sweep's in its `lanes/.lanes-edit.lock`, as a takeover from another + container does. The checkout is held, and that writer's lock is left as + it is, as `lanes-edit.sh`'s `mutex_release` leaves a lock it no longer + holds.""" + box = sweep.box + c = box.checkout(CLOSED) + pid = c / "lanes" / ".lanes-edit.lock" / "pid" + _hook_on_fetch_of(box, c, f"printf '1 other-container otherhost\\n' > {pid}") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert "is no longer this sweep's (1 other-container otherhost)" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" + + @pytest.mark.parametrize("op,note", [ ("fetch", "fetch failed — reading the logs as they stand locally"), ("ls-remote", "NO FETCH WAS MADE"), @@ -954,6 +1001,15 @@ def test_a_removal_that_does_not_finish_exits_1(sweep): assert f"the removal of {asides[0]} did not finish" in result.stderr, result.stderr assert not os.path.lexists(c) assert (asides[0] / "scratch" / "kept.txt").is_file() + # A removal that did not finish is counted apart from the ones that + # did (rv4's NIT 3), on the summary line and in the NOTED line, and + # its ignored files are named at the act (NIT 2). + assert "ignored, which holds nothing: 1 (scratch/)" in rows[str(c)][1], rows[str(c)] + assert " 1 removed, 1 moved aside and not all removed, " in result.stdout, result.stdout + noted = _noted_on_origin(box) + assert len(noted) == 1, noted + assert (f"removed 1 register checkout(s) under brett-wip-lanes/ that held nothing " + f"unpublished: {OLD}; moved aside and not all removed: {asides[0].name};") in noted[0], noted finally: for aside in box.lanes_root.glob(f".{CLOSED}.removing-*"): (aside / "scratch").chmod(0o755) @@ -1166,6 +1222,25 @@ def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): box.git("-C", str(borrower), "fsck", "--connectivity-only") +def test_a_git_file_naming_a_clones_git_directory_holds_it(sweep): + """A `.git` FILE IS READ AS AN ALTERNATES FILE IS (rv4's NIT 1, P3): a + directory beside W whose `.git` file names the CLOSED lane's `.git` works + in that clone's git directory, and removing the clone takes it away. The + clone is held, by the dry run and by `--yes`, naming the directory.""" + box = sweep.box + c = box.checkout(CLOSED) + sharer = box.w.parent / "sharer" + sharer.mkdir() + (sharer / ".git").write_text(f"gitdir: {c}/.git\n", encoding="utf-8") + assert box.git("-C", str(sharer), "rev-parse", "--git-dir").strip() == f"{c}/.git" + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert f"its git directory is named by the .git file of {sharer}" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + def test_a_directory_sharing_Ws_git_directory_is_kept_as_W(sweep): """W IS NEVER A CANDIDATE, by its git directory as well as by its path: a directory under `-lanes/`, named for a CLOSED lane, whose own `.git` From 483a72f4688ffa0bcadfcae0186c6604cb447926 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 02:53:41 +0000 Subject: [PATCH 13/53] Amendment 23 S6.2 tests: the `--yes` refusals before anything is removed are one parametrized case, which adds D19's refusal when `LANES_SESSION` is unset and `last-session` names no session (rv4's NIT 5, #198, slice 4) Three near-identical cases (no `LANES_LANE`, `LANES_NO_FETCH=1`, a `LANES_SESSION` that is no uuid) each asserted rc 2, their sentence, and nothing under `-lanes/` or on origin changed. They are one case, `test_yes_is_refused_before_anything_is_removed`, which also asserts "Nothing was removed" for all of them. A fourth parameter is D19's other branch, untested until now: `LANES_SESSION` unset and a `last-session` that names none (a wrapper exits 8), refused naming both. With D19's uuid check disabled (`p-uuid`), both session cases fail. The wrapper helper takes the verb it stands in for (`_wrapped`). Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit a376e5b0bb67242a6212822b5e05408c57d435be) --- tests/test_a23_sweep_checkouts.py | 76 ++++++++++++++++--------------- 1 file changed, 40 insertions(+), 36 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 2c254ea..71dd97e 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -215,28 +215,46 @@ def test_dry_run_with_nothing_to_remove_exits_0(sweep): assert "there is nothing to remove" in result.stdout, result.stdout -def test_yes_without_LANES_LANE_is_refused(sweep): - """`--yes` writes a NOTED line into the log of the lane that runs it, so a - run that names no lane is refused before anything is read or removed.""" - box = sweep.box - before, head = box.everything(), box.origin_head() - result, _ = sweep("--yes") - assert result.returncode == 2, (result.stdout, result.stderr) - assert "LANES_LANE= lane-end --sweep-checkouts --yes" in result.stderr - assert "Nothing was removed" in result.stderr, result.stderr - assert box.everything() == before - assert box.origin_head() == head +def _no_last_session(box: Sweep) -> dict: + """`LANES_LANE` set and `LANES_SESSION` unset, with a `last-session` that + names no session for any lane (8, no record).""" + return {"LANES_LANE": RUNNER, + "LANES_EDIT": _wrapped(box, "lanes-edit-no-last-session", "last-session", " exit 8\n")} + + +REFUSALS = [ + pytest.param(lambda box: {}, ("LANES_LANE= lane-end --sweep-checkouts --yes",), + id="no LANES_LANE"), + pytest.param(lambda box: {"LANES_LANE": RUNNER, "LANES_NO_FETCH": "1"}, + ("LANES_NO_FETCH=1 is set",), id="LANES_NO_FETCH=1"), + pytest.param(lambda box: {"LANES_LANE": RUNNER, "LANES_SESSION": "unknown"}, + ("has no transcript uuid for the session field", + "LANES_SESSION is 'unknown', which is not a transcript uuid"), + id="a LANES_SESSION that is no uuid"), + pytest.param(_no_last_session, + ("has no transcript uuid for the session field", + f"LANES_SESSION is unset, and `lanes-edit.sh last-session {RUNNER}` names none"), + id="no LANES_SESSION and no last session"), +] -def test_yes_under_LANES_NO_FETCH_is_refused(sweep): - """A writer whose fetch did not run is a writer S3 refuses (ratified choice - 2), and `--yes` fetches each checkout before it decides: refused before - the first removal, not met by the NOTED line after it (rv1b's MUST 3).""" +@pytest.mark.parametrize("env,said", REFUSALS) +def test_yes_is_refused_before_anything_is_removed(sweep, env, said): + """`--yes` IS REFUSED (2) BEFORE ANYTHING IS READ OR REMOVED where the + NOTED line it ends with could not be written: with no `LANES_LANE`, since + the line goes into the log of the lane that runs it; under + `LANES_NO_FETCH=1`, a writer S3 refuses (ratified choice 2), since + `--yes` fetches each checkout before it decides (rv1b's MUST 3); and with + no transcript uuid for the line's session field (Amendment 7(b), D19), + whether `LANES_SESSION` names something else or is unset and + `last-session` names none (rv4's NIT 5). Each is met here, and not by the + line after the removals.""" box = sweep.box before, head = box.everything(), box.origin_head() - result, _ = sweep("--yes", LANES_LANE=RUNNER, LANES_NO_FETCH="1") + result, _ = sweep("--yes", **env(box)) assert result.returncode == 2, (result.stdout, result.stderr) - assert "LANES_NO_FETCH=1 is set" in result.stderr, result.stderr + for text in (*said, "Nothing was removed"): + assert text in result.stderr, result.stderr assert box.everything() == before assert box.origin_head() == head @@ -1072,12 +1090,12 @@ def test_nothing_removed_writes_nothing(sweep): assert "NOTED in lane" not in result.stdout -def _wrapped_log(box: Sweep, name: str, log_body: str) -> str: - """A `LANES_EDIT` that runs `log_body` for `log` and the real +def _wrapped(box: Sweep, name: str, verb: str, body: str) -> str: + """A `LANES_EDIT` that runs `body` for `verb` and the real `lanes-edit.sh`, which `$REAL` names, for every other verb.""" wrapper = box.fakebin / name wrapper.write_text(f'#!/usr/bin/env bash\nREAL={REPO / "lanes-edit.sh"}\n' - f'if [ "${{1-}}" = log ]; then\n{log_body}fi\nexec bash "$REAL" "$@"\n', + f'if [ "${{1-}}" = {verb} ]; then\n{body}fi\nexec bash "$REAL" "$@"\n', encoding="utf-8") wrapper.chmod(0o755) return str(wrapper) @@ -1107,7 +1125,7 @@ def test_noted_line_not_written_exits_1_with_the_command(sweep): success: exit 1, and the exact command that writes it by hand, which then writes it.""" box = sweep.box - refusing = _wrapped_log(box, "lanes-edit-refusing-log", ( + refusing = _wrapped(box, "lanes-edit-refusing-log", "log", ( " printf 'FAKE: the register refused this line\\n' >&2\n" " exit 2\n")) result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=refusing) @@ -1131,7 +1149,7 @@ def test_noted_line_unconfirmed_is_never_written_again(sweep): lacks it by content: run, it finds nothing to send, and origin still holds the line once.""" box = sweep.box - exit3 = _wrapped_log(box, "lanes-edit-unconfirmed-log", ( + exit3 = _wrapped(box, "lanes-edit-unconfirmed-log", "log", ( ' bash "$REAL" "$@"; rc=$?\n' " if [ \"$rc\" = 0 ]; then printf 'FAKE: pushed, not confirmed\\n' >&2; exit 3; fi\n" ' exit "$rc"\n')) @@ -1178,20 +1196,6 @@ def test_noted_line_kept_unpublished_is_published_not_written_again(sweep): assert len(_noted_on_origin(box)) == 1, _noted_on_origin(box) -def test_yes_without_a_session_uuid_is_refused(sweep): - """The NOTED line's session field is a transcript uuid and only that - (Amendment 7(b)): a `LANES_SESSION` that is none is refused before - anything is removed, not met by the line after the removals.""" - box = sweep.box - before, head = box.everything(), box.origin_head() - result, _ = sweep("--yes", LANES_LANE=RUNNER, LANES_SESSION="unknown") - assert result.returncode == 2, (result.stdout, result.stderr) - assert "has no transcript uuid for the session field" in result.stderr, result.stderr - assert "Nothing was removed" in result.stderr - assert box.everything() == before - assert box.origin_head() == head - - def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): """D18: a repository LEANS ON A CLONE when its alternates file names an object directory anywhere AT OR UNDER that clone, not only the clone's own From a0d058ae63a8af5085273861d92df649b1536601 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 03:02:10 +0000 Subject: [PATCH 14/53] Amendment 23 S6.2 tests: a LIVE lane is kept by the LIVE rule's own reason, and the runner's own checkout and the one a caller's `LANES_REPO` names are kept even where their lane is CLOSED, so the battery's three mutants the module missed are killed (#198, slice 4) The 31-mutant battery at `cede054` killed `n-runner` and `n-xrepo` by their keep reason alone (the fixture's runner is IDLE, and its `LANES_REPO` names the PAUSED lane's checkout, so the class check kept both), and `n-live` survived. - The runner is the CLOSED lane: its own checkout is kept as the runner's, and only the former name's checkout is swept, by the dry run and by `--yes`. Under `n-runner` that checkout is a candidate. - `LANES_REPO` names the CLOSED lane's checkout: kept as `LANES_REPO`'s. Under `n-xrepo` it is a candidate. - A LIVE lane, and an ENDED lane whose transcript a live session record here names: each kept with "lane X is LIVE: a live session record here names it", dry run and `--yes`. No case can kill `n-live` by behaviour: `lanes` gives a LIVE lane no class at all (`lanes-edit.sh`: `ENDED|RETIRED) lr_class=closed`, `LIVE) :`), so no listing has a lane that is LIVE and classed closed, and the class check keeps every LIVE lane too. The LIVE rule is defence in depth, and the reason is pinned. Each of the three mutants was run once against these cases in a separate clone: all KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 459c1f6b2a696bc20885558db04c4a00378e1a9b) --- tests/test_a23_sweep_checkouts.py | 48 +++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 71dd97e..06d91f4 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -305,6 +305,54 @@ def test_a_paused_lane_is_kept_without_LANES_REPO(sweep): assert rows[str(box.checkout(PAUSED))] == ("keep", f"lane {PAUSED} is PAUSED, not CLOSED (Amendment 19)") +@pytest.mark.parametrize("lane", [LIVE, CLOSED], + ids=["a LIVE lane", "an ENDED lane a live session record names"]) +def test_a_live_lanes_checkout_is_kept_by_the_live_rule(sweep, lane): + """A LANE THAT IS LIVE HERE IS KEPT BY THE LIVE RULE, whose reason names + the live session record; so is a lane whose log says ENDED while a live + session record here names its transcript, which is LIVE and not CLOSED. + The class check would keep both as well, because `lanes` gives a LIVE + lane no class at all (`lanes-edit.sh`: `ENDED|RETIRED) lr_class=closed`, + `LIVE) :`), so no listing has a lane that is LIVE and classed closed: + the reason is what shows that the LIVE rule kept it.""" + box = sweep.box + if lane == CLOSED: + record, body = _live_record(box, CLOSED) + record.write_text(body + "\n", encoding="utf-8") + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert rows[str(box.checkout(lane))] == ( + "keep", f"lane {lane} is LIVE: a live session record here names it"), rows + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert box.checkout(lane).is_dir() + + +@pytest.mark.parametrize("yes", [False, True], ids=["dry run", "--yes"]) +def test_a_closed_runner_keeps_its_own_checkout(sweep, yes): + """THE RUNNER'S OWN CHECKOUT IS NEVER A CANDIDATE, even where the runner's + own lane is CLOSED: the NOTED line goes through that checkout. Only the + former name's checkout is swept.""" + box = sweep.box + result, rows = sweep(*(("--yes",) if yes else ()), LANES_LANE=CLOSED) + assert rows[str(box.checkout(CLOSED))] == ( + "keep", f"lane {CLOSED} runs this sweep: its checkout is the one its NOTED line goes through"), rows + assert result.returncode == (0 if yes else 3), (result.stdout, result.stderr) + assert _tagged(rows, "removed" if yes else "remove") == {str(box.checkout(OLD))}, rows + assert box.checkout(CLOSED).is_dir() + assert os.path.lexists(box.checkout(OLD)) is not yes + + +def test_a_closed_lanes_checkout_named_by_LANES_REPO_is_kept(sweep): + """THE CHECKOUT A CALLER'S `LANES_REPO` NAMES IS NEVER A CANDIDATE, even a + CLOSED lane's: a write of that window may go through it.""" + box = sweep.box + result, rows = sweep(LANES_REPO=str(box.checkout(CLOSED))) + assert rows[str(box.checkout(CLOSED))] == ( + "keep", "the checkout LANES_REPO names, which a write of this window may go through"), rows + assert result.returncode == 3, (result.stdout, result.stderr) + assert _tagged(rows, "remove") == {str(box.checkout(OLD))}, rows + + def test_a_W_that_is_a_lane_checkout_is_refused(sweep): """FROM A LANE WINDOW, `LANES_WORKSPACE_ROOT` names the lane's own checkout, and every OTHER lane's checkout would stand "beside W". The sweep refuses, From 7cf3147293232b42c121c522e38edb46c52dd1a2 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 03:10:40 +0000 Subject: [PATCH 15/53] Amendment 23 S6.2 tests: a lock taken over is left to its holder by `sc_lock_drop`, when something else holds the checkout first, and by `cleanup`, when a signal ends the sweep (rv4's NIT 4, #198, slice 4) `10468ac` made `sc_lock_drop` and `cleanup` leave a lock whose pid line names another holder, as `mutex_release` does, but its case reached neither: the check before the move clears the sweep's hold on the lock first. Two cases now reach them: - the takeover case is parametrized: the pid line is replaced at the checkout's fetch and that fetch then fails, so the checkout is held for the fetch and `sc_lock_drop` runs; the other writer's pid line is left; - a new case: the pid line is replaced at the fetch and the sweep is sent TERM; it exits 143, and `cleanup` on the way out leaves that pid line and the checkout. With either ownership check made unconditional, its case fails. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit cd060f09a9644397be53e8c3e6169e97f45b72b0) --- tests/test_a23_sweep_checkouts.py | 34 +++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 06d91f4..fdbfab8 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -780,21 +780,43 @@ def test_a_checkout_whose_origin_changes_after_the_listing_is_held(sweep): assert c.is_dir() -def test_a_lock_taken_over_under_the_sweep_holds_and_is_left(sweep): +@pytest.mark.parametrize("then,why", [ + ("", "its writers' lock lanes/.lanes-edit.lock is no longer this sweep's (1 other-container otherhost)"), + ("; exit 1", "its fetch of origin main failed"), +], ids=["then the move is asked", "then the fetch fails"]) +def test_a_lock_taken_over_under_the_sweep_holds_and_is_left(sweep, then, why): """THE LOCK IS THE SWEEP'S UNTIL THE MOVE, OR NOTHING IS MOVED (rv4's NIT 4): at the checkout's fetch another writer's pid line replaces the sweep's in its `lanes/.lanes-edit.lock`, as a takeover from another - container does. The checkout is held, and that writer's lock is left as - it is, as `lanes-edit.sh`'s `mutex_release` leaves a lock it no longer - holds.""" + container does. The checkout is held, by the pid line read again before + the move, or by whatever else holds it first (here, a fetch that fails), + and in both that writer's lock is left as it is, as `lanes-edit.sh`'s + `mutex_release` leaves a lock it no longer holds.""" box = sweep.box c = box.checkout(CLOSED) pid = c / "lanes" / ".lanes-edit.lock" / "pid" - _hook_on_fetch_of(box, c, f"printf '1 other-container otherhost\\n' > {pid}") + _hook_on_fetch_of(box, c, f"printf '1 other-container otherhost\\n' > {pid}{then}") result, rows = sweep("--yes", LANES_LANE=RUNNER) assert rows[str(c)][0] == "hold", rows[str(c)] assert result.returncode == 0, (result.stdout, result.stderr) - assert "is no longer this sweep's (1 other-container otherhost)" in rows[str(c)][1], rows[str(c)] + assert why in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" + + +def test_a_lock_taken_over_is_left_by_a_sweep_ended_by_a_signal(sweep): + """THE WAY OUT LEAVES A LOCK TAKEN OVER, TOO (rv4's NIT 4): at the + checkout's fetch, another writer's pid line replaces the sweep's and the + sweep is sent TERM. It exits 143, and `cleanup`, run on the way out, + leaves that writer's lock as it is and the checkout where it was.""" + box = sweep.box + c = box.checkout(CLOSED) + pid = c / "lanes" / ".lanes-edit.lock" / "pid" + _hook_on_fetch_of(box, c, ( + f"read -r swept _ < {pid}; printf '1 other-container otherhost\\n' > {pid}; " + 'kill -TERM "$swept"; exit 1')) + result, _ = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 143, (result.stdout, result.stderr) assert c.is_dir() assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" From ee326d2d40526983c9b0044d82ec42d5ccdd8764 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 08:00:04 +0000 Subject: [PATCH 16/53] Amendment 23 S6.2: a former name's checkout is a candidate only once its lane is CLOSED, as every other checkout is, at the listing and in the re-ask under its lock, since a process started before the rename may still write through it (rv4-c3's SHOULD c3-1, D29, #198, slice 4) D27 kept a former name's checkout beside its new one only while its lane was LIVE. LIVE is a pid test in this container, so it missed two things that still carry the old `LANES_REPO=-lanes/`: a process started from a window before the rename (a landing pilot) that outlives that window's session, and a pre-rename window in another container that shares `-lanes/`. Either way the lane reads PAUSED or IDLE, the old checkout was removed, and that process's next register write (a HOLD's release, say) was refused by S3 as "not a git checkout", so an open HOLD blocked every landing. Nothing was lost, since the holds passed. - D29 takes rv4-c3's first option and supersedes D27's LIVE test: `sc_judge` judges a former name's checkout beside its new one through `sc_judge_lane`, as the lane's own checkout is (the runner's lane kept, a lane the register does not know kept, LIVE kept, anything but CLOSED kept, CLOSED a candidate). The keep line says first that it is "a former name of lane X, beside that lane's own checkout, kept until that lane is CLOSED: a process started before the rename may still write through this path"; the candidate line names the new checkout and "lane X is CLOSED ()". - `sc_still`: the former-name re-ask under the lock no longer returns after its rename checks; it falls through to the LIVE and CLOSED checks every candidate gets, so a lane that RESUMES or comes LIVE after the listing holds the checkout. - The cost: a renamed lane's old checkout stays until the lane closes. - Tests, red at `4f31084` (17 of 19 cases; the former name is `remove` or `removed`) and green here (19 passed): - new: rv4-c3's P1 (a `sleep` carrying `LANES_REPO=-lanes/repoS4-8`, lane repoS4-9 IDLE: kept by the dry run and `--yes`); a former name of a CLOSED lane is removed, with its lane's own checkout; a former name whose lane RESUMES at its fetch is held "IDLE now, not CLOSED"; - the D27 cases read D29's reason, and the one that comes LIVE after the listing starts from a CLOSED lane; - the cases that counted repoS4-8's checkout as removable by default now count repoS4-2's alone, or a CHAIN lane's checkout where they need a second candidate (`_closed_checkout`); `_new_name_ended` ends repoS4-9 on origin and in W. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 67e3433a30f07283d1090a07e8d3edf423ee2699) --- lane-end | 59 +++++----- tests/test_a23_sweep_checkouts.py | 180 ++++++++++++++++++++++-------- 2 files changed, 167 insertions(+), 72 deletions(-) diff --git a/lane-end b/lane-end index 1a74d24..0e1aeee 100755 --- a/lane-end +++ b/lane-end @@ -937,7 +937,8 @@ fi # register calls CLOSED (`lanes --closed`'s class, Amendment 19), or the old # name's checkout beside its new one after a rename (S1: *"the writer writes # through `-lanes//` and reports `-lanes//`, which S6's sweep -# lists"*). NEVER W, never a LIVE lane's checkout, never the checkout of the +# lists"*) once that lane is CLOSED too. NEVER W, never a LIVE lane's +# checkout, never a lane's that is not CLOSED, never the checkout of the # lane that runs the sweep or of the `LANES_REPO` its caller names, never a # symbolic link and never anything a link leads to. A CLONE BESIDE W IS LISTED, # WITH WHAT IT HOLDS, AND NEVER REMOVED BY THIS BUILD: the brief this slice was @@ -1184,25 +1185,32 @@ SC_IDX sc_cands+=("$1"); sc_cand_lane+=("$2"); sc_cand_why+=("$3"); sc_cand_new+=("${4-}") } # A lane's checkout, judged by the lane's state: CLOSED is a candidate, - # LIVE never is, and every other state is a lane somebody may pick up. - sc_judge_lane() { # + # LIVE never is, and every other state is a lane somebody may pick up. A + # former name's checkout beside the new name's own (its third argument) is + # judged by the same rule, and says first what it is. + sc_judge_lane() { # [] + sjl_new="${3-}"; sjl_keep=""; sjl_cand="" + if [ -n "$sjl_new" ]; then + sjl_keep="a former name of lane $2, beside that lane's own checkout, kept until that lane is CLOSED: a process started before the rename may still write through this path; " + sjl_cand="a former name of lane $2, beside that lane's own checkout $sjl_new; " + fi if [ -n "$sc_runner" ] && [ "$(sc_lc "$2")" = "$(sc_lc "$sc_runner")" ]; then - sc_keep "$1" "lane $2 runs this sweep: its checkout is the one its NOTED line goes through" + sc_keep "$1" "${sjl_keep}lane $2 runs this sweep: its checkout is the one its NOTED line goes through" return 0 fi if ! sc_lookup "$2"; then - sc_keep "$1" "no lane $2 is in the register or the logs, so it is no CLOSED lane's" + sc_keep "$1" "${sjl_keep}no lane $2 is in the register or the logs, so it is no CLOSED lane's" return 0 fi if [ "$SC_L_STATE" = LIVE ]; then - sc_keep "$1" "lane $SC_L_LANE is LIVE: a live session record here names it" + sc_keep "$1" "${sjl_keep}lane $SC_L_LANE is LIVE: a live session record here names it" return 0 fi if [ "$SC_L_CLASS" != closed ]; then - sc_keep "$1" "lane $SC_L_LANE is $SC_L_STATE, not CLOSED (Amendment 19)" + sc_keep "$1" "${sjl_keep}lane $SC_L_LANE is $SC_L_STATE, not CLOSED (Amendment 19)" return 0 fi - sc_cand "$1" "$SC_L_LANE" "lane $SC_L_LANE is CLOSED ($SC_L_STATE)" + sc_cand "$1" "$SC_L_LANE" "${sjl_cand}lane $SC_L_LANE is CLOSED ($SC_L_STATE)" "$sjl_new" } sc_judge() { # sj_p="$1"; sj_n="$2" @@ -1231,23 +1239,24 @@ SC_IDX sc_keep "$sj_p" "no lane $sj_n is in the register or the logs, so it is no CLOSED lane's" return 0 fi - # A FORMER NAME OF LANE `sj_c`. Beside the new name's own checkout it is a - # candidate, because no write of the lane's goes through it any more (S1), - # UNLESS THE LANE IS LIVE (rv4's SHOULD 4, D27): since 10-19 the launcher - # exports `LANES_REPO=-lanes/`, and a window of the lane started - # before the rename still names the old path, so removing it would fail - # that window's writes until it is relaunched. A window of the lane on - # this workstation is a live session record naming it, which is what - # LIVE is. With no checkout under the new name it IS the lane's checkout + # A FORMER NAME OF LANE `sj_c`, JUDGED AS THAT LANE'S OWN CHECKOUT IS: a + # candidate only once the lane is CLOSED (rv4-c3's SHOULD c3-1, D29, which + # supersedes D27's LIVE test). Beside the new name's own checkout, no + # write the tooling starts goes through it any more (S1). But since 10-19 + # the launcher exports `LANES_REPO=-lanes/`, and whatever was + # started before the rename still names the old path: a window, or a + # landing pilot that outlives its window's session, so that the lane no + # longer reads LIVE; or one in another container that shares `-lanes/`, + # where no session record here names it. LIVE sees neither, and removing + # the path would refuse that process's next register write (S3: it is no + # git checkout), a HOLD's release among them. CLOSED is the rule every + # other checkout is swept by: the lane has ended, and its own checkout + # goes then too. The cost is the old checkout's tree, kept until the lane + # closes. With no checkout under the new name it IS the lane's checkout # until the next write renames it (S1), and is judged as one. sj_new="$sc_home/$sj_c" if [ -d "$sj_new" ] && [ ! -L "$sj_new" ] && [ ! "$sj_new" -ef "$sj_p" ]; then - if sc_lookup "$sj_c" && [ "$SC_L_STATE" = LIVE ]; then - sc_keep "$sj_p" "a former name of lane $SC_L_LANE, which is LIVE: a window of it started before the rename may still write through this path, which its LANES_REPO names" - return 0 - fi - sc_cand "$sj_p" "$sj_c" "a former name of lane $sj_c, beside that lane's own checkout $sj_new" "$sj_new" - return 0 + sc_judge_lane "$sj_p" "$sj_c" "$sj_new"; return 0 fi sc_judge_lane "$sj_p" "$sj_c" } @@ -1635,15 +1644,13 @@ SCC IFS="$sc_tab" read -r ss_lane ss_state ss_ws ss_pf ss_win ss_sid ss_dir ss_obj ss_age ss_cmd ss_home ss_fk ss_loc ss_class ss_rest </dev/null)" || ss_c="" if [ "$(sc_lc "$ss_c")" != "$(sc_lc "$2")" ] || [ ! -d "$3" ] || [ -L "$3" ] || [ "$3" -ef "$1" ]; then SC_WHY="it is no longer a former name of lane $2 beside that lane's own checkout"; return 1 fi - if [ "${ss_state:-}" = LIVE ]; then - SC_WHY="lane $2 is LIVE now, and a window of it started before the rename may still write through this path (D27)"; return 1 - fi - return 0 fi if [ "${ss_state:-}" = LIVE ]; then SC_WHY="lane $2 is LIVE now"; return 1; fi if [ "${ss_class:-}" != closed ]; then SC_WHY="lane $2 is ${ss_state:-in no state} now, not CLOSED"; return 1; fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index fdbfab8..1c5eadc 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -183,19 +183,37 @@ def _tagged(rows: dict, tag: str) -> set: return {path for path, (t, _) in rows.items() if t == tag} +def _closed_checkout(box: Sweep) -> Path: + """The first CHAIN lane's checkout, made, borrowing from W alone: a CLOSED + lane's checkout for a case that needs a candidate besides repoS4-2's.""" + return box.clone(box.checkout(CHAIN[0]), reference=box.w) + + +def _new_name_ended(box: Sweep) -> None: + """Lane repoS4-9, OLD's new name, ENDS on origin (its log and its row), + and W reads it, since a dry run reads what W last fetched: OLD's checkout + is then a former name of a CLOSED lane, and repoS4-9's own is CLOSED too.""" + box.peer_edits(f"lanes/log/{NEW}.md", _log(NEW, "STARTED"), _log(NEW, "STARTED", "ENDED"), + f"{NEW} ends") + box.peer_edits(REGISTER, _row(NEW, "LIVE · renamed from repoS4-8"), _row(NEW, "ENDED · closed"), + f"{NEW}'s row says so") + box.git("-C", str(box.w), "pull", "-q", "--ff-only", "origin", "main") + + # ------------------------------------------------------------- the dry run def test_dry_run_lists_and_removes_nothing(sweep): - """THE DEFAULT IS THE DRY RUN: it names the CLOSED lane's checkout and the - former name's beside its new one as candidates, and removes, fetches and - locks nothing (Amendment 19(c)'s shape, S6.2). 3 says something is there - to remove.""" + """THE DEFAULT IS THE DRY RUN: it names the CLOSED lane's checkout as a + candidate, keeps the former name's beside its new one while that lane is + not CLOSED (D29), and removes, fetches and locks nothing (Amendment + 19(c)'s shape, S6.2). 3 says something is there to remove.""" box = sweep.box before, head = box.everything(), box.origin_head() result, rows = sweep() assert result.returncode == 3, (result.stdout, result.stderr) assert "DRY RUN" in result.stdout, result.stdout - assert _tagged(rows, "remove") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert _tagged(rows, "remove") == {str(box.checkout(CLOSED))}, rows + assert rows[str(box.checkout(OLD))][0] == "keep", rows assert rows[str(box.checkout(OLD))][1].startswith(f"a former name of lane {NEW}"), rows assert box.everything() == before, "a dry run changes nothing under -lanes/ or beside W" assert box.origin_head() == head, "nothing reached origin" @@ -204,11 +222,10 @@ def test_dry_run_lists_and_removes_nothing(sweep): def test_dry_run_with_nothing_to_remove_exits_0(sweep): - """0 is nothing to remove: with the CLOSED lane's checkout and the former - name's gone, every entry is kept.""" + """0 is nothing to remove: with the CLOSED lane's checkout gone, every + entry is kept, the former name's too (its lane is not CLOSED, D29).""" box = sweep.box - for lane in (CLOSED, OLD): - box.checkout(lane).rename(box.root / f"aside-{lane}") + box.checkout(CLOSED).rename(box.root / f"aside-{CLOSED}") result, rows = sweep() assert result.returncode == 0, (result.stdout, result.stderr) assert not _tagged(rows, "remove"), rows @@ -295,7 +312,7 @@ def test_W_and_live_lane_never_candidates(sweep): assert f"lane {NEW} is IDLE" in keep[str(box.checkout(NEW))], keep assert not any(".lock" in path for path in rows), "hidden entries are never looked at" assert "1 hidden entr(ies)" in result.stdout, result.stdout - assert _tagged(rows, "remove") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))} + assert _tagged(rows, "remove") == {str(box.checkout(CLOSED))}, rows def test_a_paused_lane_is_kept_without_LANES_REPO(sweep): @@ -330,27 +347,29 @@ def test_a_live_lanes_checkout_is_kept_by_the_live_rule(sweep, lane): @pytest.mark.parametrize("yes", [False, True], ids=["dry run", "--yes"]) def test_a_closed_runner_keeps_its_own_checkout(sweep, yes): """THE RUNNER'S OWN CHECKOUT IS NEVER A CANDIDATE, even where the runner's - own lane is CLOSED: the NOTED line goes through that checkout. Only the - former name's checkout is swept.""" + own lane is CLOSED: the NOTED line goes through that checkout. Only + another CLOSED lane's checkout is swept.""" box = sweep.box + other = _closed_checkout(box) result, rows = sweep(*(("--yes",) if yes else ()), LANES_LANE=CLOSED) assert rows[str(box.checkout(CLOSED))] == ( "keep", f"lane {CLOSED} runs this sweep: its checkout is the one its NOTED line goes through"), rows assert result.returncode == (0 if yes else 3), (result.stdout, result.stderr) - assert _tagged(rows, "removed" if yes else "remove") == {str(box.checkout(OLD))}, rows + assert _tagged(rows, "removed" if yes else "remove") == {str(other)}, rows assert box.checkout(CLOSED).is_dir() - assert os.path.lexists(box.checkout(OLD)) is not yes + assert os.path.lexists(other) is not yes def test_a_closed_lanes_checkout_named_by_LANES_REPO_is_kept(sweep): """THE CHECKOUT A CALLER'S `LANES_REPO` NAMES IS NEVER A CANDIDATE, even a CLOSED lane's: a write of that window may go through it.""" box = sweep.box + other = _closed_checkout(box) result, rows = sweep(LANES_REPO=str(box.checkout(CLOSED))) assert rows[str(box.checkout(CLOSED))] == ( "keep", "the checkout LANES_REPO names, which a write of this window may go through"), rows assert result.returncode == 3, (result.stdout, result.stderr) - assert _tagged(rows, "remove") == {str(box.checkout(OLD))}, rows + assert _tagged(rows, "remove") == {str(other)}, rows def test_a_W_that_is_a_lane_checkout_is_refused(sweep): @@ -391,14 +410,15 @@ def _no_aside(box: Sweep) -> None: def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): - """`--yes`, run by a lane, removes the CLOSED lane's checkout and the former - name's beside its new one, and leaves every other entry — the LIVE lane's, - the PAUSED one's, a non-clone, a link — and every clone beside W exactly - as they were. Its own checkout moves only by the NOTED commit it pushed.""" + """`--yes`, run by a lane, removes the CLOSED lane's checkout, and leaves + every other entry — the LIVE lane's, the PAUSED one's, the former name's + beside its new one while that lane is not CLOSED (D29), a non-clone, a + link — and every clone beside W exactly as they were. Its own checkout + moves only by the NOTED commit it pushed.""" box = sweep.box before = box.everything() runner = str(box.checkout(RUNNER)) - gone = {str(box.checkout(CLOSED)), str(box.checkout(OLD))} + gone = {str(box.checkout(CLOSED))} result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 0, (result.stdout, result.stderr) assert _tagged(rows, "removed") == gone, rows @@ -408,7 +428,7 @@ def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): after = box.everything() assert {k: v for k, v in before.items() if k not in gone and k != runner} == \ {k: v for k, v in after.items() if k != runner}, \ - "nothing but the two removed entries changed, under -lanes/ or beside W" + "nothing but the removed entry changed, under -lanes/ or beside W" assert box.git("-C", runner, "rev-parse", "HEAD").strip() == box.origin_head(), \ "the runner's checkout moved only to the NOTED commit it pushed" assert after[runner][1:] == before[runner][1:], "and holds nothing new" @@ -688,11 +708,13 @@ def test_a_lane_resumed_after_the_listing_is_kept(sweep): def test_a_former_name_whose_new_checkout_goes_after_the_listing_is_kept(sweep): """A FORMER NAME IS ASKED AGAIN UNDER THE LOCK (rv4's SHOULD 1, R3). The - listing finds repoS4-8's checkout beside its new name's; at its own fetch - the new name's checkout goes, so the former name's checkout IS that - lane's own again until its next write renames it (S1), and is kept.""" + listing finds repoS4-8's checkout beside its new name's, a CLOSED lane's + (D29); at its own fetch the new name's checkout goes, so the former + name's checkout IS that lane's own again until its next write renames it + (S1), and is kept.""" box = sweep.box old, new = box.checkout(OLD), box.checkout(NEW) + _new_name_ended(box) _, rows = sweep() assert rows[str(old)][0] == "remove", rows[str(old)] _hook_on_fetch_of(box, old, f"mv -- {new} {box.root / 'aside-new'}") @@ -718,31 +740,97 @@ def _live_session_of(box: Sweep, lane: str) -> str: return f"printf '%s\\n' '{body}' > {record}" +def _former_name_kept(lane_said: str) -> str: + """The keep reason of OLD's checkout beside repoS4-9's (D29).""" + return (f"a former name of lane {NEW}, beside that lane's own checkout, kept until that " + f"lane is CLOSED: a process started before the rename may still write through " + f"this path; {lane_said}") + + +def test_a_former_name_of_a_lane_not_closed_is_kept(sweep): + """A FORMER NAME'S CHECKOUT IS SWEPT ONLY ONCE ITS LANE IS CLOSED, as every + other checkout is (rv4-c3's SHOULD c3-1, P1; D29, superseding D27's LIVE + test). Since 10-19 the launcher exports `LANES_REPO=-lanes/`, and + a process started before the rename still names the old path: here a + landing pilot's stand-in, a `sleep` carrying `LANES_REPO=-lanes/OLD`, + working elsewhere, that outlived its window's session, so repoS4-9 is + IDLE and not LIVE. The checkout is kept by the dry run and by `--yes`.""" + box = sweep.box + old = box.checkout(OLD) + pilot = subprocess.Popen(["sleep", "600"], cwd=str(box.root), + env=dict(box.env, LANES_REPO=str(old))) + try: + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert rows[str(old)] == ("keep", _former_name_kept( + f"lane {NEW} is IDLE, not CLOSED (Amendment 19)")), rows[str(old)] + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert old.is_dir() + finally: + pilot.kill() + pilot.wait() + + +def test_a_former_name_of_a_closed_lane_is_removed(sweep): + """ONCE ITS LANE IS CLOSED, A FORMER NAME'S CHECKOUT BESIDE ITS NEW ONE IS A + CANDIDATE (S1: *"reports `-lanes//`, which S6's sweep lists"*; + D29): repoS4-9 ENDS, so the dry run lists repoS4-8's checkout as a former + name to remove, and `--yes` removes it, and repoS4-9's own CLOSED one.""" + box = sweep.box + old, new = box.checkout(OLD), box.checkout(NEW) + _new_name_ended(box) + result, rows = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert rows[str(old)] == ("remove", f"a former name of lane {NEW}, beside that lane's own " + f"checkout {new}; lane {NEW} is CLOSED (ENDED)"), rows[str(old)] + assert rows[str(new)][0] == "remove", rows[str(new)] + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(old), str(new)}, rows + assert not os.path.lexists(old) + assert not os.path.lexists(new) + + +def test_a_former_name_whose_lane_resumes_after_the_listing_is_kept(sweep): + """A FORMER NAME'S LANE IS ASKED AGAIN UNDER THE LOCK, as every lane is + (D29): repoS4-9 is CLOSED at the listing and RESUMES on origin at its + former name's own fetch, so the re-ask reads it IDLE, and the checkout + is kept.""" + box = sweep.box + old = box.checkout(OLD) + _new_name_ended(box) + _hook_on_fetch_of(box, old, _resumed_on_origin(box, NEW)) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(old)][0] == "hold", rows[str(old)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert f"lane {NEW} is IDLE now, not CLOSED" in rows[str(old)][1], rows[str(old)] + assert old.is_dir() + + def test_a_former_name_of_a_live_lane_is_kept(sweep): - """A FORMER NAME'S CHECKOUT IS KEPT WHILE ITS LANE IS LIVE (rv4's SHOULD 4, - D27). Since 10-19 the launcher exports `LANES_REPO=-lanes/`, and - a window of the lane started before the rename still names the old path, - so removing it makes that window's writes fail until it is relaunched. - Here repoS4-9 is LIVE: a live session record names its transcript. Its - former name's checkout is kept by the dry run and by `--yes`.""" + """A FORMER NAME'S CHECKOUT IS KEPT WHILE ITS LANE IS LIVE (rv4's SHOULD 4), + as while it is anything but CLOSED (D29). Here repoS4-9 is LIVE: a live + session record names its transcript. Its former name's checkout is kept + by the dry run and by `--yes`.""" box = sweep.box old = box.checkout(OLD) record, body = _live_record(box, NEW) record.write_text(body + "\n", encoding="utf-8") for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): result, rows = sweep(*args, **env) - assert rows[str(old)][0] == "keep", rows[str(old)] + assert rows[str(old)] == ("keep", _former_name_kept( + f"lane {NEW} is LIVE: a live session record here names it")), rows[str(old)] assert result.returncode in (0, 3), (result.stdout, result.stderr) - assert f"a former name of lane {NEW}, which is LIVE" in rows[str(old)][1], rows[str(old)] assert old.is_dir() def test_a_former_name_whose_lane_goes_live_after_the_listing_is_kept(sweep): - """THE SAME RULE, ASKED AGAIN UNDER THE LOCK (D27): repoS4-9 comes LIVE at - its former name's own fetch, after the listing found that checkout a - candidate. It is kept.""" + """THE SAME RULE, ASKED AGAIN UNDER THE LOCK (D29): repoS4-9 is CLOSED at + the listing, so its former name's checkout is a candidate, and comes LIVE + at that checkout's own fetch. It is kept.""" box = sweep.box old = box.checkout(OLD) + _new_name_ended(box) _hook_on_fetch_of(box, old, _live_session_of(box, NEW)) result, rows = sweep("--yes", LANES_LANE=RUNNER) assert rows[str(old)][0] == "hold", rows[str(old)] @@ -1077,6 +1165,7 @@ def test_a_removal_that_does_not_finish_exits_1(sweep): the run exits 1, and the aside stays, hidden, under `-lanes/`.""" box = sweep.box c = box.checkout(CLOSED) + other = _closed_checkout(box) (c / ".git" / "info" / "exclude").write_text("scratch/\n", encoding="utf-8") box.write(c, "scratch/kept.txt", "an ignored file in a read-only directory\n") (c / "scratch").chmod(0o555) @@ -1097,7 +1186,7 @@ def test_a_removal_that_does_not_finish_exits_1(sweep): noted = _noted_on_origin(box) assert len(noted) == 1, noted assert (f"removed 1 register checkout(s) under brett-wip-lanes/ that held nothing " - f"unpublished: {OLD}; moved aside and not all removed: {asides[0].name};") in noted[0], noted + f"unpublished: {other.name}; moved aside and not all removed: {asides[0].name};") in noted[0], noted finally: for aside in box.lanes_root.glob(f".{CLOSED}.removing-*"): (aside / "scratch").chmod(0o755) @@ -1119,14 +1208,14 @@ def test_noted_line_written_through_runner_checkout(sweep): w_head = box.git("-C", str(box.w), "rev-parse", "HEAD").strip() result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 0, (result.stdout, result.stderr) - assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows noted = [line for line in box.origin_file(f"lanes/log/{RUNNER}.md").splitlines() if line.startswith("NOTED")] assert len(noted) == 1, noted assert f"lane {RUNNER}, session {UUID[RUNNER]}@" in noted[0], noted assert noted[0].endswith( - "sweep-checkouts (Amendment 23 S6.2): removed 2 register checkout(s) under " - f"brett-wip-lanes/ that held nothing unpublished: {CLOSED} {OLD}; " + "sweep-checkouts (Amendment 23 S6.2): removed 1 register checkout(s) under " + f"brett-wip-lanes/ that held nothing unpublished: {CLOSED}; " "3 clone(s) beside W listed and left"), noted assert str(box.root) not in noted[0], "no host path is written into the log" assert box.git("-C", str(runner), "rev-parse", "HEAD").strip() == box.origin_head(), \ @@ -1144,10 +1233,9 @@ def test_noted_line_written_through_runner_checkout(sweep): def test_nothing_removed_writes_nothing(sweep): """Nothing removed is nothing to record: with the CLOSED lane's checkout - held and the former name's gone, `--yes` removes nothing, writes no line - and makes no commit.""" + held, and the former name's kept (its lane is not CLOSED, D29), `--yes` + removes nothing, writes no line and makes no commit.""" box = sweep.box - box.checkout(OLD).rename(box.root / f"aside-{OLD}") _commit_unpushed(box, box.checkout(CLOSED)) head = box.origin_head() runner_head = box.git("-C", str(box.checkout(RUNNER)), "rev-parse", "HEAD").strip() @@ -1200,7 +1288,7 @@ def test_noted_line_not_written_exits_1_with_the_command(sweep): " exit 2\n")) result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=refusing) assert result.returncode == 1, (result.stdout, result.stderr) - assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows assert "the NOTED line recording this sweep was NOT written" in result.stderr, result.stderr assert "Nothing was lost" in result.stderr by_hand = result.stderr.split("Write the line by hand: ", 1)[1].strip() @@ -1208,7 +1296,7 @@ def test_noted_line_not_written_exits_1_with_the_command(sweep): assert not _noted_on_origin(box) done = _by_hand(box, by_hand) assert done.returncode == 0, (done.stdout, done.stderr) - assert "sweep-checkouts (Amendment 23 S6.2): removed 2" in box.origin_file(f"lanes/log/{RUNNER}.md") + assert "sweep-checkouts (Amendment 23 S6.2): removed 1" in box.origin_file(f"lanes/log/{RUNNER}.md") def test_noted_line_unconfirmed_is_never_written_again(sweep): @@ -1225,7 +1313,7 @@ def test_noted_line_unconfirmed_is_never_written_again(sweep): ' exit "$rc"\n')) result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=exit3) assert result.returncode == 1, (result.stdout, result.stderr) - assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows assert len(_noted_on_origin(box)) == 1 assert "was NOT written" not in result.stderr, result.stderr assert "Write the line by hand" not in result.stderr, result.stderr @@ -1251,7 +1339,7 @@ def test_noted_line_kept_unpublished_is_published_not_written_again(sweep): box.use_net(hook=f'case "$1" in push) if [ -e {gate} ]; then exit 1; fi ;; esac\nexit 0\n') result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 1, (result.stdout, result.stderr) - assert _tagged(rows, "removed") == {str(box.checkout(CLOSED)), str(box.checkout(OLD))}, rows + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows assert not _noted_on_origin(box) assert "was NOT written" not in result.stderr, result.stderr assert "kept on HEAD" in result.stderr, result.stderr From 513364dc778282f4f54d5b143ce121e580fb77de Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 08:03:15 +0000 Subject: [PATCH 17/53] Amendment 23 S6.2: a sweep whose pid line cannot be written into the writers' lock it just took lets that lock go and holds the checkout, so it no longer leaves a lock that names no holder, with a false "taken over" reason (rv4-c3's NIT c3-2, #198, slice 4) `sc_lock_take` wrote its pid line with `|| :`. When the line was never written (a full disk, a directory that cannot be written in), the check before the move read an empty line, held the checkout as "a writer took it over", and cleared `SC_LOCK`, so the empty lock directory the sweep had made stayed. Every later sweep then held that CLOSED lane's checkout for it, and its writers waited out `mutex_stale`'s ten minutes. - `sc_lock_take`: a pid line that cannot be written empties and removes the lock just made, and holds the checkout: "its writers' lock was taken, and this sweep's pid line could not be written into it, so the lock was let go again" (or, where `rmdir` fails too, that the lock is left). No writer can have taken a lock from a live holder in under ten minutes, so the lock is still the sweep's to remove. An empty line at the check before the move is then only ever another writer's. - Test (red at the parent: the lock is left; green here): `test_a_lock_whose_pid_line_cannot_be_written_is_let_go`. A `mkdir` first on PATH leaves the new lock directory read-only (mode 500), so the sweep's own `printf` fails, as rv4-c3's P2 has it. P2 itself took the pid file away at the fetch, after the line was written; under this fix that lock reads as another writer's and is left to it, as the takeover case already pins, so the module makes the write itself fail. Skipped as root. - Not changed: `sc_lock_drop` and `cleanup` still release a lock whose pid line cannot be read, as `lanes-edit.sh`'s `mutex_release` does. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 95f62f1d830e1770a6beed1ebce80e423fa93cc8) --- lane-end | 20 +++++++++++++++++++- tests/test_a23_sweep_checkouts.py | 30 ++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/lane-end b/lane-end index 0e1aeee..3e6b746 100755 --- a/lane-end +++ b/lane-end @@ -1561,7 +1561,25 @@ SCC SC_WHY="its writers' lock lanes/.lanes-edit.lock is held: a writer is in it, or one left it behind"; return 1 fi SC_LOCK="$slt_d" - printf '%s %s\n' "$$" "$sc_ident" > "$slt_d/pid" 2>/dev/null || : + # ITS PID LINE IS WRITTEN, OR THE LOCK IS LET GO AT ONCE (rv4-c3's NIT + # c3-2). A lock that names no holder is one its writers wait ten minutes + # on (`mutex_stale`), and the check before the move would read this + # sweep's own pid-less lock as taken over and leave it, so every later + # sweep would hold the checkout for it. A line that cannot be written (a + # full disk, a directory that cannot be written in) empties and removes + # the lock just made, which no writer can have taken from a live holder + # in under ten minutes, and holds the checkout. So a lock whose line is + # not this sweep's is only ever another writer's. + if ! printf '%s %s\n' "$$" "$sc_ident" 2>/dev/null > "$slt_d/pid"; then + rm -f -- "$slt_d/pid" 2>/dev/null || : + SC_LOCK="" + if rmdir -- "$slt_d" 2>/dev/null; then + SC_WHY="its writers' lock was taken, and this sweep's pid line could not be written into it, so the lock was let go again: one that names no holder is one its writers wait ten minutes on" + else + SC_WHY="its writers' lock was taken, and this sweep's pid line could not be written into it, and the lock could not be let go again: lanes/.lanes-edit.lock is left, and its writers wait out its ten minutes" + fi + return 1 + fi return 0 } # THE LOCK STAYS THE SWEEP'S, OR IT IS LEFT AS IT IS (rv4's NIT 4). A holder diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 1c5eadc..db1405f 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -909,6 +909,36 @@ def test_a_lock_taken_over_is_left_by_a_sweep_ended_by_a_signal(sweep): assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root writes in a directory it may not write in") +def test_a_lock_whose_pid_line_cannot_be_written_is_let_go(sweep): + """THE SWEEP'S PID LINE IS WRITTEN, OR ITS LOCK IS LET GO (rv4-c3's NIT + c3-2, P2). A lock that names no holder is one its writers wait ten + minutes on (`mutex_stale`), and the check before the move read the + sweep's own pid-less lock as taken over and left it there, so every + later sweep held that checkout. Here the sweep's `mkdir` of the CLOSED + lane's lock leaves the new directory read-only (a `mkdir` first on + PATH), so the line cannot be written into it: the checkout is held with + that reason, the lock is gone, and the next sweep removes the checkout.""" + box = sweep.box + c = box.checkout(CLOSED) + lock = box.lock_of(c) + shim = box.fakebin / "mkdir" + shim.write_text(f'#!/usr/bin/env bash\n{shutil.which("mkdir")} "$@" || exit\n' + f'case " $* " in *" {lock} "*) chmod 500 {lock} ;; esac\n', encoding="utf-8") + shim.chmod(0o755) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert not os.path.lexists(lock), "a lock that names no holder is not left behind" + assert ("its writers' lock was taken, and this sweep's pid line could not be written " + "into it, so the lock was let go again") in rows[str(c)][1], rows[str(c)] + shim.rename(box.root / "mkdir.aside") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "removed", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + + @pytest.mark.parametrize("op,note", [ ("fetch", "fetch failed — reading the logs as they stand locally"), ("ls-remote", "NO FETCH WAS MADE"), From 7f45bf8d42091ae54da49df3637867e16cfb6a40 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 08:06:00 +0000 Subject: [PATCH 18/53] Amendment 23 S6.2: a `.git` file whose `gitdir:` line ends in CRLF is read as git reads it, so a directory working in a clone's git directory through one still holds that clone (rv4-c3's NIT c3-3, #198, slice 4) `sc_dep_scan` took the `gitdir:` line with `sed`, which kept the line's trailing CR, so the path it named did not resolve and the directory was not seen. git strips the CR and works in the clone's git directory, so the dry run listed that clone as `remove`, and `--yes` would have removed the git directory another checkout works in. - `sc_dep_scan`: `tr -d '\r'` strips the CR before `sed` reads the line. The `\r` is `tr`'s, not `sed`'s, because BSD `sed` reads no `\r` in its script. - Test: `test_a_git_file_naming_a_clones_git_directory_holds_it` is now parametrized over how the `.git` file spells the path. The new `CRLF` case (rv4-c3's P3) is red at the parent (`remove` for `hold`) and green here, as is the `absolute` case. `git rev-parse --absolute-git-dir` proves the file works in the clone's git directory first. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 5b029c471f8854879a0ded51bec6106fb26197b5) --- lane-end | 4 +++- tests/test_a23_sweep_checkouts.py | 17 +++++++++++++---- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/lane-end b/lane-end index 3e6b746..cdf1f31 100755 --- a/lane-end +++ b/lane-end @@ -1431,7 +1431,9 @@ SH_REFS for sds_e in "$sc_w" "$sc_root"/* "$sc_home"/* "$sc_home"/.[!.]* "$sc_home"/..?*; do [ -d "$sds_e" ] || continue if [ -f "$sds_e/.git" ] && [ ! -L "$sds_e/.git" ]; then - sds_g="$(sed -n -e '1s/^gitdir: //p' -- "$sds_e/.git" 2>/dev/null)" || sds_g="" + # git strips a trailing CR from the line (rv4-c3's NIT c3-3), and so + # does `tr`, first: BSD `sed` reads no `\r` in its script. + sds_g="$(tr -d '\r' 2>/dev/null < "$sds_e/.git" | sed -n -e '1s/^gitdir: //p')" || sds_g="" case "$sds_g" in '') : ;; /*) : ;; *) sds_g="$sds_e/$sds_g" ;; esac if [ -n "$sds_g" ]; then sds_r="$(sc_real "$sds_g")" diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index db1405f..3a2dcae 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -1414,17 +1414,26 @@ def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): box.git("-C", str(borrower), "fsck", "--connectivity-only") -def test_a_git_file_naming_a_clones_git_directory_holds_it(sweep): +GIT_FILES = [ + pytest.param(lambda c, sharer: f"gitdir: {c}/.git\n", id="absolute"), + pytest.param(lambda c, sharer: f"gitdir: {c}/.git\r\n", id="CRLF, whose CR git strips"), +] + + +@pytest.mark.parametrize("spelled", GIT_FILES) +def test_a_git_file_naming_a_clones_git_directory_holds_it(sweep, spelled): """A `.git` FILE IS READ AS AN ALTERNATES FILE IS (rv4's NIT 1, P3): a directory beside W whose `.git` file names the CLOSED lane's `.git` works in that clone's git directory, and removing the clone takes it away. The - clone is held, by the dry run and by `--yes`, naming the directory.""" + clone is held, by the dry run and by `--yes`, naming the directory. The + file is read as git reads it, so a CRLF line works in that directory too + (rv4-c3's NIT c3-3, P3).""" box = sweep.box c = box.checkout(CLOSED) sharer = box.w.parent / "sharer" sharer.mkdir() - (sharer / ".git").write_text(f"gitdir: {c}/.git\n", encoding="utf-8") - assert box.git("-C", str(sharer), "rev-parse", "--git-dir").strip() == f"{c}/.git" + (sharer / ".git").write_bytes(spelled(c, sharer).encode("utf-8")) + assert box.git("-C", str(sharer), "rev-parse", "--absolute-git-dir").strip() == f"{c}/.git" for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): result, rows = sweep(*args, **env) assert rows[str(c)][0] == "hold", rows[str(c)] From c4c8e9b4ec4133f4b77e55f1e4d520f810d1e336 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 08:07:48 +0000 Subject: [PATCH 19/53] Amendment 23 S6.2 tests: the lock's refresh before the re-ask under it, and a `.git` file whose `gitdir:` is relative to the directory it is in, each get a module case, so neither branch can be undone unseen (rv4-c3's NIT c3-4, #198, slice 4) rv4-c3 found two branches of the last fix round with no module case and no mutant: `sc_lock_beat`, which refreshes the lock's age before the candidate's fetch and before the re-ask, and `sc_dep_scan`'s reading of a relative `gitdir:` against the directory its `.git` file is in. Both behave at the head (rv4-c3's P5 and P4 pass there); nothing pinned them. - `test_the_lock_is_refreshed_before_the_reask` (P5): the candidate's own fetch ages its lock twenty minutes (`TZ=UTC0 touch -t`, POSIX, where the probe had GNU's `touch -d`), and a `lanes` wrapper finds it under ten minutes old (`find -mmin -10`, as `mutex_stale` ages it) at the re-ask. The case also checks that the hook did age it. - `test_a_git_file_naming_a_clones_git_directory_holds_it[relative to the directory it is in]` (P4): the third spelling of the `.git` file case. - Each is shown to bite by the mutant that undoes its branch, run once in the mutant clone: `sc_lock_beat`'s body emptied (`sc_lock_beat() { :; }`) and the relative branch dropped. Both KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit e6cb1e8a7926c554c004d2a645ba17da2534312b) --- tests/test_a23_sweep_checkouts.py | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 3a2dcae..7b2e23b 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -26,6 +26,7 @@ import shutil import signal import subprocess +import time from pathlib import Path import pytest @@ -868,6 +869,31 @@ def test_a_checkout_whose_origin_changes_after_the_listing_is_held(sweep): assert c.is_dir() +def test_the_lock_is_refreshed_before_the_reask(sweep): + """THE SWEEP KEEPS ITS LOCK FRESH (rv4's NIT 4; rv4-c3's NIT c3-4, P5). A + writer in another container ages a lock by its directory's mtime alone + (`mutex_stale`'s ten minutes, `find -mmin -10`), so the sweep refreshes + it (`sc_lock_beat`, `touch -c`) before each slow step. Here the lock is + aged twenty minutes during the candidate's own fetch; when the lane's + state is asked again under that lock, it reads under ten minutes old.""" + box = sweep.box + c = box.checkout(CLOSED) + lock = box.lock_of(c) + aged, fresh = box.root / "aged.txt", box.root / "fresh.txt" + stamp = time.strftime("%Y%m%d%H%M.%S", time.gmtime(time.time() - 20 * 60)) + _hook_on_fetch_of(box, c, ( + f"TZ=UTC0 touch -t {stamp} {lock} && " + f'printf "[%s]\\n" "$(find {lock} -maxdepth 0 -mmin +10)" >> {aged}')) + wrapper = _wrapped(box, "lanes-edit-age", "lanes", ( + f' case " $* " in *" --lane {CLOSED} "*) ' + f'printf "[%s]\\n" "$(find {lock} -maxdepth 0 -mmin -10)" >> {fresh} ;; esac\n')) + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=wrapper) + assert aged.read_text(encoding="utf-8") == f"[{lock}]\n", "the hook aged the lock" + assert fresh.read_text(encoding="utf-8") == f"[{lock}]\n", "fresh again at the re-ask" + assert rows[str(c)][0] == "removed", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + + @pytest.mark.parametrize("then,why", [ ("", "its writers' lock lanes/.lanes-edit.lock is no longer this sweep's (1 other-container otherhost)"), ("; exit 1", "its fetch of origin main failed"), @@ -1417,6 +1443,8 @@ def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): GIT_FILES = [ pytest.param(lambda c, sharer: f"gitdir: {c}/.git\n", id="absolute"), pytest.param(lambda c, sharer: f"gitdir: {c}/.git\r\n", id="CRLF, whose CR git strips"), + pytest.param(lambda c, sharer: f"gitdir: {os.path.relpath(c / '.git', sharer)}\n", + id="relative to the directory it is in"), ] @@ -1427,7 +1455,8 @@ def test_a_git_file_naming_a_clones_git_directory_holds_it(sweep, spelled): in that clone's git directory, and removing the clone takes it away. The clone is held, by the dry run and by `--yes`, naming the directory. The file is read as git reads it, so a CRLF line works in that directory too - (rv4-c3's NIT c3-3, P3).""" + (rv4-c3's NIT c3-3, P3), and so does a path relative to the directory + the file is in (NIT c3-4, P4).""" box = sweep.box c = box.checkout(CLOSED) sharer = box.w.parent / "sharer" From 8ca153ee5b8e32c4d13614d18568b75e5041764a Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 09:04:04 +0000 Subject: [PATCH 20/53] lane-end --sweep-checkouts: a former name of the runner's own lane is kept for its own true reason, not "until that lane is CLOSED" (rv4-c4's NIT c4-1, #198, slice 4) rv4-c4 found one thing wrong in the last round's reasons. Where lane repoS4-8 is renamed repoS4-9 and repoS4-9 runs the sweep, the sweep kept repoS4-8's checkout with the former-name prefix of D29 in front of the runner's reason: "... kept until that lane is CLOSED: ...; lane repoS4-9 runs this sweep: its checkout is the one its NOTED line goes through". Both halves are wrong for that checkout. The lane may be CLOSED already, and the NOTED line goes through repoS4-9's own checkout, not this one. `sc_judge_lane`'s runner branch now says, for a former name (its third argument, the new name's checkout beside it): "a former name of lane X, which runs this sweep: a process it started before the rename may still write through this path". That is what the checkout is kept for: a process started before the rename, whose LANES_REPO names the old path (D29). The branch for the runner's own checkout is unchanged. Nothing is kept or removed differently, only the reason said. `test_a_former_name_of_the_runners_own_lane_is_kept_for_its_own_reason` (rv4-c4's P6, as a module case, the dry run and `--yes`): repoS4-9 ENDS and runs the sweep; the former name's row and repoS4-9's own are asserted whole, repoS4-2's is the only checkout swept, and both stay on disk. Red at e6cb1e8 (the "kept until that lane is CLOSED" text), green with the fix. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Sonnet 5.5 (cherry picked from commit 78caf60bedc19501f6af8d7d72a201b8ec17f9dc) --- lane-end | 10 +++++++++- tests/test_a23_sweep_checkouts.py | 24 ++++++++++++++++++++++++ 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/lane-end b/lane-end index cdf1f31..92683aa 100755 --- a/lane-end +++ b/lane-end @@ -1195,7 +1195,15 @@ SC_IDX sjl_cand="a former name of lane $2, beside that lane's own checkout $sjl_new; " fi if [ -n "$sc_runner" ] && [ "$(sc_lc "$2")" = "$(sc_lc "$sc_runner")" ]; then - sc_keep "$1" "${sjl_keep}lane $2 runs this sweep: its checkout is the one its NOTED line goes through" + # A FORMER NAME OF THE RUNNER'S LANE has its own reason (rv4-c4's NIT + # c4-1): the lane may be CLOSED already, and the NOTED line goes through + # the lane's own checkout, not this one. What it is kept for is a process + # started before the rename, which still names the old path. + if [ -n "$sjl_new" ]; then + sc_keep "$1" "a former name of lane $2, which runs this sweep: a process it started before the rename may still write through this path" + else + sc_keep "$1" "lane $2 runs this sweep: its checkout is the one its NOTED line goes through" + fi return 0 fi if ! sc_lookup "$2"; then diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 7b2e23b..d8b3352 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -361,6 +361,30 @@ def test_a_closed_runner_keeps_its_own_checkout(sweep, yes): assert os.path.lexists(other) is not yes +@pytest.mark.parametrize("yes", [False, True], ids=["dry run", "--yes"]) +def test_a_former_name_of_the_runners_own_lane_is_kept_for_its_own_reason(sweep, yes): + """A FORMER NAME OF THE RUNNER'S OWN LANE IS KEPT FOR ITS OWN TRUE REASON + (rv4-c4's NIT c4-1, P6). Lane repoS4-8 is renamed repoS4-9, and repoS4-9 + ENDS and runs the sweep: its former name's checkout is kept, but not + "until that lane is CLOSED", which it is already. What it may still be + written through is a process started before the rename, whose + `LANES_REPO` names the old path; the NOTED line goes through the lane's + own checkout, which is kept by its own reason. Only repoS4-2's, a CLOSED + lane that is not the runner's, is swept.""" + box = sweep.box + old, new = box.checkout(OLD), box.checkout(NEW) + _new_name_ended(box) + result, rows = sweep(*(("--yes",) if yes else ()), LANES_LANE=NEW) + assert rows[str(old)] == ( + "keep", f"a former name of lane {NEW}, which runs this sweep: a process it started " + f"before the rename may still write through this path"), rows[str(old)] + assert rows[str(new)] == ( + "keep", f"lane {NEW} runs this sweep: its checkout is the one its NOTED line goes through"), rows[str(new)] + assert result.returncode == (0 if yes else 3), (result.stdout, result.stderr) + assert _tagged(rows, "removed" if yes else "remove") == {str(box.checkout(CLOSED))}, rows + assert old.is_dir() and new.is_dir() + + def test_a_closed_lanes_checkout_named_by_LANES_REPO_is_kept(sweep): """THE CHECKOUT A CALLER'S `LANES_REPO` NAMES IS NEVER A CANDIDATE, even a CLOSED lane's: a write of that window may go through it.""" From 7541a21103a39fb7753c988c6e9a649feb64c615 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 09:06:28 +0000 Subject: [PATCH 21/53] Amendment 23 S6.2 tests: D30's case also fails the sweep's pid-line write the way a full disk does, so the `rm -f` of the pid file is pinned (rv4-c4's NIT c4-2, #198, slice 4; tests only) rv4-c4 found `sc_lock_take`'s `rm -f` of the pid file unpinned. The module's case for a pid line that cannot be written made the lock directory mode 500, so the open of the pid file failed and no pid file was ever made: nothing for the `rm -f` to remove. On a full disk the open succeeds and leaves an EMPTY pid file, and then the `rmdir` of the lock fails unless the sweep has removed that file, so the lock is left (c3-2's harm). rv4-c4's probes showed it: the module's case passes against a lane-end without the `rm -f` (P8m), and a case with a link to /dev/full as the pid file fails against it, the lock left holding ['pid'] (P7m). `test_a_lock_whose_pid_line_cannot_be_written_is_let_go` takes a second parameter, rv4-c4's P7 (`ln -s /dev/full /pid` after the `mkdir` shim's own `mkdir`): the pid file's write fails with ENOSPC. The first parameter is the case as it was. The case's assertions are the same for both: the checkout is held with the lock-was-let-go reason, the lock is gone, and the next sweep removes the checkout. The new parameter skips where there is no /dev/full; the first skips as root, as it did. The code is right at e6cb1e8, so the new parameter passes there. It bites against the mutant that drops the `rm -f` (`c4-2-rm`): KILLED, the lock left holding the pid file. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Sonnet 5.5 (cherry picked from commit 53b7bf3349dfa94fc01dd52e830a0f94f90364bb) --- tests/test_a23_sweep_checkouts.py | 36 ++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index d8b3352..077a4d2 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -959,28 +959,40 @@ def test_a_lock_taken_over_is_left_by_a_sweep_ended_by_a_signal(sweep): assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" -@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, - reason="root writes in a directory it may not write in") -def test_a_lock_whose_pid_line_cannot_be_written_is_let_go(sweep): +@pytest.mark.parametrize("after", [ + pytest.param("chmod 500 {lock}", id="the lock directory cannot be written in", + marks=pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root writes in a directory it may not write in")), + pytest.param("ln -s /dev/full {lock}/pid", id="the pid file's write fails, as on a full disk", + marks=pytest.mark.skipif(not os.path.exists("/dev/full"), + reason="no /dev/full here to fail a write")), +]) +def test_a_lock_whose_pid_line_cannot_be_written_is_let_go(sweep, after): """THE SWEEP'S PID LINE IS WRITTEN, OR ITS LOCK IS LET GO (rv4-c3's NIT - c3-2, P2). A lock that names no holder is one its writers wait ten - minutes on (`mutex_stale`), and the check before the move read the - sweep's own pid-less lock as taken over and left it there, so every - later sweep held that checkout. Here the sweep's `mkdir` of the CLOSED - lane's lock leaves the new directory read-only (a `mkdir` first on - PATH), so the line cannot be written into it: the checkout is held with - that reason, the lock is gone, and the next sweep removes the checkout.""" + c3-2, P2; rv4-c4's NIT c4-2, P7). A lock that names no holder is one its + writers wait ten minutes on (`mutex_stale`), and the check before the + move read the sweep's own pid-less lock as taken over and left it there, + so every later sweep held that checkout. Here the sweep's `mkdir` of the + CLOSED lane's lock (a `mkdir` first on PATH) leaves a lock its line + cannot be written into, in two ways. The directory is read-only, so the + open fails and no pid file is made. Or a pid file is there whose writes + fail: on a full disk the open succeeds and leaves an EMPTY pid file, and + the `rmdir` takes the lock away only once the sweep's `rm -f` has removed + that file. Either way the checkout is held with that reason, the lock is + gone, and the next sweep removes the checkout.""" box = sweep.box c = box.checkout(CLOSED) lock = box.lock_of(c) shim = box.fakebin / "mkdir" shim.write_text(f'#!/usr/bin/env bash\n{shutil.which("mkdir")} "$@" || exit\n' - f'case " $* " in *" {lock} "*) chmod 500 {lock} ;; esac\n', encoding="utf-8") + f'case " $* " in *" {lock} "*) {after.format(lock=lock)} ;; esac\n', + encoding="utf-8") shim.chmod(0o755) result, rows = sweep("--yes", LANES_LANE=RUNNER) assert rows[str(c)][0] == "hold", rows[str(c)] assert result.returncode == 0, (result.stdout, result.stderr) - assert not os.path.lexists(lock), "a lock that names no holder is not left behind" + assert not os.path.lexists(lock), ( + "a lock that names no holder is not left behind", sorted(os.listdir(lock))) assert ("its writers' lock was taken, and this sweep's pid line could not be written " "into it, so the lock was let go again") in rows[str(c)][1], rows[str(c)] shim.rename(box.root / "mkdir.aside") From 2f5613cd2efdab69cceded4095343d23afc3d679 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 12:19:12 +0000 Subject: [PATCH 22/53] lane-end --sweep-checkouts: a hand edit that `lanes-edit.sh commit` kept in `.git/lanes-edit/` holds the checkout, so the sweep no longer deletes a person's only copy of it (lane openRepoTools-3's MUST 351, #198, slice 4) S3 step 4's carry saves a hand edit of the register to `.git/lanes-edit/hand-edit--.diff`. Where the edit no longer applies to origin's version, `commit` refuses, KEEPS the diff there and leaves the register at origin's version, so `git status` and the untracked list see nothing. `sc_holds` read nothing in the git directory but the operation markers, `index.lock` and `worktrees/`, so lane 3's reproduction ended with the CLOSED checkout and the kept diff both gone, and a NOTED line saying it "held nothing unpublished". - `sc_holds`: anything in `.git/lanes-edit/` holds, but the writer's own scratch: git's words from a write (`apply.err`, `fetch.err`, `ff.err`, `reset.err`) and the lines a carry removed (`removed`). It says "lanes-edit.sh keeps N () in .git/lanes-edit/: a hand edit of the register its commit could not carry, or the copy of one it carried (read it; remove it once origin has the edit)". A `.git/lanes-edit/` that cannot be listed holds as well, with its own reason. - This is lane 3's tested fix, with `reset.err` added to the scratch: the push replay's drop writes it there (`lanes-edit.sh:3033` here, `:3161` at fc6f34d), so every checkout whose write once lost a push race would otherwise be held for good. - Tests (red at 53b7bf3: `remove` for `hold`; green here): `test_a_hand_edit_lanes_edit_kept_holds`, end to end: the row is edited by hand in the CLOSED lane's checkout, a peer changes the same row on origin, and the real `lanes-edit.sh commit` exits 2 and keeps the diff; the dry run and `--yes` both hold the checkout and name the file, and both stay. Its second parameter makes the directory mode 000 (skipped as root). `test_lanes_edit_scratch_alone_holds_nothing` is the control: the five scratch names alone hold nothing, and the checkout is removed. - Not changed: the copy a successful carry leaves is held too, until a person removes it. Lane 3 gives that companion to #307 (delete or rename the copy once the commit is confirmed on origin). Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit e78e9fd81213ff161f40a79b589390e4454f3b2c) --- lane-end | 23 +++++++++- tests/test_a23_sweep_checkouts.py | 70 +++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+), 2 deletions(-) diff --git a/lane-end b/lane-end index 92683aa..5e63218 100755 --- a/lane-end +++ b/lane-end @@ -1322,8 +1322,9 @@ SC_IDX # assume-unchanged or skip-worktree, which `git status` does not look # at; untracked files ANYWHERE, not only under `handoffs/` and # `lanes/`, because no tool can tell a person's work from scratch; an - # operation in progress, unmerged entries, a held `index.lock`, and - # the checkout's own writers' lock. No `origin/` to compare + # operation in progress, unmerged entries, a held `index.lock`, the + # checkout's own writers' lock, and a hand edit of the register that + # `lanes-edit.sh commit` keeps in `.git/lanes-edit/`. No `origin/` to compare # with is itself a hold: nothing can be proved published. IGNORED # files do not hold — the register's own rules keep them out of every # commit — and they are named, so a person sees them first; @@ -1353,6 +1354,24 @@ SC_IDX if [ -e "$sh_gd/$sh_op" ]; then sc_hold "an operation in progress ($sh_op)"; fi done if [ -e "$sh_gd/index.lock" ]; then sc_hold "a git process holds its index.lock"; fi + # A HAND EDIT `lanes-edit.sh commit` KEPT (lane openRepoTools-3's MUST + # 351). S3 step 4's carry saves a hand edit of the register to + # `.git/lanes-edit/hand-edit--.diff`, and where it no longer + # applies it is KEPT there with the register left at origin's version, so + # git status sees nothing: that file may be a person's only copy. Anything + # there holds, but the writer's own scratch: git's words from a write and + # the lines a carry removed, which every write leaves. + if [ -e "$sh_gd/lanes-edit" ] || [ -L "$sh_gd/lanes-edit" ]; then + if ! sh_x="$(ls -A -- "$sh_gd/lanes-edit" 2>/dev/null)"; then + sc_hold "its .git/lanes-edit/, where lanes-edit.sh keeps a hand edit of the register its commit could not carry, cannot be listed, so whether it keeps one is unknown" + else + sh_x="$(printf '%s\n' "$sh_x" | awk 'NF && $0 !~ /^(apply|fetch|ff|reset)\.err$/ && $0 != "removed"')" || + sh_x="(the list could not be read)" + if [ -n "$sh_x" ]; then + sc_hold "lanes-edit.sh keeps $(sc_first "$sh_x") in .git/lanes-edit/: a hand edit of the register its commit could not carry, or the copy of one it carried (read it; remove it once origin has the edit)" + fi + fi + fi sh_x="$(git -C "$sh_p" ls-files -u 2>/dev/null)" || sh_x="unreadable" if [ -n "$sh_x" ]; then sc_hold "unmerged entries"; fi if ! git -C "$sh_p" rev-parse -q --verify "$sh_o^{commit}" >/dev/null 2>&1; then diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 077a4d2..ab89ded 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -633,6 +633,76 @@ def test_no_origin_main_holds_the_dry_run(sweep): _no_aside(box) +def _hand_edit_kept(box: Sweep, c: Path) -> Path: + """S3 step 4's carry, refused (lane openRepoTools-3's MUST 351): the + CLOSED lane's row is edited by hand in its checkout, a peer changes the + same row on origin, and `lanes-edit.sh commit` KEEPS the edit in a file + under `.git/lanes-edit/` and leaves the register at origin's version, so + `git status` and the untracked list see nothing. The kept file's path.""" + reg = c / REGISTER + reg.write_text(reg.read_text(encoding="utf-8").replace( + _row(CLOSED, "ENDED · closed"), _row(CLOSED, "ENDED · closed, and a person's hand note")), + encoding="utf-8") + box.peer_edits(REGISTER, _row(CLOSED, "ENDED · closed"), + _row(CLOSED, "ENDED · closed, and a peer's note"), "a peer's note") + done = box.helper("commit", "a person's hand note", LANES_LANE=CLOSED) + assert done.returncode == 2, (done.stdout, done.stderr) + kept = list((c / ".git" / "lanes-edit").glob("hand-edit-*.diff")) + assert len(kept) == 1, (kept, done.stderr) + assert box.git("-C", str(c), "status", "--porcelain") == "", "git status sees nothing" + return kept[0] + + +@pytest.mark.parametrize("listable", [ + True, + pytest.param(False, marks=pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root lists a directory whose mode is 000")), +], ids=["the kept edit is named", "its directory cannot be listed"]) +def test_a_hand_edit_lanes_edit_kept_holds(sweep, listable): + """A HAND EDIT THAT `commit` KEPT IN `.git/lanes-edit/` HOLDS THE CHECKOUT + (lane openRepoTools-3's MUST 351): it may be a person's only copy of the + edit, and nothing else in the checkout shows it. The dry run and `--yes` + both hold the checkout and name the file, and both stay. A directory + that cannot be listed holds it as well: what it keeps is unknown.""" + box = sweep.box + c = box.checkout(CLOSED) + kept = _hand_edit_kept(box, c) + said = (f"lanes-edit.sh keeps 1 ({kept.name}) in .git/lanes-edit/: a hand edit" if listable + else "its .git/lanes-edit/, where lanes-edit.sh keeps a hand edit of the register " + "its commit could not carry, cannot be listed") + if not listable: + kept.parent.chmod(0) + try: + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert said in rows[str(c)][1], rows[str(c)] + finally: + kept.parent.chmod(0o755) + assert c.is_dir() + assert kept.is_file() + + +def test_lanes_edit_scratch_alone_holds_nothing(sweep): + """`.git/lanes-edit/` ALSO HOLDS `lanes-edit.sh`'S OWN SCRATCH, git's words + from a write (`apply.err`, `fetch.err`, `ff.err`, `reset.err`) and the + lines a carry removed (`removed`), which every write leaves behind. That + alone holds nothing (MUST 351's fix): the checkout is removed.""" + box = sweep.box + c = box.checkout(CLOSED) + scratch = c / ".git" / "lanes-edit" + scratch.mkdir() + for name in ("apply.err", "fetch.err", "ff.err", "removed", "reset.err"): + (scratch / name).write_text("git's own words\n", encoding="utf-8") + _, rows = sweep() + assert rows[str(c)][0] == "remove", rows[str(c)] + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "removed", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert not os.path.lexists(c) + + def test_a_live_cwd_holds(sweep): """S6.2 (c): a live process working inside the checkout holds it. The process is a `sleep` started there, killed by its PID afterwards.""" From d6521bc767457d13d2bb9090f97b8c6f34eb3de7 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 13:29:56 +0000 Subject: [PATCH 23/53] lane-end --sweep-checkouts: a checkout with a file system mounted at or under it is held, and the removal never crosses a mount, so `rm -rf` no longer deletes what a mount shows from outside `-lanes/` (lane openRepoTools-3's SHOULD 371 and 357, #198, slice 4) A mount point at an ignored path, or inside `.git/`, held nothing: ignored files hold nothing, and nothing reads `.git/` for mounts. The `mv` then carried the mount along with its directory, and `rm -rf --` of the aside deleted what the mount showed, which lives outside `-lanes/` (lane 3 showed it with the sweep's own two commands in a privileged container). - `sc_mount_scan` reads the mount table: `/proc/self/mountinfo`'s field 5, or, where there is no such file, `mount`'s output (`X on DIR [type T] (opts)`, Linux's and BSD's). A table that cannot be read, or that names no `/`, is no mount table: the sweep cannot tell, and holds every candidate with "whether a file system is mounted in it cannot be told". - `sc_mounts_inside` spells the checkout's path with mountinfo's octal escapes (backslash, space, TAB) and matches it, through ENVIRON, as the mount point itself or a prefix with `/`. `sc_judge_holds` holds with "a file system is mounted at or under it: N (...), and removing it would remove what that mount shows, which is not this checkout's". - It is read at the listing, with the working directories, and again under each candidate's lock with the dependents and the working directories, just before the move. - A second layer only: `rm -rf --one-file-system` (GNU) or `-x` (BSD), each asked once of a path that is not there; an `rm` that takes neither removes as before. Alone it is not enough: a bind mount of the same file system is one `rm` cannot see. - `LANE_END_MOUNTINFO` names another table: the test seam, in the ENVIRONMENT block. - Tests (red at e78e9fd: `remove`/`removed` for `hold`; green here): `test_a_mount_inside_a_checkout_holds` (at an ignored path, inside `.git`, at a path with a space, which mountinfo spells `\040`), the dry run and `--yes`; `test_a_mount_made_after_the_listing_holds` (the table names the mount only from the checkout's own fetch on: the re-read under the lock holds it); `test_a_mount_table_that_cannot_be_read_holds` (no table and a failing `mount`; a table naming no `/`). - Not run: a real bind mount through `lane-end` end to end. This host has no user namespaces (`unshare -rm`: "Operation not permitted"), so the tests read a table through the seam. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 50a37b7c584ce6576feac02b3f15d94666629308) --- lane-end | 80 +++++++++++++++++++++++++++-- tests/test_a23_sweep_checkouts.py | 85 +++++++++++++++++++++++++++++++ 2 files changed, 160 insertions(+), 5 deletions(-) diff --git a/lane-end b/lane-end index 5e63218..7066033 100755 --- a/lane-end +++ b/lane-end @@ -288,6 +288,9 @@ # gate's sweep # LANES_WORKSPACE_ROOT the workspace repository, overriding workspace.yaml's # `path:` — the test seam +# LANE_END_MOUNTINFO the mount table `--sweep-checkouts` reads, in +# `/proc/self/mountinfo`'s form (default that file) — the test +# seam; one that names no `/` holds every checkout # AGENT_PROTOCOL_ROOT dir holding workspace.yaml (default ~/.agents) # OPENREPOTOOLS_BIN_DIR where `--install` put the commands (default # ~/.local/bin) — searched for lanes-edit.sh @@ -1335,7 +1338,10 @@ SC_IDX # held as "cannot tell". A process whose directory cannot be read # (another user's, or one that ended meanwhile) is counted and said, # and does not hold — `lsof` has the same reach, and (a) is what keeps - # the work. + # the work; + # and a file system mounted at or under it (lane openRepoTools-3's + # SHOULD 371), whose files are not the checkout's, or a mount table + # that cannot be read. SC_HOLD=""; SC_IGN="" sc_hold() { SC_HOLD="${SC_HOLD:+$SC_HOLD; }$1"; } sc_first() { # — the count and the first three, as `N (a b c)` @@ -1545,7 +1551,48 @@ SCC printf '%s' "$scc_out" } - # ALL THREE, for one candidate: SC_HOLD empty means nothing holds it. + # A FILE SYSTEM MOUNTED AT OR UNDER IT (lane openRepoTools-3's SHOULD 371, + # and 357). A mount point at an ignored path, or inside `.git/`, holds + # nothing else, and the `mv` carries a mount along with its directory, so + # `rm -rf` would delete what the mount shows, which is not under + # `-lanes/`. The mount table is read: `/proc/self/mountinfo`'s field 5, + # its octal escapes kept and each path spelled the same way to match it, or + # `mount`'s output where there is no such file. A table that cannot be + # read, or that names no `/`, is no mount table, and with none the sweep + # cannot tell, and holds. `LANE_END_MOUNTINFO` names another table: the + # test seam. + sc_mount_pts=""; sc_mount_mode=none + sc_mount_scan() { + sc_mount_pts=""; sc_mount_mode=none + sms_f="${LANE_END_MOUNTINFO:-/proc/self/mountinfo}" + if [ -r "$sms_f" ] && sms_t="$(awk '{ print $5 }' < "$sms_f" 2>/dev/null)"; then + sms_m=mountinfo + elif sms_t="$(mount 2>/dev/null | awk '{ i = index($0, " on "); if (!i) next; s = substr($0, i + 4); j = 0 + for (k = length(s) - 1; k > 0; k--) if (substr(s, k, 2) == " (") { j = k; break } + if (j) s = substr(s, 1, j - 1); sub(/ type [^ ]*$/, "", s); print s }')"; then + sms_m=mount + else + return 0 + fi + case " +$sms_t +" in + *" +/ +"*) sc_mount_pts="$sms_t"; sc_mount_mode="$sms_m" ;; + esac + } + sc_mounts_inside() { # — the mount points at or under it, one per line + smi_e="$1" + if [ "$sc_mount_mode" = mountinfo ]; then + smi_e="${smi_e//\\/\\134}"; smi_e="${smi_e// /\\040}"; smi_e="${smi_e//$sc_tab/\\011}" + fi + SC_MOUNT_AT="$smi_e" awk '$0 == ENVIRON["SC_MOUNT_AT"] || index($0, ENVIRON["SC_MOUNT_AT"] "/") == 1' < <1 when this run holds its lock> [] sc_holds "$1" "$2" sjh_deps="$(sc_dependents "$1" "${3-}")" @@ -1558,6 +1605,14 @@ SCC sjh_pids="$(sc_cwd_inside "$1")" if [ -n "$sjh_pids" ]; then sc_hold "a live process works in it (pid $sjh_pids)"; fi ;; esac + case "$sc_mount_mode" in + none) sc_hold "whether a file system is mounted in it cannot be told: no mount table that names / could be read (${LANE_END_MOUNTINFO:-/proc/self/mountinfo}, or mount)" ;; + *) + sjh_m="$(sc_mounts_inside "$1")" || sjh_m="(the mount table could not be matched)" + if [ -n "$sjh_m" ]; then + sc_hold "a file system is mounted at or under it: $(sc_first "$sjh_m"), and removing it would remove what that mount shows, which is not this checkout's" + fi ;; + esac return 0 } @@ -1719,7 +1774,7 @@ SS_ROW if [ "$sc_hidden" -gt 0 ]; then printf ' (%s hidden entr(ies) under %s/ - creation locks, partial clones, the sidecar lock - are never looked at)\n' "$sc_hidden" "$sc_home" fi - sc_cwd_scan + sc_cwd_scan; sc_mount_scan if [ "$sc_cwd_unread" -gt 0 ]; then printf ' (%s process(es) whose working directory could not be read - another user'"'"'s, or ended while read - do not hold anything)\n' "$sc_cwd_unread" fi @@ -1771,6 +1826,21 @@ SLU sc_i=$((sc_i + 1)) done + # NEVER ACROSS A MOUNT, AS A SECOND LAYER ONLY (SHOULD 371): the mount + # table above is what holds a checkout with a mount in it, since a bind + # mount of the same file system is one `rm` cannot see. GNU `rm` takes + # `--one-file-system`, BSD's `-x`; each is asked once, of a path that is + # not there, and an `rm` that takes neither removes as before. + sc_rm_x="" + if [ "$sweep_yes" = 1 ]; then + sc_rm_probe="$sc_home/.sweep-checkouts-rm-probe-$$" + if [ ! -e "$sc_rm_probe" ] && [ ! -L "$sc_rm_probe" ]; then + if rm -f --one-file-system -- "$sc_rm_probe" 2>/dev/null; then sc_rm_x=--one-file-system + elif rm -f -x -- "$sc_rm_probe" 2>/dev/null; then sc_rm_x=-x + fi + fi + fi + printf '\n under %s/:\n' "$sc_home" sc_gone=""; sc_n_removed=0; sc_n_partial=0; sc_n_held=0; sc_n_removable=0; sc_partial="" sc_removed_names=""; sc_held_names=""; sc_partial_names="" @@ -1813,7 +1883,7 @@ SLU elif ! sc_is_register_clone "$sc_c"; then sc_act_hold="it is no longer what made it a candidate: $SC_WHY" else - sc_dep_scan; sc_cwd_scan + sc_dep_scan; sc_cwd_scan; sc_mount_scan sc_judge_holds "$sc_c" 1 sc_act_hold="$SC_HOLD"; sc_act_ign="$SC_IGN" fi @@ -1834,7 +1904,7 @@ SLU sc_say_held "$sc_c" "$sc_why" "it could not be moved aside to $sc_aside"; continue fi SC_LOCK="" # it went with the checkout - if sc_under_home "$sc_aside"; then rm -rf -- "$sc_aside" 2>/dev/null || :; fi + if sc_under_home "$sc_aside"; then rm -rf ${sc_rm_x:+"$sc_rm_x"} -- "$sc_aside" 2>/dev/null || :; fi if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then printf ' PARTLY %s %s; moved aside to %s, which could not all be removed%s\n' "$sc_c" "$sc_why" "$sc_aside" "${sc_act_ign:+; $sc_act_ign}" sc_partial="${sc_partial:+$sc_partial }$sc_aside" diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index ab89ded..1827395 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -720,6 +720,91 @@ def test_a_live_cwd_holds(sweep): sleeper.wait() +def _mountinfo_path(path: Path) -> str: + """A path as `/proc/self/mountinfo` spells it: space, TAB, newline and + backslash as octal escapes.""" + return (str(path).replace("\\", "\\134").replace(" ", "\\040") + .replace("\t", "\\011").replace("\n", "\\012")) + + +def _mount_table(box: Sweep, *points: Path, name: str = "mountinfo") -> str: + """A mount table in `/proc/self/mountinfo`'s form, naming `/` and each of + `points`, for `LANE_END_MOUNTINFO`, the test seam: this suite mounts + nothing (a bind mount needs privileges a test host does not give).""" + table = box.root / name + lines = ["21 1 0:20 / / rw,relatime - overlay overlay rw", + "22 21 0:21 / /proc rw,nosuid - proc proc rw"] + lines += [f"{30 + i} 21 0:{40 + i} /outside-{i} {_mountinfo_path(p)} rw,relatime - ext4 /dev/x rw" + for i, p in enumerate(points)] + table.write_text("\n".join(lines) + "\n", encoding="utf-8") + return str(table) + + +@pytest.mark.parametrize("where", ["ignored-mnt", ".git/mnt", "ignored mnt"], + ids=["at an ignored path", "inside .git", "at a path with a space"]) +def test_a_mount_inside_a_checkout_holds(sweep, where): + """A FILE SYSTEM MOUNTED AT OR UNDER A CHECKOUT HOLDS IT (lane + openRepoTools-3's SHOULD 371, and 357). A mount point at an ignored path, + or inside `.git/`, holds nothing else, and the `mv` carries the mount + along with its directory, so `rm -rf` would delete what the mount shows, + which is not under `-lanes/`. Here the mount table names one there: + the dry run and `--yes` both hold the checkout, naming the mount point.""" + box = sweep.box + c = box.checkout(CLOSED) + (c / ".git" / "info" / "exclude").write_text("ignored-mnt/\nignored mnt/\n", encoding="utf-8") + box.write(c, f"{where}/shown.txt", "what the mount shows, which lives elsewhere\n") + table = _mount_table(box, c / where) + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, LANE_END_MOUNTINFO=table, **env) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (f"a file system is mounted at or under it: 1 ({_mountinfo_path(c / where)})" + in rows[str(c)][1]), rows[str(c)] + assert (c / where / "shown.txt").is_file() + + +def test_a_mount_made_after_the_listing_holds(sweep): + """THE MOUNT TABLE IS READ AGAIN UNDER THE LOCK (SHOULD 371: "under the + lock, just before the mv"): the table names nothing in the checkout at + the listing, and a mount point inside its `.git/` at its own fetch. It + is held.""" + box = sweep.box + c = box.checkout(CLOSED) + box.write(c, ".git/mnt/shown.txt", "what the mount shows\n") + table = _mount_table(box) + later = _mount_table(box, c / ".git" / "mnt", name="mountinfo.later") + _, rows = sweep(LANE_END_MOUNTINFO=table) + assert rows[str(c)][0] == "remove", rows[str(c)] + _hook_on_fetch_of(box, c, f"cp -- {later} {table}") + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANE_END_MOUNTINFO=table) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert f"a file system is mounted at or under it: 1 ({c}/.git/mnt)" in rows[str(c)][1], rows[str(c)] + assert (c / ".git" / "mnt" / "shown.txt").is_file() + + +@pytest.mark.parametrize("table", ["no table", "a table that names no /"]) +def test_a_mount_table_that_cannot_be_read_holds(sweep, table): + """WHERE NO MOUNT TABLE CAN BE READ, WHETHER A FILE SYSTEM IS MOUNTED IN A + CHECKOUT CANNOT BE TOLD, AND IT IS HELD (SHOULD 371): `LANE_END_MOUNTINFO` + names no file and `mount` fails, or names a table without `/`, which is + no host's mount table. The dry run and `--yes` both hold it.""" + box = sweep.box + c = box.checkout(CLOSED) + path = box.root / "mountinfo" + if table != "no table": + path.write_text("30 21 0:40 /x /elsewhere rw,relatime - ext4 /dev/x rw\n", encoding="utf-8") + shim = box.fakebin / "mount" + shim.write_text("#!/usr/bin/env bash\nexit 1\n", encoding="utf-8") + shim.chmod(0o755) + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, LANE_END_MOUNTINFO=str(path), **env) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert "whether a file system is mounted in it cannot be told" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + + def test_a_cwd_that_appears_after_the_listing_holds(sweep): """S6.2 (c) IS READ AGAIN UNDER THE LOCK, AFTER THE FETCH (rv4's SHOULD 1, R1, P4): a process that starts working in the checkout after the listing, From b097a0a8a82401a85f809b0fb68c76a72738685b Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 13:37:19 +0000 Subject: [PATCH 24/53] Amendment 23 S6.2 tests: a checkout whose own path holds a space is matched against the mount table as `/proc/self/mountinfo` spells it, so 371's escaping is pinned (lane openRepoTools-3's SHOULD 371, #198, slice 4; tests only) The escape mutant (the checkout's path matched unescaped) survived 371's cases: in each, the space was in the MOUNT POINT under the checkout, and the checkout's own path, the prefix that is matched, held none. - `test_a_mount_in_a_checkout_whose_own_path_has_a_space_holds`: a box whose W stands in `with space/`, and a mount table naming a mount point at an ignored path in the CLOSED lane's checkout, as mountinfo spells it (`with\040space`). The dry run holds the checkout and names the mount point. With the escaping undone it is `remove`. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 596aeeeb32732fb274d5fdd855d8c95061c7e4a7) --- tests/test_a23_sweep_checkouts.py | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 1827395..c5efc51 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -763,6 +763,32 @@ def test_a_mount_inside_a_checkout_holds(sweep, where): assert (c / where / "shown.txt").is_file() +def test_a_mount_in_a_checkout_whose_own_path_has_a_space_holds(tmp_path): + """THE CHECKOUT'S OWN PATH IS SPELLED AS THE MOUNT TABLE SPELLS IT (SHOULD + 371): `/proc/self/mountinfo` writes a space in a mount point as `\\040`, + so a checkout whose own path holds one (here W's parent, `with space/`) + matches its mount points only once that path is escaped the same way. + The dry run holds it, naming the mount point as the table spells it.""" + root = Path(os.path.realpath(tmp_path)) / "with space" + box = Sweep(root) + try: + assert " " in str(box.lanes_root) + c = box.checkout(CLOSED) + (c / ".git" / "info" / "exclude").write_text("ignored-mnt/\n", encoding="utf-8") + box.write(c, "ignored-mnt/shown.txt", "what the mount shows\n") + table = _mount_table(box, c / "ignored-mnt") + result = subprocess.run(["bash", str(REPO / "lane-end"), "--sweep-checkouts"], + env=dict(box.env, LANE_END_MOUNTINFO=table), cwd=str(box.root), + capture_output=True, text=True, timeout=300, check=False) + rows = _rows(result.stdout) + assert rows[str(c)][0] == "hold", (rows.get(str(c)), result.stdout, result.stderr) + assert (f"a file system is mounted at or under it: 1 ({_mountinfo_path(c / 'ignored-mnt')})" + in rows[str(c)][1]), rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + finally: + box.close() + + def test_a_mount_made_after_the_listing_holds(sweep): """THE MOUNT TABLE IS READ AGAIN UNDER THE LOCK (SHOULD 371: "under the lock, just before the mv"): the table names nothing in the checkout at From d22503da86c848e61faec65b007882bbba876224 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 15:32:20 +0000 Subject: [PATCH 25/53] lane-end --sweep-checkouts: an interrupted `--yes` lets the removal in hand finish and prints its NOTED line by hand, and every run names an unfinished removal's aside with the command that removes it (lane openRepoTools-3's SHOULD 372, #198, slice 4) - INT and TERM wait while a checkout is moved aside, removed and counted: the `mv` and the `rm` run in subshells that ignore both (`exec` keeps an ignored signal ignored), so a Ctrl-C to the process group no longer stops `rm` halfway and leaves a whole checkout under a hidden name. - A signal then ends the act. After any removal it prints the NOTED line by hand for what was done, ending "the sweep was stopped by a signal". One that comes once the NOTED write has started says the line may be on origin already and hands out the publish act first. - Every run lists each `..removing-*` under `-lanes/` as a `leftover` line, apart from the hidden count, with `chmod -R u+rwX --

&& rm -rf --

`. No sweep removes one. - A removal that did not finish is said before the NOTED write, so a refused write no longer exits before saying it. Tests: five new cases. All five fail at 596aeee and pass with the fix, with their neighbours (11 passed). Hygiene: 253 passed. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit ff12b0154850bf0b0bf823e29f1100fa813b945d) --- lane-end | 130 ++++++++++++++++++++++----- tests/test_a23_sweep_checkouts.py | 144 ++++++++++++++++++++++++++++++ 2 files changed, 251 insertions(+), 23 deletions(-) diff --git a/lane-end b/lane-end index 7066033..d93e0e1 100755 --- a/lane-end +++ b/lane-end @@ -279,7 +279,9 @@ # recorded, or none of its rows is DORMANT. `--retire `: nothing # of any kind is live for the lane # 130, 143 stopped by INT or TERM (128 plus the signal): the run ends where -# the signal found it and removes its temporary file, never resumes +# the signal found it and removes its temporary file, never resumes. +# `--sweep-checkouts --yes` lets a checkout's move and removal in hand +# finish first, and after any removal prints its NOTED line by hand # # ENVIRONMENT # LANES_FILE default /lanes/LANES.md @@ -1274,7 +1276,16 @@ SC_IDX # `-lanes/` ITSELF must be a directory at its own real path: a link there # would lead the sweep out of it, and nothing in it is then looked at. - sc_home_why=""; sc_hidden=0 + # + # A REMOVAL THAT DID NOT FINISH IS NAMED AGAIN BY EVERY RUN (lane + # openRepoTools-3's SHOULD 372). Its aside, `..removing--`, + # is hidden, so a later run would count it with the creation locks and say + # no more: a checkout moved aside by a sweep that was stopped, or whose + # `rm` met a file it could not remove, would be named nowhere again. Each + # one is listed by name, apart from the hidden count, with the command that + # removes it. The sweep never removes one itself: a person looks at why + # first, and one may be another sweep's, still running. + sc_home_why=""; sc_hidden=0; sc_leftovers=() if [ -L "$sc_home" ]; then sc_home_why="$sc_home is a symbolic link, which this sweep never follows: nothing in it is looked at" elif [ ! -e "$sc_home" ]; then @@ -1287,7 +1298,11 @@ SC_IDX die "$sc_home cannot be listed, so what is in it is unknown — which is not an empty directory. Nothing was removed." 1 else for sc_e in "$sc_home"/.[!.]* "$sc_home"/..?*; do - if [ -e "$sc_e" ] || [ -L "$sc_e" ]; then sc_hidden=$((sc_hidden + 1)); fi + if [ ! -e "$sc_e" ] && [ ! -L "$sc_e" ]; then continue; fi + case "${sc_e##*/}" in + .?*.removing-*) sc_leftovers+=("$sc_e") ;; + *) sc_hidden=$((sc_hidden + 1)) ;; + esac done for sc_e in "$sc_home"/*; do if [ ! -e "$sc_e" ] && [ ! -L "$sc_e" ]; then continue; fi @@ -1848,6 +1863,68 @@ SLU printf ' hold %s %s; holds: %s\n' "$1" "$2" "$3" sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${1##*/}" } + # THE NOTED LINE'S TEXT, from what this run has done so far, and the two + # commands that finish its record by hand. The paths are spelled from the + # projects root, `-lanes/`, never from the host's own root, and + # every name in the line is one this sweep already holds to letters, + # digits, `.`, `_` and `-`, so the free text's own rules (no ` — `, no + # newline, no leading arrow) hold for it. + sc_rel="${sc_home##*/}" + case "$sc_rel" in '' | *[!A-Za-z0-9._-]*) sc_rel="-lanes" ;; esac + sc_noted_text() { # [] + snt_t="sweep-checkouts (Amendment 23 S6.2): removed $sc_n_removed register checkout(s) under $sc_rel/ that held nothing unpublished" + if [ -n "$sc_removed_names" ]; then snt_t="$snt_t: $sc_removed_names"; fi + if [ -n "$sc_partial_names" ]; then snt_t="$snt_t; moved aside and not all removed: $sc_partial_names"; fi + if [ -n "$sc_held_names" ]; then snt_t="$snt_t; held and left: $sc_held_names"; fi + snt_t="$snt_t; ${#sc_beside[@]} clone(s) beside W listed and left" + if [ -n "${1-}" ]; then snt_t="$snt_t; $1"; fi + printf '%s' "$snt_t" + } + sc_by_hand_of() { # + printf 'LANES_LANE=%q LANES_SESSION=%s lanes-edit.sh log NOTED %q %q' \ + "$sc_runner" "$sc_uuid" "lane:$sc_runner" "$1" + } + sc_publish="LANES_LANE=$(printf '%q' "$sc_runner") lanes-edit.sh commit $(printf '%q' "the NOTED line of lane-end --sweep-checkouts")" + sc_partial_said=0 + sc_say_partial() { + if [ -n "$sc_partial" ] && [ "$sc_partial_said" = 0 ]; then + sc_partial_said=1 + note "the removal of $sc_partial did not finish (above): what is left is no checkout, under a hidden name, so the next write of that lane clones afresh. Remove it by hand once you have looked at why; every later run names it again, with the command." + fi + } + # A SIGNAL NEVER CUTS A REMOVAL IN TWO, AND ONE THAT ENDS THE ACT LEAVES + # ITS RECORD (lane openRepoTools-3's SHOULD 372). INT and TERM wait while a + # checkout is moved aside, removed and counted: the `mv` and the `rm` run + # in subshells that ignore both, and an ignored signal stays ignored across + # `exec`, so a Ctrl-C to the process group does not stop them halfway, and + # this shell takes the signal once they are done. A signal ends the act + # then, or at once outside those steps, and after any removal it prints + # the NOTED line by hand for what was done. One that comes while the line + # is being written says the line may be on origin already, and hands out + # the publish act first, so the sweep is never recorded twice. + sc_sig=""; sc_noting=0; sc_text="" + sc_on_signal() { # + trap '' INT TERM + if [ "$sc_noting" = 1 ]; then + note "a signal stopped this sweep while its NOTED line was being written, so the line may be on origin already, or committed in the checkout it went through and not yet published. Nothing was lost: each checkout removed held nothing unpublished. Look at lane $sc_runner's log on origin first. Where origin lacks the line, the publish act sends a commit that is kept:" + note " $sc_publish" + note "Only where origin lacks the line and the publish act finds nothing to send is it written by hand:" + note " $(sc_by_hand_of "$sc_text")" + elif [ $((sc_n_removed + sc_n_partial)) -gt 0 ]; then + note "a signal stopped this sweep after the removals above, before its NOTED line was written, so lane $sc_runner's log does not say what was removed. Nothing was lost: each checkout removed held nothing unpublished. Write the line by hand:" + note " $(sc_by_hand_of "$(sc_noted_text "the sweep was stopped by a signal")")" + else + note "a signal stopped this sweep. Nothing was removed." + fi + sc_say_partial + exit "$1" + } + sc_sig_defer() { sc_sig=""; trap 'sc_sig=130' INT; trap 'sc_sig=143' TERM; } + sc_sig_arm() { + trap 'sc_on_signal 130' INT; trap 'sc_on_signal 143' TERM + if [ -n "$sc_sig" ]; then sc_on_signal "$sc_sig"; fi + } + if [ "$sweep_yes" = 1 ]; then sc_sig_arm; fi for sc_i in ${sc_order[@]+"${sc_order[@]}"}; do sc_c="${sc_cands[$sc_i]}"; sc_why="${sc_cand_why[$sc_i]}" sc_l="${sc_cand_lane[$sc_i]}"; sc_new="${sc_cand_new[$sc_i]}" @@ -1899,12 +1976,17 @@ SLU sc_say_held "$sc_c" "$sc_why" "$sc_act_hold"; continue fi sc_aside="$sc_home/.${sc_c##*/}.removing-$sc_utc-$$" - if ! sc_under_home "$sc_c" || [ -e "$sc_aside" ] || [ -L "$sc_aside" ] || ! mv -- "$sc_c" "$sc_aside" 2>/dev/null; then + sc_sig_defer + if ! sc_under_home "$sc_c" || [ -e "$sc_aside" ] || [ -L "$sc_aside" ] || + ! ( trap '' INT TERM; exec mv -- "$sc_c" "$sc_aside" ) 2>/dev/null; then sc_lock_drop - sc_say_held "$sc_c" "$sc_why" "it could not be moved aside to $sc_aside"; continue + sc_say_held "$sc_c" "$sc_why" "it could not be moved aside to $sc_aside" + sc_sig_arm; continue fi SC_LOCK="" # it went with the checkout - if sc_under_home "$sc_aside"; then rm -rf ${sc_rm_x:+"$sc_rm_x"} -- "$sc_aside" 2>/dev/null || :; fi + if sc_under_home "$sc_aside"; then + ( trap '' INT TERM; exec rm -rf ${sc_rm_x:+"$sc_rm_x"} -- "$sc_aside" ) 2>/dev/null || : + fi if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then printf ' PARTLY %s %s; moved aside to %s, which could not all be removed%s\n' "$sc_c" "$sc_why" "$sc_aside" "${sc_act_ign:+; $sc_act_ign}" sc_partial="${sc_partial:+$sc_partial }$sc_aside" @@ -1916,9 +1998,14 @@ SLU sc_n_removed=$((sc_n_removed + 1)) fi sc_gone="${sc_gone:+$sc_gone }$sc_c" + sc_sig_arm done printf '%s' "$sc_keep_lines" - if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ]; then printf ' (nothing)\n'; fi + for sc_e in ${sc_leftovers[@]+"${sc_leftovers[@]}"}; do + printf ' leftover %s the aside of a removal that did not finish (a sweep stopped by a signal, or an rm that met what it could not remove; or one still running): no checkout, and no write goes through it. Look at why, then remove it by hand: chmod -R u+rwX -- %q && rm -rf -- %q\n' \ + "$sc_e" "$sc_e" "$sc_e" + done + if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ] && [ "${#sc_leftovers[@]}" = 0 ]; then printf ' (nothing)\n'; fi printf '\n beside W (Amendment 23 S6.2), listed and never removed by this build:\n' for sc_e in ${sc_beside[@]+"${sc_beside[@]}"}; do @@ -1944,6 +2031,9 @@ SLU exit 3 fi printf '\n DRY RUN — nothing was removed, and there is nothing to remove.\n' + if [ "${#sc_leftovers[@]}" -gt 0 ]; then + printf ' (%s leftover aside(s), above, are a person'"'"'s to remove: no sweep removes one)\n' "${#sc_leftovers[@]}" + fi exit 0 fi printf '\n %s removed, %s moved aside and not all removed, %s held under %s/; %s clone(s) beside W left.\n' \ @@ -1954,19 +2044,15 @@ SLU # and S4. It is written with `LANES_NO_FETCH` UNSET, because S3 refuses a # writer whose fetch did not run (ratified choice 2; rv1b's MUST 3 on # `lane-worktrees`). Nothing removed is nothing to record, and nothing is - # written. The paths are spelled from the projects root, `-lanes/`, - # never from the host's own root, and every name in the line is one this - # sweep already holds to letters, digits, `.`, `_` and `-`, so the free - # text's own rules (no ` — `, no newline, no leading arrow) hold for it. + # written (`sc_noted_text`, above, says how the line is spelled). A removal + # that did not finish is said FIRST, before the write, so no exit of the + # write can leave it unsaid (SHOULD 372). From the moment the write starts, + # a signal says the line may be on origin already (`sc_noting`). + sc_say_partial if [ $((sc_n_removed + sc_n_partial)) -gt 0 ]; then - sc_rel="${sc_home##*/}" - case "$sc_rel" in '' | *[!A-Za-z0-9._-]*) sc_rel="-lanes" ;; esac - sc_text="sweep-checkouts (Amendment 23 S6.2): removed $sc_n_removed register checkout(s) under $sc_rel/ that held nothing unpublished" - if [ -n "$sc_removed_names" ]; then sc_text="$sc_text: $sc_removed_names"; fi - if [ -n "$sc_partial_names" ]; then sc_text="$sc_text; moved aside and not all removed: $sc_partial_names"; fi - if [ -n "$sc_held_names" ]; then sc_text="$sc_text; held and left: $sc_held_names"; fi - sc_text="$sc_text; ${#sc_beside[@]} clone(s) beside W listed and left" + sc_text="$(sc_noted_text)" sc_noted_rc=0 + sc_noting=1 ( unset LANES_NO_FETCH LANES_LANE="$sc_runner" LANES_SESSION="$sc_uuid" "$LANES_EDIT" log NOTED "lane:$sc_runner" "$sc_text" ) >&2 || sc_noted_rc=$? @@ -1982,8 +2068,7 @@ SLU # nothing, so it is safe to run whichever happened. Only a 6 whose publish # act finds nothing to send, and every other refusal, gets the line by hand. if [ "$sc_noted_rc" != 0 ]; then - sc_by_hand="LANES_LANE=$(printf '%q' "$sc_runner") LANES_SESSION=$sc_uuid lanes-edit.sh log NOTED $(printf '%q' "lane:$sc_runner") $(printf '%q' "$sc_text")" - sc_publish="LANES_LANE=$(printf '%q' "$sc_runner") lanes-edit.sh commit $(printf '%q' "the NOTED line of lane-end --sweep-checkouts")" + sc_by_hand="$(sc_by_hand_of "$sc_text")" case "$sc_noted_rc" in 3) note "the NOTED line recording this sweep was committed and pushed, and the read-back could not confirm it on origin (\`lanes-edit.sh log\` exited 3, above), so it may be on origin already, and it is not to be written again. Nothing was lost: each checkout removed held nothing unpublished. The publish act sends its commit only where origin lacks it by content, and says so where origin has it:" @@ -2000,9 +2085,8 @@ SLU fi printf ' NOTED in lane %s'"'"'s log: %s\n' "$sc_runner" "$sc_text" fi - if [ -n "$sc_partial" ]; then - die "the removal of $sc_partial did not finish (above): what is left is no checkout, under a hidden name, so the next write of that lane clones afresh. Remove it by hand once you have looked at why." 1 - fi + # A removal that did not finish was said before the write (above). + if [ -n "$sc_partial" ]; then exit 1; fi exit 0 fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index c5efc51..9ba2838 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -1467,6 +1467,150 @@ def test_a_removal_that_does_not_finish_exits_1(sweep): (c / "scratch").chmod(0o755) +# ------------------------------------------- a stopped act, and its leftovers + +def _run_own_group(box: Sweep, *args: str, **env: str): + """`lane-end --sweep-checkouts` in a process group of its own, so a + signal sent to the group, as a terminal's Ctrl-C is, reaches the sweep and + its children and nothing of the test's.""" + result = subprocess.run(["bash", str(REPO / "lane-end"), "--sweep-checkouts", *args], + env=dict(box.env, **env), cwd=str(box.root), capture_output=True, + text=True, timeout=300, check=False, start_new_session=True) + return result, _rows(result.stdout) + + +def _rm_shim(box: Sweep, lane: str, action: str) -> None: + """An `rm` first on PATH: for the aside of `lane`'s checkout it runs + `action` (a signal) first; every `rm` then runs the real one.""" + real = shutil.which("rm") + assert real, "no rm on the test's own PATH" + shim = box.fakebin / "rm" + shim.write_text("#!/usr/bin/env bash\n" + f'case "$*" in *"/.{lane}.removing-"*) {action} ;; esac\n' + f'exec {real} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + + +@pytest.mark.parametrize("send,code", [ + pytest.param("kill -INT 0", 130, id="INT to its process group, as a Ctrl-C"), + pytest.param("kill -TERM $PPID", 143, id="TERM to the sweep alone"), +]) +def test_a_signal_during_a_removal_lets_it_finish_and_prints_the_record(sweep, send, code): + """AN INTERRUPTED `--yes` LEAVES ITS RECORD, AND NO CHECKOUT HALF REMOVED + (lane openRepoTools-3's SHOULD 372). Two CLOSED lanes' checkouts are + removed, and the signal comes as the second one's `rm` starts. The `rm` + finishes, so no aside is left, and the sweep then exits with the + signal's code and the NOTED line by hand for both removals; no line is + written. Run as printed, it lands once.""" + box = sweep.box + c, other = box.checkout(CLOSED), _closed_checkout(box) + _rm_shim(box, other.name, send) + result, rows = _run_own_group(box, "--yes", LANES_LANE=RUNNER) + assert result.returncode == code, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows + assert rows[str(other)][0] == "removed", rows + assert not os.path.lexists(c) + assert not os.path.lexists(other) + _no_aside(box) + assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "no lock is left" + assert not _noted_on_origin(box) + assert "a signal stopped this sweep after the removals above" in result.stderr, result.stderr + remedies = _remedies(result.stderr) + assert len(remedies) == 1, remedies + assert remedies[0].startswith( + f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), remedies + done = _by_hand(box, remedies[0]) + assert done.returncode == 0, (done.stdout, done.stderr) + noted = _noted_on_origin(box) + assert len(noted) == 1, noted + assert (f"removed 2 register checkout(s) under brett-wip-lanes/ that held nothing " + f"unpublished: {CLOSED} {other.name};") in noted[0], noted + assert noted[0].endswith("; the sweep was stopped by a signal"), noted + + +def test_a_signal_while_the_noted_line_is_written_never_writes_it_twice(sweep): + """A SIGNAL DURING THE NOTED WRITE MAY COME AFTER THE LINE LANDED (SHOULD + 372): here the write finishes and the sweep is then sent TERM. It exits + 143, says the line may be on origin already, and hands out the publish + act first, which finds nothing to send: origin holds the line once.""" + box = sweep.box + stopping = _wrapped(box, "lanes-edit-stopped-log", "log", ( + ' bash "$REAL" "$@"; rc=$?\n' + " trap '' TERM; kill -TERM 0\n" + ' exit "$rc"\n')) + result, rows = _run_own_group(box, "--yes", LANES_LANE=RUNNER, LANES_EDIT=stopping) + assert result.returncode == 143, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows + assert len(_noted_on_origin(box)) == 1 + assert "the line may be on origin already" in result.stderr, result.stderr + remedies = _remedies(result.stderr) + assert len(remedies) == 2, remedies + assert remedies[0].startswith(f"LANES_LANE={RUNNER} lanes-edit.sh commit "), remedies + done = _by_hand(box, remedies[0]) + assert done.returncode == 0, (done.stdout, done.stderr) + assert len(_noted_on_origin(box)) == 1, "the publish act wrote no second line" + + +def test_a_removal_left_unfinished_is_named_by_every_run(sweep, tmp_path): + """AN ASIDE A REMOVAL LEFT IS NAMED AGAIN, BY NAME, IN EVERY RUN (SHOULD + 372), apart from the hidden entries it would otherwise be counted with, + and with the command that removes it; no sweep removes it. The command, + run as printed, removes it, a read-only directory inside included.""" + box = sweep.box + aside = box.lanes_root / f".{CLOSED}.removing-20261020T100000Z-1" + box.write(aside, "scratch/kept.txt", "what an earlier sweep could not remove\n") + (aside / ".git").mkdir() + (aside / "scratch").chmod(0o555) + try: + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(aside)][0] == "leftover", rows + assert "(1 hidden entr(ies) under" in result.stdout, "the creation lock alone" + assert aside.is_dir() + command = rows[str(aside)][1].split("remove it by hand: ", 1)[1] + assert command == f"chmod -R u+rwX -- {aside} && rm -rf -- {aside}", command + done = subprocess.run(["bash", "-c", command], cwd=str(box.root), capture_output=True, + text=True, env=box.env, timeout=60, check=False) + assert done.returncode == 0, (done.stdout, done.stderr) + _assert_under(Path(os.path.realpath(tmp_path)), [aside]) + assert not os.path.lexists(aside) + finally: + if (aside / "scratch").is_dir(): + (aside / "scratch").chmod(0o755) + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root removes what a read-only directory holds") +def test_a_removal_that_does_not_finish_is_said_when_the_noted_write_fails(sweep): + """A REMOVAL THAT DID NOT FINISH IS SAID BEFORE THE NOTED WRITE (SHOULD + 372), so a write that is refused, and exits at once, cannot leave it + unsaid: the PARTLY aside is named, the run exits 1, and the by-hand line + names it too.""" + box = sweep.box + c = box.checkout(CLOSED) + (c / ".git" / "info" / "exclude").write_text("scratch/\n", encoding="utf-8") + box.write(c, "scratch/kept.txt", "an ignored file in a read-only directory\n") + (c / "scratch").chmod(0o555) + refusing = _wrapped(box, "lanes-edit-refusing-log", "log", ( + " printf 'FAKE: the register refused this line\\n' >&2\n" + " exit 2\n")) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=refusing) + asides = list(box.lanes_root.glob(f".{CLOSED}.removing-*")) + assert rows[str(c)][0] == "PARTLY", rows[str(c)] + assert result.returncode == 1, (result.stdout, result.stderr) + assert len(asides) == 1, asides + assert f"the removal of {asides[0]} did not finish" in result.stderr, result.stderr + assert "was NOT written" in result.stderr, result.stderr + assert not _noted_on_origin(box) + finally: + for aside in box.lanes_root.glob(f".{CLOSED}.removing-*"): + (aside / "scratch").chmod(0o755) + if (c / "scratch").is_dir(): + (c / "scratch").chmod(0o755) + + # ----------------------------------------------------------- the NOTED line def test_noted_line_written_through_runner_checkout(sweep): From b2fee5e4ba9f6d2a12407819af69d2240523d042 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 17:41:01 +0000 Subject: [PATCH 26/53] lane-end --sweep-checkouts: a NOTED write's exit 6 is finished by what its own cause needs (lane openRepoTools-3's SHOULD 373) Exit 6 of `lanes-edit.sh log` has three causes, and the two remedies the sweep printed for every 6 (the publish act, then the line by hand) both refused for two of them. The write's words are now kept (captured inside the write's own subshell, so a signal's trap still speaks on the sweep's stderr) as well as shown, and choose the remedy: - a push refused with origin unmoved: the line is committed and only origin lacks it, so the publish act alone; - `git add failed` / `git commit failed`: the line is staged or in the tree, so `git -C restore --staged --worktree -- lanes/log/.md`, then the line by hand; - the bound ("the commit is local and kept on HEAD", or no commit to drop to): S6.3's archive, `git -C push origin HEAD:refs/heads/archive/-stranded-`, then `reset --keep refs/remotes/origin/`, then the line by hand; - words the sweep does not know: the two remedies as before. The checkout is the one `lanes-edit.sh` writes through (S2): the caller's LANES_REPO, the runner's own under LANES_ROUTE_WRITES=1, else W. Tests: a pre-commit hook in the runner's checkout, and a peer commit landing before each of the six pushes; each case runs the printed commands in order and origin holds the line once (the archive holds the stranded commit). The origin-unmoved case now expects the publish act alone. Red at ff12b01: 3 failed; green: 12 passed with the NOTED, signal and partial neighbours. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 3373d5be2b3599e6cec93a8d1a3d054ee1e98c8a) --- lane-end | 76 +++++++++++++++++++++---- tests/test_a23_sweep_checkouts.py | 95 +++++++++++++++++++++++++++---- 2 files changed, 150 insertions(+), 21 deletions(-) diff --git a/lane-end b/lane-end index d93e0e1..4951572 100755 --- a/lane-end +++ b/lane-end @@ -1885,6 +1885,20 @@ SLU "$sc_runner" "$sc_uuid" "lane:$sc_runner" "$1" } sc_publish="LANES_LANE=$(printf '%q' "$sc_runner") lanes-edit.sh commit $(printf '%q' "the NOTED line of lane-end --sweep-checkouts")" + # THE CHECKOUT THE LINE GOES THROUGH, as `lanes-edit.sh` picks it (S2): the + # caller's `LANES_REPO`; under `LANES_ROUTE_WRITES=1`, the runner's own + # checkout; and W without either. A refused write is finished there. + if [ -n "$LE_REPO_GIVEN" ]; then sc_wco="${sc_xrepo:-$LE_REPO_GIVEN}" + elif [ "${LANES_ROUTE_WRITES:-}" = 1 ]; then sc_wco="$sc_home/$sc_runner" + else sc_wco="$sc_w"; fi + # The write, its words on stderr, or in a file: sent there inside its own + # subshell, so a signal's trap, run in this shell once the write is done, + # still speaks on this shell's own stderr. + sc_log_noted() { # [] + ( unset LANES_NO_FETCH + if [ -n "${1-}" ]; then exec 2>"$1" || exit 1; fi + LANES_LANE="$sc_runner" LANES_SESSION="$sc_uuid" "$LANES_EDIT" log NOTED "lane:$sc_runner" "$sc_text" >&2 ) + } sc_partial_said=0 sc_say_partial() { if [ -n "$sc_partial" ] && [ "$sc_partial_said" = 0 ]; then @@ -1906,6 +1920,7 @@ SLU sc_on_signal() { # trap '' INT TERM if [ "$sc_noting" = 1 ]; then + if [ -n "${LE_TMP:-}" ]; then cat -- "$LE_TMP" >&2 2>/dev/null || :; fi note "a signal stopped this sweep while its NOTED line was being written, so the line may be on origin already, or committed in the checkout it went through and not yet published. Nothing was lost: each checkout removed held nothing unpublished. Look at lane $sc_runner's log on origin first. Where origin lacks the line, the publish act sends a commit that is kept:" note " $sc_publish" note "Only where origin lacks the line and the publish act finds nothing to send is it written by hand:" @@ -2052,21 +2067,32 @@ SLU if [ $((sc_n_removed + sc_n_partial)) -gt 0 ]; then sc_text="$(sc_noted_text)" sc_noted_rc=0 + # WHAT THE WRITE SAYS IS KEPT, AS WELL AS SHOWN (lane openRepoTools-3's + # SHOULD 373): its exit 6 has three causes, and each is finished by other + # commands, so its own words choose them. A capture that cannot be made + # leaves its words on stderr only, and the remedies for an unknown cause. + sc_log_err="" + LE_TMP="$(mktemp "${TMPDIR:-/tmp}/lane-end-sweep.XXXXXX" 2>/dev/null || printf '')" sc_noting=1 - ( unset LANES_NO_FETCH - LANES_LANE="$sc_runner" LANES_SESSION="$sc_uuid" "$LANES_EDIT" log NOTED "lane:$sc_runner" "$sc_text" ) >&2 || - sc_noted_rc=$? + if [ -n "$LE_TMP" ]; then + sc_log_noted "$LE_TMP" || sc_noted_rc=$? + sc_log_err="$(cat -- "$LE_TMP" 2>/dev/null)" || sc_log_err="" + rm -f -- "$LE_TMP"; LE_TMP="" + if [ -n "$sc_log_err" ]; then printf '%s\n' "$sc_log_err" >&2; fi + else + sc_log_noted || sc_noted_rc=$? + fi # A WRITE THAT DID NOT CONFIRM IS NOT A WRITE THAT DID NOT HAPPEN (rv4's # SHOULD 2). `log`'s 3 is `pushed, not confirmed`: the line may be on # origin already. Its 6 is, in most of its causes, the commit KEPT in the # checkout the write went through and not on origin, and in the rest a # `git add` or `commit` that failed, with nothing committed. Writing the # line again after either would record the sweep twice, or meet S3's - # refusal of a checkout holding a commit origin lacks. So both are handed - # THE PUBLISH ACT first (Amendment 23 S4, #194): `commit` with nothing + # refusal of a checkout holding a commit origin lacks. So a 3 is handed + # THE PUBLISH ACT alone (Amendment 23 S4, #194): `commit` with nothing # staged sends what origin lacks by content and says so where it lacks - # nothing, so it is safe to run whichever happened. Only a 6 whose publish - # act finds nothing to send, and every other refusal, gets the line by hand. + # nothing, so it is safe to run whichever happened. A 6 is handed what + # its own cause needs (below), and every other refusal the line by hand. if [ "$sc_noted_rc" != 0 ]; then sc_by_hand="$(sc_by_hand_of "$sc_text")" case "$sc_noted_rc" in @@ -2075,10 +2101,38 @@ SLU note " $sc_publish" exit 1 ;; 6) - note "the NOTED line recording this sweep was not published (\`lanes-edit.sh log\` exited 6, above). Nothing was lost: each checkout removed held nothing unpublished. Where that says its commit is kept, the line is committed and only origin lacks it, so it is not to be written again: the publish act sends what origin lacks by content:" - note " $sc_publish" - note "Only where the publish act finds nothing to send (git could not add or commit the line, above) is it written by hand:" - note " $sc_by_hand" + # EXIT 6 HAS THREE CAUSES, AND EACH IS FINISHED ITS OWN WAY (lane + # openRepoTools-3's SHOULD 373), chosen by the write's own words: + # a push refused with origin unmoved leaves the line committed, and + # the publish act sends it once what refused it is put right; a + # `git add` or `git commit` that failed leaves the line staged or + # in the tree, which the publish act and the line by hand would + # both refuse, so the log is put back first; and a push that lost + # the race to its bound leaves the line committed on a base origin + # has moved past, which the publish act (a fast-forward only) and + # the line by hand (S3: a commit origin lacks) would both refuse, + # so it is archived on origin (S6.3) and taken off the checkout + # first. Words this sweep does not know get both remedies, in the + # order that is safe whichever happened. + case "$sc_log_err" in + *"with origin/$sc_branch unmoved"*) + note "the NOTED line recording this sweep is committed in $sc_wco and was not published: its push was refused with origin/$sc_branch unmoved (\`lanes-edit.sh log\` exited 6, above), so only origin lacks it, and it is not to be written again. Nothing was lost: each checkout removed held nothing unpublished. Once what refused the push is put right, the publish act sends it:" + note " $sc_publish" ;; + *"git add failed"* | *"git commit failed"*) + note "the NOTED line recording this sweep was NOT committed: git could not add or commit it in $sc_wco (\`lanes-edit.sh log\` exited 6, above), so it is left there, staged or in the working tree, and none of it is on origin. Nothing was lost: each checkout removed held nothing unpublished. Once what refused the commit is put right, put the log back and write the line by hand:" + note " git -C $(printf '%q' "$sc_wco") restore --staged --worktree -- $(printf '%q' "lanes/log/$sc_runner.md")" + note " $sc_by_hand" ;; + *"the commit is local and kept on HEAD"* | *"git could not name the commit to drop this write's to"*) + note "the NOTED line recording this sweep is committed in $sc_wco and could not be published: origin/$sc_branch moved under its push until the write gave up (\`lanes-edit.sh log\` exited 6, above), and the publish act makes only a fast-forward. Nothing was lost: each checkout removed held nothing unpublished. Archive the commit on origin (Amendment 23 S6.3), take it off the checkout, and write the line by hand:" + note " git -C $(printf '%q' "$sc_wco") push origin HEAD:refs/heads/archive/$sc_runner-stranded-$sc_utc" + note " git -C $(printf '%q' "$sc_wco") reset --keep refs/remotes/origin/$sc_branch" + note " $sc_by_hand" ;; + *) + note "the NOTED line recording this sweep was not published (\`lanes-edit.sh log\` exited 6, above). Nothing was lost: each checkout removed held nothing unpublished. Where that says its commit is kept, the line is committed and only origin lacks it, so it is not to be written again: the publish act sends what origin lacks by content:" + note " $sc_publish" + note "Only where the publish act finds nothing to send (git could not add or commit the line, above) is it written by hand:" + note " $sc_by_hand" ;; + esac exit 1 ;; esac die "the NOTED line recording this sweep was NOT written (\`lanes-edit.sh log\` exited $sc_noted_rc, above), so lane $sc_runner's log does not say what was removed. Nothing was lost: each checkout removed held nothing unpublished. Write the line by hand: $sc_by_hand" 1 diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 9ba2838..5b7b08a 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -1682,9 +1682,10 @@ def _noted_on_origin(box: Sweep) -> list: def _remedies(stderr: str) -> list: - """The commands a failed NOTED write hands out on lines of their own.""" + """The commands a failed NOTED write hands out, each on a line of its own, + in the order they are to be run.""" return [line.split(":", 1)[1].strip() for line in stderr.splitlines() - if line.startswith("lane-end: LANES_LANE=")] + if line.startswith("lane-end: ")] def _by_hand(box: Sweep, command: str) -> subprocess.CompletedProcess: @@ -1747,9 +1748,9 @@ def test_noted_line_kept_unpublished_is_published_not_written_again(sweep): """A NOTED write that exits 6 with its commit KEPT in the checkout it was made in (rv4's SHOULD 2): every push is refused here with origin unmoved. The sweep exits 1 without saying "NOT written", and hands out the publish - act first, and the by-hand line only for a publish act that finds nothing - to send. With pushes allowed again, the publish act sends the kept commit, - and origin holds the line once.""" + act alone (lane openRepoTools-3's SHOULD 373: the line is committed, so + a line by hand would record the sweep twice). With pushes allowed again, + the publish act sends the kept commit, and origin holds the line once.""" box = sweep.box gate = box.root / "push.refused" gate.write_text("", encoding="utf-8") @@ -1761,16 +1762,90 @@ def test_noted_line_kept_unpublished_is_published_not_written_again(sweep): assert "was NOT written" not in result.stderr, result.stderr assert "kept on HEAD" in result.stderr, result.stderr remedies = _remedies(result.stderr) - assert len(remedies) == 2, remedies - publish, by_hand = remedies - assert publish.startswith(f"LANES_LANE={RUNNER} lanes-edit.sh commit "), remedies - assert by_hand.startswith(f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), remedies + assert len(remedies) == 1, remedies + assert remedies[0].startswith(f"LANES_LANE={RUNNER} lanes-edit.sh commit "), remedies gate.rename(box.root / "push.allowed") - done = _by_hand(box, publish) + done = _by_hand(box, remedies[0]) assert done.returncode == 0, (done.stdout, done.stderr) assert len(_noted_on_origin(box)) == 1, _noted_on_origin(box) +def _run_in_order(box: Sweep, remedies: list) -> None: + """Runs the commands a failed NOTED write handed out, in order, as a + person would; each must succeed.""" + for command in remedies: + done = _by_hand(box, command) + assert done.returncode == 0, (command, done.stdout, done.stderr) + + +def test_noted_line_whose_commit_fails_is_put_back_then_written_by_hand(sweep): + """A NOTED WRITE WHOSE `git commit` FAILS (lane openRepoTools-3's SHOULD + 373): a `pre-commit` hook in the runner's checkout refuses it, so `log` + exits 6 with the line staged there and nothing committed. The publish act + refuses a staged entry it did not make, and so does the line by hand, so + neither is handed out first: the sweep says to put the log back, then to + write the line by hand. With the hook gone, the two, run in order, land + the line once.""" + box = sweep.box + runner = box.checkout(RUNNER) + hook = runner / ".git" / "hooks" / "pre-commit" + hook.parent.mkdir(parents=True, exist_ok=True) + hook.write_text("#!/bin/sh\necho 'FAKE: a pre-commit hook refuses this commit' >&2\nexit 1\n", + encoding="utf-8") + hook.chmod(0o755) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 1, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows + assert "git commit failed" in result.stderr, result.stderr + assert "was NOT committed" in result.stderr, result.stderr + assert not _noted_on_origin(box) + remedies = _remedies(result.stderr) + assert len(remedies) == 2, remedies + assert remedies[0] == f"git -C {runner} restore --staged --worktree -- lanes/log/{RUNNER}.md", remedies + assert remedies[1].startswith( + f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), remedies + hook.rename(hook.with_name("pre-commit.off")) + _run_in_order(box, remedies) + assert len(_noted_on_origin(box)) == 1, _noted_on_origin(box) + + +def test_noted_line_that_loses_six_push_races_is_archived_then_written_by_hand(sweep): + """A NOTED WRITE THAT LOSES THE RACE SIX TIMES (lane openRepoTools-3's + SHOULD 373): a peer's commit lands on origin before every push from the + runner's checkout, so `log` exits 6 at its bound with the line committed + there and origin moved past it. The publish act makes only a + fast-forward, and the line by hand meets S3's refusal of a checkout + holding a commit origin lacks, so neither is handed out first: the sweep + says to archive the commit on origin (S6.3), take it off the checkout, + then write the line by hand. Run in order, the archive holds the stranded + commit and origin's log holds the line once.""" + box = sweep.box + runner = box.checkout(RUNNER) + gate = box.root / "push.races" + gate.write_text("", encoding="utf-8") + peer = _push_from_peer(box, "handoffs/README.md", "a peer lands first\n", "a-peer-lands-first") + box.use_net(hook=(f'case "$1" in push) case " $* " in *" -C {runner} "*)\n' + f" if [ -e {gate} ]; then {peer} || exit 1; fi ;;\n" + "esac ;; esac\nexit 0\n")) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 1, (result.stdout, result.stderr) + assert _tagged(rows, "removed") == {str(box.checkout(CLOSED))}, rows + assert "after six attempts" in result.stderr, result.stderr + assert not _noted_on_origin(box) + remedies = _remedies(result.stderr) + assert len(remedies) == 3, remedies + archive, reset, by_hand = remedies + assert archive.startswith(f"git -C {runner} push origin HEAD:refs/heads/archive/{RUNNER}-stranded-"), remedies + assert reset == f"git -C {runner} reset --keep refs/remotes/origin/main", remedies + assert by_hand.startswith(f"LANES_LANE={RUNNER} LANES_SESSION={UUID[RUNNER]} lanes-edit.sh log NOTED "), remedies + stranded = box.git("-C", str(runner), "rev-parse", "HEAD").strip() + gate.rename(box.root / "push.free") + _run_in_order(box, remedies) + ref = archive.split("HEAD:", 1)[1] + assert box.git("--git-dir", str(box.origin), "rev-parse", ref).strip() == stranded, "the archive holds it" + assert len(_noted_on_origin(box)) == 1, _noted_on_origin(box) + + def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): """D18: a repository LEANS ON A CLONE when its alternates file names an object directory anywhere AT OR UNDER that clone, not only the clone's own From cb512871a783aca60b499daee9bdc0dc31d0632a Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 20:13:49 +0000 Subject: [PATCH 27/53] Amendment 23 S6.2 tests: the `.git/lanes-edit/` exemption is pinned to its five exact names and to its `-L` guard, and a drift guard reads the names `lanes-edit.sh` writes there from its own text (lane openRepoTools-3's SHOULD 391, #198, slice 4; tests only) Two mutants of MUST 351's hold passed the whole module: a prefix match (`/^removed/` would remove a person's `removed.bak`), and the guard's `|| [ -L ... ]` dropped (a dangling `.git/lanes-edit` link then holds nothing). Lane openRepoTools-3's 19 proposed cases, copied into the module in its style: - `test_a_name_that_only_looks_like_scratch_holds`: 12 look-alike names (a backup, a suffix, another case, a hidden file, a space, a leading dash) each hold, in the dry run and under `--yes`, and the file stays; - a subdirectory holds; a dangling link holds; an empty folder holds nothing, and the checkout is removed; - the drift guard: one case per name `lanes-edit.sh` writes into the folder (`$cs_tmp/`, `.../lanes-edit/`), read from the script of the tree under test, each removable alone, so a sixth scratch name fails until `lane-end` knows it. One more case asserts the guard finds `fetch.err`, so a renamed variable cannot leave it with no case. Both mutants are KILLED by these cases. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 6b3c308cb702a7b53ef98afa1717fb2c39f8ddc2) --- tests/test_a23_sweep_checkouts.py | 98 +++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 5b7b08a..112c96f 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -23,6 +23,7 @@ from __future__ import annotations import os +import re import shutil import signal import subprocess @@ -703,6 +704,103 @@ def test_lanes_edit_scratch_alone_holds_nothing(sweep): assert not os.path.lexists(c) +def _lanes_edit_dir(box: Sweep) -> tuple: + """The CLOSED lane's checkout and its `.git/lanes-edit/`, not yet made.""" + c = box.checkout(CLOSED) + return c, c / ".git" / "lanes-edit" + + +LOOK_ALIKES = ["apply.err.bak", "fetch.errx", "xff.err", "reset.err~", "Reset.err", + "removed.bak", "removed2", "xremoved", "hand-edit-1.reset.err", + ".hidden-diff", "hand edit.diff", "-hand-edit.diff"] + + +@pytest.mark.parametrize("name", LOOK_ALIKES) +def test_a_name_that_only_looks_like_scratch_holds(sweep, name): + """THE SCRATCH EXEMPTION IS BY EXACT NAME (lane openRepoTools-3's SHOULD + 391): a file whose name only begins or ends like one of the five, a + backup, a suffix, another case, a hidden file, a name with a space or a + leading dash, is a person's, and holds the checkout in the dry run and + under `--yes`. The file stays.""" + box = sweep.box + c, d = _lanes_edit_dir(box) + d.mkdir() + (d / name).write_text("a person's note\n", encoding="utf-8") + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + _, rows = sweep(*args, **env) + assert rows[str(c)][0] == "hold", (name, rows[str(c)]) + assert "in .git/lanes-edit/: a hand edit" in rows[str(c)][1], (name, rows[str(c)]) + assert (d / name).is_file() + + +def test_a_subdirectory_of_lanes_edit_holds(sweep): + """A directory in `.git/lanes-edit/` is not one of the five names, and + what is in it may be a person's: it holds (SHOULD 391).""" + box = sweep.box + c, d = _lanes_edit_dir(box) + (d / "sub").mkdir(parents=True) + (d / "sub" / "kept.diff").write_text("a person's note\n", encoding="utf-8") + _, rows = sweep() + assert rows[str(c)][0] == "hold", rows[str(c)] + + +def test_a_dangling_link_for_lanes_edit_holds(sweep): + """A `.git/lanes-edit` that is a link to nowhere holds (SHOULD 391: the + `-L` half of the guard). `-e` alone is false for a dangling link, so only + `-L` brings it in, and what it once named is unknown.""" + box = sweep.box + c, d = _lanes_edit_dir(box) + d.symlink_to(box.root / "nowhere-at-all") + _, rows = sweep() + assert rows[str(c)][0] == "hold", rows[str(c)] + assert ".git/lanes-edit/" in rows[str(c)][1], rows[str(c)] + + +def test_an_empty_lanes_edit_holds_nothing(sweep): + """An empty `.git/lanes-edit/`, which every write makes, holds nothing: + the checkout is removed.""" + box = sweep.box + c, d = _lanes_edit_dir(box) + d.mkdir() + _, rows = sweep() + assert rows[str(c)][0] == "remove", rows[str(c)] + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "removed", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert not os.path.lexists(c) + + +def _names_lanes_edit_writes_there() -> list: + """Every name `lanes-edit.sh` writes into `.git/lanes-edit/` but the hand + edit's own (`hand-edit--.diff`), read from the script of the + tree under test: `$cs_tmp/` and `.../lanes-edit/`.""" + text = (REPO / "lanes-edit.sh").read_text(encoding="utf-8") + found = set(re.findall(r"(?:\$cs_tmp|/lanes-edit)/([A-Za-z0-9_.][A-Za-z0-9_.-]*)", text)) + return sorted(n for n in found if not n.startswith("hand-edit-")) + + +def test_the_names_lanes_edit_writes_there_are_found(): + """The drift guard below reads its cases from `lanes-edit.sh`'s text, so + it must find them: `fetch.err`, which every write leaves, is among them. + Without this, a renamed variable would leave the guard with no case.""" + assert "fetch.err" in _names_lanes_edit_writes_there() + + +@pytest.mark.parametrize("name", _names_lanes_edit_writes_there()) +def test_every_other_name_lanes_edit_writes_there_holds_nothing(sweep, name): + """THE TWO LISTS MUST NOT DRIFT (SHOULD 391). The sweep exempts five + names by hand; a slice that makes `lanes-edit.sh` write a sixth + (`pull.err`, say) would hold every checkout that ever wrote, for good. + The cases are made from the script's own text, so a sixth name fails + here until `lane-end` knows it.""" + box = sweep.box + c, d = _lanes_edit_dir(box) + d.mkdir() + (d / name).write_text("git's own words\n", encoding="utf-8") + _, rows = sweep() + assert rows[str(c)][0] == "remove", (name, rows[str(c)]) + + def test_a_live_cwd_holds(sweep): """S6.2 (c): a live process working inside the checkout holds it. The process is a `sleep` started there, killed by its PID afterwards.""" From 4c73d6d5edde42bcf9d2c1dd42c6dd3970941f82 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 20:23:04 +0000 Subject: [PATCH 28/53] lane-end --sweep-checkouts: `find` decides what in `.git/lanes-edit/` is the writer's scratch, so only a plain file named exactly as one of the five is, and a name of blanks, a name with a newline or a folder that carries a scratch name holds the checkout (lane openRepoTools-3's NIT 392, #198, slice 4) MUST 351's hold read `ls -A` text and matched names, not kinds. Six shapes read as scratch, and `--yes` removed the checkout with the file in it: a name of only blanks, a name of only a TAB, `removed` followed by a newline, `fetch.err`, a newline, then `ff.err`, a folder named `removed` holding a file, and a folder named `fetch.err` holding a file. Lane openRepoTools-3's tested patch (`fix392b.diff`), applied by hand: `find /lanes-edit/ -mindepth 1 -maxdepth 1 ! ( -type f ( -name apply.err -o ... -o -name removed ) ) -print`. Anything it prints holds, named by its last path component; any `find` error is the "cannot be listed" hold. Its comment is re-wrapped to 80 columns. `-mindepth` and `-maxdepth` are in BSD `find` too. - `test_only_a_plain_file_with_a_scratch_name_is_scratch`: the six shapes, each held in the dry run and under `--yes`, the file kept. Red before the fix: 6 failed, each `remove`. Green after, with 391's cases and 351's: 29 passed. Hygiene 253 passed. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 4c45223e05dc594e273bd067097f0fe63b90ea8c) --- lane-end | 16 ++++++----- tests/test_a23_sweep_checkouts.py | 44 +++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 7 deletions(-) diff --git a/lane-end b/lane-end index 4951572..db0e807 100755 --- a/lane-end +++ b/lane-end @@ -1383,14 +1383,16 @@ SC_IDX # there holds, but the writer's own scratch: git's words from a write and # the lines a carry removed, which every write leaves. if [ -e "$sh_gd/lanes-edit" ] || [ -L "$sh_gd/lanes-edit" ]; then - if ! sh_x="$(ls -A -- "$sh_gd/lanes-edit" 2>/dev/null)"; then + # WHAT IS SCRATCH is decided by find's own tests, a PLAIN FILE named + # exactly as one of the five, and no text is parsed (lane + # openRepoTools-3's NIT 392): a name of blanks or with a newline in + # it, or a folder or link that only carries a scratch name, is not + # scratch. Whatever find cannot open or look at is "cannot be + # listed" (find exits 1). + if ! sh_x="$(find "$sh_gd/lanes-edit/" -mindepth 1 -maxdepth 1 ! \( -type f \( -name apply.err -o -name fetch.err -o -name ff.err -o -name reset.err -o -name removed \) \) -print 2>/dev/null)"; then sc_hold "its .git/lanes-edit/, where lanes-edit.sh keeps a hand edit of the register its commit could not carry, cannot be listed, so whether it keeps one is unknown" - else - sh_x="$(printf '%s\n' "$sh_x" | awk 'NF && $0 !~ /^(apply|fetch|ff|reset)\.err$/ && $0 != "removed"')" || - sh_x="(the list could not be read)" - if [ -n "$sh_x" ]; then - sc_hold "lanes-edit.sh keeps $(sc_first "$sh_x") in .git/lanes-edit/: a hand edit of the register its commit could not carry, or the copy of one it carried (read it; remove it once origin has the edit)" - fi + elif [ -n "$sh_x" ]; then + sc_hold "lanes-edit.sh keeps $(sc_first "$(printf '%s\n' "$sh_x" | awk -F/ '{ print $NF }')") in .git/lanes-edit/: a hand edit of the register its commit could not carry, or the copy of one it carried (read it; remove it once origin has the edit)" fi fi sh_x="$(git -C "$sh_p" ls-files -u 2>/dev/null)" || sh_x="unreadable" diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 112c96f..33f0e81 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -770,6 +770,50 @@ def test_an_empty_lanes_edit_holds_nothing(sweep): assert not os.path.lexists(c) +def _a_file(name: str): + def make(d: Path) -> Path: + d.mkdir() + (d / name).write_text("a person's note\n", encoding="utf-8") + return d / name + return make + + +def _a_folder(name: str): + def make(d: Path) -> Path: + (d / name).mkdir(parents=True) + (d / name / "kept.diff").write_text("a person's note\n", encoding="utf-8") + return d / name / "kept.diff" + return make + + +NOT_SCRATCH_BY_KIND = [ + pytest.param(_a_file(" "), id="a name of only blanks"), + pytest.param(_a_file("\t"), id="a name of only a TAB"), + pytest.param(_a_file("removed\n"), id="removed and a newline"), + pytest.param(_a_file("fetch.err\nff.err"), id="fetch.err, a newline, ff.err"), + pytest.param(_a_folder("removed"), id="a folder named removed"), + pytest.param(_a_folder("fetch.err"), id="a folder named fetch.err"), +] + + +@pytest.mark.parametrize("make", NOT_SCRATCH_BY_KIND) +def test_only_a_plain_file_with_a_scratch_name_is_scratch(sweep, make): + """`find` DECIDES WHAT IS SCRATCH, AND NO TEXT IS PARSED (lane + openRepoTools-3's NIT 392): only a plain file named exactly as one of + the five is the writer's. A name of blanks or with a newline in it, and + a folder that only carries a scratch name, are a person's, and hold the + checkout, in the dry run and under `--yes`; what they keep stays.""" + box = sweep.box + c, d = _lanes_edit_dir(box) + kept = make(d) + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert "in .git/lanes-edit/: a hand edit" in rows[str(c)][1], rows[str(c)] + assert kept.is_file() + + def _names_lanes_edit_writes_there() -> list: """Every name `lanes-edit.sh` writes into `.git/lanes-edit/` but the hand edit's own (`hand-edit--.diff`), read from the script of the From 6ff075ac63ba9c457590a51b7f2c3b2d2fef0297 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 21:43:38 +0000 Subject: [PATCH 29/53] Amendment 23 S6.2 tests: three guards of the act that no case reached are pinned, the move asked again whether its path is directly under `-lanes/`, the lock let go when the move fails, and a run whose only removal did not finish recorded (lane openRepoTools-3's SHOULD 374, #198, slice 4; tests only) Three one-line mutants of the act passed the whole module. A case for each: - `test_a_lanes_folder_swapped_for_a_link_under_the_lock_holds`: a `git` first on PATH, after the candidate's last hold read under its lock, copies `-lanes/` (its lock and pid line included) to a decoy and puts a link to the decoy in its place. Only `sc_under_home`, asked again at the move, stops it: the checkout is held, and the decoy's copy stays. So its false branch is reachable. - `test_a_checkout_that_cannot_be_moved_aside_lets_its_lock_go`: an `mv` first on PATH refuses the move to the aside; the checkout is held, its lock is gone, and the next candidate is removed (its own lock would otherwise leave the first one's for good). - `test_a_removal_that_does_not_finish_exits_1` is parametrized: alone, with no removal that finished, the PARTLY run still writes its NOTED line, naming the aside. Each of the three mutants is KILLED by its case. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 75c711859fb8cbfe571ed96c678096211333cbb3) --- tests/test_a23_sweep_checkouts.py | 82 ++++++++++++++++++++++++++++--- 1 file changed, 76 insertions(+), 6 deletions(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 33f0e81..ee438b3 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -1324,6 +1324,69 @@ def test_a_lock_whose_pid_line_cannot_be_written_is_let_go(sweep, after): assert result.returncode == 0, (result.stdout, result.stderr) +def test_a_lanes_folder_swapped_for_a_link_under_the_lock_holds(sweep): + """WHAT IS MOVED IS ASKED AGAIN AT THE MOVE: DIRECTLY UNDER `-lanes/`, BY + ITS REAL PARENT (lane openRepoTools-3's SHOULD 374; `sc_under_home`). A + `git` first on PATH, after the last read of the candidate's holds under + its lock (its ignored files), copies `-lanes/` to a decoy outside it, + the lock and its pid line included, and puts a link to the decoy where + `-lanes/` was. Every check after that reads the decoy's copy as the + checkout and the pid line as the sweep's own; only the real parent, + asked again, stops the move. The checkout is held, and the decoy's copy + stays.""" + box = sweep.box + c = box.checkout(CLOSED) + decoy, real = box.root / "decoy-lanes", box.root / "real-lanes" + marker = box.root / "swapped.marker" + git = shutil.which("git") + assert git, "no git on the test's own PATH" + shim = box.fakebin / "git" + shim.write_text("#!/usr/bin/env bash\n" + f'case " $* " in *" -C {c} ls-files --others --ignored "*)\n' + f' {git} "$@"; rc=$?\n' + f" if [ ! -e {marker} ]; then : > {marker} && cp -a -- {box.lanes_root} {decoy} && " + f"mv -- {box.lanes_root} {real} && ln -s -- {decoy} {box.lanes_root}; fi\n" + ' exit "$rc" ;;\n' + "esac\n" + f'exec {git} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert marker.exists() and box.lanes_root.is_symlink(), "the shim swapped -lanes/ for a link" + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"it could not be moved aside to {box.lanes_root}/.{CLOSED}.removing-" in rows[str(c)][1], rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (decoy / CLOSED / ".git").is_dir(), "the decoy's copy, outside -lanes/, stays" + assert (real / CLOSED / ".git").is_dir(), "and so does the checkout" + assert not list(decoy.glob(".*.removing-*")), "nothing was moved aside in the decoy" + + +def test_a_checkout_that_cannot_be_moved_aside_lets_its_lock_go(sweep): + """A MOVE THAT FAILS LETS THE CHECKOUT'S LOCK GO (lane openRepoTools-3's + SHOULD 374). An `mv` first on PATH refuses the move of the CLOSED lane's + checkout to its aside, as a parent it cannot write in would. It is held + with that reason, and the writers' lock the sweep took in it is gone: + the next candidate takes a lock of its own, so a lock the first one + left would be left for good, its writers waiting out ten minutes on it + (`mutex_stale`). The next candidate is removed.""" + box = sweep.box + c, other = box.checkout(CLOSED), _closed_checkout(box) + real = shutil.which("mv") + assert real, "no mv on the test's own PATH" + shim = box.fakebin / "mv" + shim.write_text("#!/usr/bin/env bash\n" + f'case "$*" in *"/.{CLOSED}.removing-"*) echo "FAKE: mv refused" >&2; exit 1 ;; esac\n' + f'exec {real} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"it could not be moved aside to {box.lanes_root}/.{CLOSED}.removing-" in rows[str(c)][1], rows[str(c)] + assert rows[str(other)][0] == "removed", rows[str(other)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert c.is_dir() + assert not os.path.lexists(box.lock_of(c)), "the lock it took in the held checkout is let go" + _no_aside(box) + + @pytest.mark.parametrize("op,note", [ ("fetch", "fetch failed — reading the logs as they stand locally"), ("ls-remote", "NO FETCH WAS MADE"), @@ -1572,15 +1635,20 @@ def test_held_parent_keeps_its_dependents_chain(sweep, held, expect): @pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, reason="root removes what a read-only directory holds") -def test_a_removal_that_does_not_finish_exits_1(sweep): +@pytest.mark.parametrize("finished", [True, False], + ids=["beside a removal that finished", "alone"]) +def test_a_removal_that_does_not_finish_exits_1(sweep, finished): """A REMOVAL THAT DOES NOT FINISH IS SAID, BY PATH, AND EXITS 1 (rv4's SHOULD 1, R4). The CLOSED lane's checkout holds an IGNORED directory (ignored files hold nothing) that is read-only, so the checkout is moved aside and the removal of the aside stops partway: it is named PARTLY, - the run exits 1, and the aside stays, hidden, under `-lanes/`.""" + the run exits 1, and the aside stays, hidden, under `-lanes/`. It is + RECORDED ALONE TOO (lane openRepoTools-3's SHOULD 374): with no other + removal that finished, the NOTED line is still written, for what was + moved aside.""" box = sweep.box c = box.checkout(CLOSED) - other = _closed_checkout(box) + other = _closed_checkout(box) if finished else None (c / ".git" / "info" / "exclude").write_text("scratch/\n", encoding="utf-8") box.write(c, "scratch/kept.txt", "an ignored file in a read-only directory\n") (c / "scratch").chmod(0o555) @@ -1597,11 +1665,13 @@ def test_a_removal_that_does_not_finish_exits_1(sweep): # did (rv4's NIT 3), on the summary line and in the NOTED line, and # its ignored files are named at the act (NIT 2). assert "ignored, which holds nothing: 1 (scratch/)" in rows[str(c)][1], rows[str(c)] - assert " 1 removed, 1 moved aside and not all removed, " in result.stdout, result.stdout + n = 1 if finished else 0 + assert f" {n} removed, 1 moved aside and not all removed, " in result.stdout, result.stdout noted = _noted_on_origin(box) assert len(noted) == 1, noted - assert (f"removed 1 register checkout(s) under brett-wip-lanes/ that held nothing " - f"unpublished: {other.name}; moved aside and not all removed: {asides[0].name};") in noted[0], noted + names = f": {other.name}" if other else "" + assert (f"removed {n} register checkout(s) under brett-wip-lanes/ that held nothing " + f"unpublished{names}; moved aside and not all removed: {asides[0].name};") in noted[0], noted finally: for aside in box.lanes_root.glob(f".{CLOSED}.removing-*"): (aside / "scratch").chmod(0o755) From 1b3de12c47cf08795cf3010d9346ce840cb52597 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 22 Oct 2026 21:54:18 +0000 Subject: [PATCH 30/53] lane-end --sweep-checkouts: only `last-session`'s 8 is a lane that names no transcript uuid, so a register or a log that could not be read refuses `--yes` as a read that failed, with its reason (lane openRepoTools-3's finding 545, #198, slice 4) `--yes` with no `LANES_SESSION` reads the runner's uuid with `lanes-edit.sh last-session `, and took ANY nonzero exit as "names none" (`|| sc_uuid=""`), refused with 2. `last-session` exits 8 only where no uuid is knowable, and 1 where the register or the log could not be read; lane openRepoTools-4's #303 makes the 1 come for an unreadable log too. A read that failed is not an answer. Now 0 is the uuid, 8 is "names none" (2, as before), and any other exit refuses with 1 ("Nothing was removed"), naming the exit and the writer's last line of stderr, which is captured for that (a capture that cannot be made names the exit alone). `test_a_last_session_that_cannot_be_read_is_never_none`: a `last-session` stub that exits 1 refuses with 1 and its words (red before: 2, "names none"), and one that exits 8 still refuses with 2. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 24e3c6d6bb15c51d997df855d1fdd2f5726fcc3b) --- lane-end | 18 ++++++++++++++++- tests/test_a23_sweep_checkouts.py | 33 +++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/lane-end b/lane-end index db0e807..c5c7d69 100755 --- a/lane-end +++ b/lane-end @@ -1148,11 +1148,27 @@ SC_IDX # would refuse for want of one is refused here instead, while nothing is # gone, rather than met after the removals it was to record. The writer # asks again when the line is written, and its answer is the decision. + # + # ONLY `last-session`'S 8 IS "NAMES NONE" (lane openRepoTools-3's finding + # 545): its 1 is a register or a log that could not be read, and that is + # a read that failed (1), never a lane with no uuid. Its words are kept to + # name the reason; a capture that cannot be made names the exit alone. sc_uuid="" if [ "$sweep_yes" = 1 ]; then sc_uuid="${LANES_SESSION:-}" if [ -z "$sc_uuid" ]; then - sc_uuid="$(LANES_NO_FETCH=1 "$LANES_EDIT" last-session "$sc_runner" 2>/dev/null)" || sc_uuid="" + sc_ls_rc=0; sc_ls_err="" + LE_TMP="$(mktemp "${TMPDIR:-/tmp}/lane-end-sweep.XXXXXX" 2>/dev/null || printf '')" + sc_uuid="$(LANES_NO_FETCH=1 "$LANES_EDIT" last-session "$sc_runner" 2>"${LE_TMP:-/dev/null}")" || sc_ls_rc=$? + if [ -n "$LE_TMP" ]; then + sc_ls_err="$(awk 'NF { l = $0 } END { print l }' < "$LE_TMP" 2>/dev/null)" || sc_ls_err="" + rm -f -- "$LE_TMP"; LE_TMP="" + fi + case "$sc_ls_rc" in + 0) : ;; + 8) sc_uuid="" ;; + *) die "lane $sc_runner's transcript uuid, for the session field of the NOTED line this act ends with (Amendment 7(b)), could not be read: \`lanes-edit.sh last-session $sc_runner\` exited $sc_ls_rc${sc_ls_err:+ ($sc_ls_err)}. That is not a lane that names none: the register or its log could not be read. Fix the read and re-run, or name your session: LANES_SESSION=\"\$CLAUDE_CODE_SESSION_ID\" LANES_LANE=$sc_runner $prog --sweep-checkouts --yes. Nothing was removed." 1 ;; + esac fi if ! is_fork_uuid "$sc_uuid"; then if [ -n "${LANES_SESSION:-}" ]; then diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index ee438b3..d082afd 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -278,6 +278,39 @@ def test_yes_is_refused_before_anything_is_removed(sweep, env, said): assert box.origin_head() == head +@pytest.mark.parametrize("code,rc,said", [ + pytest.param(1, 1, ("lane repoS4-1's transcript uuid, for the session field of the NOTED line", + "could not be read: `lanes-edit.sh last-session repoS4-1` exited 1 " + "(FAKE: the log of repoS4-1 cannot be read)", + "That is not a lane that names none"), + id="exit 1, a log that cannot be read"), + pytest.param(8, 2, ("has no transcript uuid for the session field", + f"`lanes-edit.sh last-session {RUNNER}` names none"), + id="exit 8, a lane that names none"), +]) +def test_a_last_session_that_cannot_be_read_is_never_none(sweep, code, rc, said): + """ONLY `last-session`'S 8 MEANS "NAMES NONE" (lane openRepoTools-3's + finding 545, ahead of lane openRepoTools-4's #303). Its 1 is a register + or a log that could not be read: `--yes` with no `LANES_SESSION` refuses + then as a read that failed (1), naming the exit and the writer's own + words, and never as a lane with no transcript uuid (2). Nothing is + removed either way.""" + box = sweep.box + stub = _wrapped(box, f"lanes-edit-last-session-{code}", "last-session", ( + f" printf 'FAKE: the log of %s cannot be read\\n' \"$2\" >&2\n" + f" exit {code}\n")) + before, head = box.everything(), box.origin_head() + result, _ = sweep("--yes", LANES_LANE=RUNNER, LANES_EDIT=stub) + assert result.returncode == rc, (result.stdout, result.stderr) + for text in (*said, "Nothing was removed"): + assert text in result.stderr, result.stderr + if code != 8: + assert "has no transcript uuid" not in result.stderr, result.stderr + assert f"last-session {RUNNER}` names none" not in result.stderr, result.stderr + assert box.everything() == before + assert box.origin_head() == head + + @pytest.mark.parametrize("extra", [ ("--retire",), ("--force",), ("--inventory-only",), ("--state-was", "x"), ("--home", "opensoft/repoS4"), ("--reason", "why"), ("--retire-dormant", "repoS4"), From 63d64a344116dc720dca65c78082e4da6e9deb6c Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 02:52:24 +0000 Subject: [PATCH 31/53] lane-end --sweep-checkouts: what leans on a checkout is looked for where clones are made, under W's parent to depth 6, hidden ones too, and under `$TMPDIR` and `$HOME` to depth 4, and a scan that cannot finish holds every candidate (lane openRepoTools-3's SHOULD 352, with NITs 378 and 356, #198, slice 4) `sc_dep_scan` read alternates files in three places only: W, the entries directly in W's parent, and `-lanes/`. A `--reference` clone of a CLOSED lane's checkout in `.lane-worktrees/`, in a project's `review/`, or in `$TMPDIR` held nothing, and `--yes` removed what it borrows from. Now `find` lists every object directory (pruned there) and every `.git` file under W's parent to depth 6 and under `$TMPDIR` and `$HOME` to depth 4, by their real paths; the lines are deduplicated, since the roots can nest. The dry run names the scope, and the scan is read again under each lock, as before. A repository at or under a candidate, or under a path the run removes, goes with it. - A scope that cannot be entered, a `find` that fails for anything but a directory it cannot read, or an alternates or `.git` file that cannot be read (NIT 356; a mode-000 `.git` file was skipped) holds every candidate: "what leans on it cannot be told: ...". - A directory `find` cannot read (another user's, or gone while read) is counted once and said, and holds nothing, by question 2's rule. - The lines are written TARGET FIRST (NIT 378), so a TAB in the dependent's own path no longer hides it. Tests: `test_a_dependent_anywhere_in_the_scope_holds` (5 places), `test_the_dry_run_names_the_dependents_scope`, `test_a_dependents_scan_that_fails_holds_every_candidate`, `test_a_directory_the_dependents_scan_cannot_read_is_counted_and_holds_nothing`, `test_a_dependents_file_that_cannot_be_read_holds_every_candidate` (2). Red before: 10 failed. Green: 10, plus 17 neighbours (leaves first, the held-parent chain, the store inside a clone, the `.git` file cases). Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit e04d26396dd20e9abbd60c7ca8dadfa289d40d02) --- lane-end | 185 ++++++++++++++++++++++-------- tests/test_a23_sweep_checkouts.py | 155 ++++++++++++++++++++++++- 2 files changed, 291 insertions(+), 49 deletions(-) diff --git a/lane-end b/lane-end index c5c7d69..9c78d4b 100755 --- a/lane-end +++ b/lane-end @@ -1362,8 +1362,9 @@ SC_IDX # with is itself a hold: nothing can be proved published. IGNORED # files do not hold — the register's own rules keep them out of every # commit — and they are named, so a person sees them first; - # (b) another repository whose alternates file names its objects, and its - # own linked worktrees; + # (b) another repository whose alternates file names its objects, looked + # for under W's parent, `$TMPDIR` and `$HOME` (SHOULD 352, below), and + # its own linked worktrees; a scan that cannot finish holds them all; # (c) a process this user can see whose working directory is at or under # it: `/proc/*/cwd` on Linux, `lsof` on macOS, and with neither it is # held as "cannot tell". A process whose directory cannot be read @@ -1480,62 +1481,140 @@ SH_REFS return 0 } - # (b) WHAT LEANS ON WHAT: ` TAB `, for W, every repository in W's own directory and - # every entry of `-lanes/`, hidden ones included. Read afresh before each - # removal, so a dependent kept earlier in the run still holds what it - # borrows from. + # (b) WHAT LEANS ON WHAT, LOOKED FOR WHERE CLONES ARE MADE (lane + # openRepoTools-3's SHOULD 352; S6.2 (b): *"no other clone's alternates + # file names it"*): every repository under W's parent to depth 6, hidden + # directories included, and under `$TMPDIR` (or /tmp) and `$HOME` to depth + # 4, by their real paths. `find` lists each object directory, pruned there, + # and each `.git` FILE. An object directory with an alternates file is a + # repository's store, and each line of that file names an object directory + # the repository borrows from: `sc_dep_lines` holds ` TAB `, THE TARGET FIRST (NIT 378), so a + # TAB in the repository's own path stays in its own field, and the target + # is what is matched. Read afresh under each lock, so a dependent kept + # earlier in the run still holds what it borrows from. # # AND A `.git` FILE IS READ THE SAME WAY (rv4's NIT 1): a directory whose # `.git` file names a clone's git directory, or one inside it, works in it, # and removing the clone takes that away. `sc_gitfile_lines` holds ` TAB `. A linked + # real git directory its .git file names> TAB `. A linked # worktree is one, and is also held through `.git/worktrees/`. - sc_dep_lines=""; sc_gitfile_lines="" + # + # A SCAN THAT CANNOT FINISH HOLDS EVERY CANDIDATE (`sc_dep_fail`), since + # what leans on each is then unknown: a scope that cannot be entered, a + # `find` that fails for anything but a directory it cannot read, and an + # alternates or `.git` file that cannot be read (NIT 356). A DIRECTORY + # `find` CANNOT READ — another user's, or one gone while read — is counted + # and said, and holds nothing, by question 2's rule for working + # directories: one never readable would hold every checkout for good. It + # is counted once, though two scopes meet it. + sc_dep_lines=""; sc_gitfile_lines=""; sc_dep_fail=""; sc_dep_unread=0 + sc_dep_at=(); sc_dep_to=(); sc_dep_named=() + sc_dep_root() { # + sdr_r="" + if [ -n "$2" ]; then sdr_r="$(sc_real "$2")"; fi + sc_dep_to+=("$1"); sc_dep_at+=("$sdr_r"); sc_dep_named+=("${sdr_r:-${2:-(unset)}}") + } + sc_dep_root 6 "$sc_root" + sc_dep_root 4 "${TMPDIR:-/tmp}" + sc_dep_root 4 "${HOME:-}" sc_dep_scan() { - sc_dep_lines=""; sc_gitfile_lines="" - for sds_e in "$sc_w" "$sc_root"/* "$sc_home"/* "$sc_home"/.[!.]* "$sc_home"/..?*; do - [ -d "$sds_e" ] || continue - if [ -f "$sds_e/.git" ] && [ ! -L "$sds_e/.git" ]; then - # git strips a trailing CR from the line (rv4-c3's NIT c3-3), and so - # does `tr`, first: BSD `sed` reads no `\r` in its script. - sds_g="$(tr -d '\r' 2>/dev/null < "$sds_e/.git" | sed -n -e '1s/^gitdir: //p')" || sds_g="" - case "$sds_g" in '') : ;; /*) : ;; *) sds_g="$sds_e/$sds_g" ;; esac - if [ -n "$sds_g" ]; then - sds_r="$(sc_real "$sds_g")" - if [ -n "$sds_r" ]; then - sc_gitfile_lines="${sc_gitfile_lines}${sds_e}${sc_tab}${sds_r} -" - fi - fi + sc_dep_lines=""; sc_gitfile_lines=""; sc_dep_fail=""; sc_dep_unread=0 + sds_dl=""; sds_gl=""; sds_unread=""; sds_i=0 + while [ "$sds_i" -lt "${#sc_dep_to[@]}" ]; do + sds_root="${sc_dep_at[$sds_i]}"; sds_depth="${sc_dep_to[$sds_i]}" + sds_named="${sc_dep_named[$sds_i]}"; sds_i=$((sds_i + 1)) + if [ -z "$sds_root" ] || [ ! -d "$sds_root" ] || [ ! -r "$sds_root" ] || [ ! -x "$sds_root" ]; then + sc_dep_fail="${sc_dep_fail:-$sds_named cannot be entered and listed}"; continue fi - if [ -f "$sds_e/.git/objects/info/alternates" ]; then - sds_a="$sds_e/.git/objects/info/alternates"; sds_o="$sds_e/.git/objects" - elif [ -f "$sds_e/objects/info/alternates" ]; then - sds_a="$sds_e/objects/info/alternates"; sds_o="$sds_e/objects" - else - continue + LE_TMP="$(mktemp "${TMPDIR:-/tmp}/lane-end-sweep.XXXXXX" 2>/dev/null || printf '')" + if [ -z "$LE_TMP" ]; then + sc_dep_fail="${sc_dep_fail:-no temporary file could be made for the list of what is under $sds_root}"; continue + fi + sds_rc=0 + sds_err="$(LC_ALL=C find "$sds_root" -maxdepth $((sds_depth + 2)) \( -type d -name objects -print0 -prune \) -o \( -type f -name .git -print0 \) 2>&1 >"$LE_TMP")" || sds_rc=$? + sds_other=""; sds_counted=0 + while IFS= read -r sds_l; do + case "$sds_l" in + '') : ;; + *": Permission denied" | *": No such file or directory") + sds_unread="$sds_unread$sds_l +"; sds_counted=1 ;; + *) sds_other="${sds_other:-$sds_l}" ;; + esac + done </dev/null)"; then + sc_dep_fail="${sc_dep_fail:-the alternates file of $sds_e cannot be read}"; continue + fi + while IFS= read -r sds_l; do + case "$sds_l" in '' | '#'*) continue ;; esac + case "$sds_l" in /*) sds_t="$sds_l" ;; *) sds_t="$sds_f/$sds_l" ;; esac + sds_r="$(sc_real "$sds_t")" + if [ -n "$sds_r" ]; then + sds_dl="$sds_dl$sds_r$sc_tab$sds_e +" + fi + done < — 0 when it is at or under one of them + while IFS= read -r sig_g; do + [ -n "$sig_g" ] || continue + case "$1" in "$sig_g" | "$sig_g"/*) return 0 ;; esac + done < [] [] — what leans on it scd_out=""; scd_lines="${3-$sc_dep_lines}" - while IFS="$sc_tab" read -r scd_d scd_t; do - [ -n "$scd_d" ] && [ "$scd_d" != "$1" ] || continue + while IFS="$sc_tab" read -r scd_t scd_d; do + [ -n "$scd_d" ] || continue case "$scd_t" in "$1" | "$1"/*) : ;; *) continue ;; esac - case " ${2-} " in *" $scd_d "*) continue ;; esac + case "$scd_d" in "$1" | "$1"/*) continue ;; esac + if sc_in_gone "$scd_d" "${2-}"; then continue; fi scd_out="${scd_out:+$scd_out }$scd_d" done < <1 when this run holds its lock> [] sc_holds "$1" "$2" + if [ -n "$sc_dep_fail" ]; then sc_hold "what leans on it cannot be told: $sc_dep_fail"; fi sjh_deps="$(sc_dependents "$1" "${3-}")" if [ -n "$sjh_deps" ]; then sc_hold "leaned on by $sjh_deps (its alternates file names this clone's objects)"; fi sjh_deps="$(sc_dependents "$1" "${3-}" "$sc_gitfile_lines")" @@ -1807,11 +1887,16 @@ SS_ROW if [ "$sc_hidden" -gt 0 ]; then printf ' (%s hidden entr(ies) under %s/ - creation locks, partial clones, the sidecar lock - are never looked at)\n' "$sc_hidden" "$sc_home" fi + printf ' dependents : every repository under %s/ to depth 6, hidden ones too, and under %s/ and %s/ to depth 4; read again under each lock\n' \ + "${sc_dep_named[0]}" "${sc_dep_named[1]}" "${sc_dep_named[2]}" sc_cwd_scan; sc_mount_scan if [ "$sc_cwd_unread" -gt 0 ]; then printf ' (%s process(es) whose working directory could not be read - another user'"'"'s, or ended while read - do not hold anything)\n' "$sc_cwd_unread" fi sc_dep_scan + if [ "$sc_dep_unread" -gt 0 ]; then + printf ' (%s director(ies) the dependents scan could not read - another user'"'"'s, or gone while read - are not looked in)\n' "$sc_dep_unread" + fi # LEAVES FIRST (S6.2 (b): *"so leaves go first"*). The candidates are taken # in dependency order, every clone before the clones it borrows objects @@ -1826,14 +1911,19 @@ SS_ROW sc_placed=(); sc_order=() sc_i=0 while [ "$sc_i" -lt "${#sc_cands[@]}" ]; do sc_placed[$sc_i]=0; sc_i=$((sc_i + 1)); done + # A repository at or under a candidate not yet placed is that candidate + # leaning (it goes with it), and one at or under this candidate is not. sc_leaned_on_by_unplaced() { # — 0 when a candidate not yet placed leans on it slu_c="${sc_cands[$1]}" - while IFS="$sc_tab" read -r slu_d slu_t; do - [ -n "$slu_d" ] && [ "$slu_d" != "$slu_c" ] || continue + while IFS="$sc_tab" read -r slu_t slu_d; do + [ -n "$slu_d" ] || continue case "$slu_t" in "$slu_c" | "$slu_c"/*) : ;; *) continue ;; esac + case "$slu_d" in "$slu_c" | "$slu_c"/*) continue ;; esac slu_j=0 while [ "$slu_j" -lt "${#sc_cands[@]}" ]; do - if [ "${sc_placed[$slu_j]}" = 0 ] && [ "${sc_cands[$slu_j]}" = "$slu_d" ]; then return 0; fi + if [ "${sc_placed[$slu_j]}" = 0 ]; then + case "$slu_d" in "${sc_cands[$slu_j]}" | "${sc_cands[$slu_j]}"/*) return 0 ;; esac + fi slu_j=$((slu_j + 1)) done done < @@ -1971,7 +2062,8 @@ SLU sc_say_held "$sc_c" "$sc_why" "$SC_HOLD" else printf ' remove %s %s%s\n' "$sc_c" "$sc_why" "${SC_IGN:+; $SC_IGN}" - sc_gone="${sc_gone:+$sc_gone }$sc_c"; sc_n_removable=$((sc_n_removable + 1)) + sc_n_removable=$((sc_n_removable + 1)); sc_gone="$sc_gone$sc_c +" fi continue fi @@ -2030,7 +2122,8 @@ SLU sc_removed_names="${sc_removed_names:+$sc_removed_names }${sc_c##*/}" sc_n_removed=$((sc_n_removed + 1)) fi - sc_gone="${sc_gone:+$sc_gone }$sc_c" + sc_gone="$sc_gone$sc_c +" sc_sig_arm done printf '%s' "$sc_keep_lines" diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index d082afd..6e5b592 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -12,7 +12,8 @@ EVERY RUN HERE IS THE REAL `lane-end` AND THE REAL `lanes-edit.sh` IN A SANDBOX: `tests/a23_box.py`'s `RegisterBox` (a bare origin, W cloned from it, -the `workspace.yaml` naming W, a `$HOME` of its own), with lanes in the +the `workspace.yaml` naming W, a `$HOME` and a `$TMPDIR` of its own, which +with W's parent are the places the dependents scan reads), with lanes in the register and the logs, their checkouts under `-lanes/`, and three per-profile clones beside W chained by alternates. The fixture is #202's `ended` style (`tests/test_precision_90a_96.py`): one function that runs the @@ -163,9 +164,10 @@ def sweep(tmp_path: Path): root = Path(os.path.realpath(tmp_path)) / "s" box = Sweep(root) # EVERY PATH A RUN COULD REMOVE IS UNDER THIS TEST'S OWN BASETEMP: W, its - # parent, `-lanes/` and the `$HOME` every helper reads are the box's. + # parent, `-lanes/` and the `$HOME` every helper reads are the box's, + # and so is the `$TMPDIR` the dependents scan reads with them. base = Path(os.path.realpath(tmp_path)) - for path in (box.w, box.w.parent, box.lanes_root, box.home): + for path in (box.w, box.w.parent, box.lanes_root, box.home, Path(box.env["TMPDIR"])): assert Path(os.path.realpath(path)).is_relative_to(base), path assert box.env["HOME"] == str(box.home) @@ -2121,6 +2123,153 @@ def test_a_store_kept_inside_a_clone_is_leaned_on(sweep): box.git("-C", str(borrower), "fsck", "--connectivity-only") +def _borrower(box: Sweep, path: Path, lender: Path) -> Path: + """A clone at `path` that borrows `lender`'s objects through its + alternates file (`git clone --no-local --reference`, lane + openRepoTools-3's probe for 352), with a commit of its own that origin + lacks: removing `lender` would leave it unable to fsck or push it.""" + path.parent.mkdir(parents=True, exist_ok=True) + box.git("clone", "-q", "--no-local", "--reference", str(lender), str(box.origin), str(path)) + box.quiet(path) + alternates = path / ".git" / "objects" / "info" / "alternates" + assert f"{lender}/.git/objects" in alternates.read_text(encoding="utf-8") + box.write(path, "rescue.md", "a commit only this clone has\n") + box.commit(path, "a commit of its own", "rescue.md") + return path + + +DEPENDENTS_AFAR = [ + pytest.param(lambda box: box.w.parent / ".lane-worktrees" / "lane-x", + id="in a hidden directory beside W"), + pytest.param(lambda box: box.w.parent / "aproject" / "review" / "lane-x-review", + id="in a project's review directory"), + pytest.param(lambda box: box.w.parent / "aproject" / "worktrees" / "wt-1" / "review" / "deep", + id="five levels under W's parent"), + pytest.param(lambda box: Path(box.env["TMPDIR"]) / "rescue", + id="in TMPDIR"), + pytest.param(lambda box: box.lanes_root / "tab\there", + id="whose own path holds a TAB"), +] + + +@pytest.mark.parametrize("where", DEPENDENTS_AFAR) +def test_a_dependent_anywhere_in_the_scope_holds(sweep, where): + """WHAT LEANS ON A CHECKOUT IS LOOKED FOR WHERE CLONES ARE MADE (lane + openRepoTools-3's SHOULD 352; S6.2 (b): *"no other clone's alternates + file names it"*): under W's parent to depth 6, hidden directories + included, and under `$TMPDIR` and `$HOME` to depth 4, not only in W, the + entries beside it and `-lanes/`. A `--reference` clone of the CLOSED + lane's checkout, with a commit of its own, holds it in each place, by + the dry run and by `--yes`, and stays whole. A TAB in the dependent's + own path splits off only its name (NIT 378: the target is written + first, and the target is what is matched).""" + box = sweep.box + c = box.checkout(CLOSED) + dep = _borrower(box, where(box), c) + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"leaned on by {dep} " in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + box.git("-C", str(dep), "fsck", "--connectivity-only") + + +def test_the_dry_run_names_the_dependents_scope(sweep): + """THE SCOPE IS SAID (SHOULD 352), so a person knows where a dependent + would not be seen.""" + box = sweep.box + result, _ = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert (f" dependents : every repository under {box.w.parent}/ to depth 6, hidden " + f"ones too, and under {box.env['TMPDIR']}/ and {box.home}/ to depth 4; read " + "again under each lock") in result.stdout, result.stdout + + +def test_a_dependents_scan_that_fails_holds_every_candidate(sweep): + """A SCAN THAT CANNOT FINISH HOLDS EVERY CANDIDATE (SHOULD 352): what + leans on them is then unknown. Here a `find` first on PATH fails the + scan with words of its own; both CLOSED lanes' checkouts are held, by + the dry run and by `--yes`, naming the failure.""" + box = sweep.box + c, other = box.checkout(CLOSED), _closed_checkout(box) + real = shutil.which("find") + assert real, "no find on the test's own PATH" + shim = box.fakebin / "find" + shim.write_text("#!/usr/bin/env bash\n" + 'case " $* " in *" -name objects "*) ' + "echo 'find: FAKE: the scan broke' >&2; exit 1 ;; esac\n" + f'exec {real} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode == 0, (result.stdout, result.stderr) + for path in (c, other): + assert rows[str(path)][0] == "hold", rows[str(path)] + assert (f"what leans on it cannot be told: find over {box.w.parent} failed: " + "find: FAKE: the scan broke") in rows[str(path)][1], rows[str(path)] + assert c.is_dir() and other.is_dir() + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root reads a directory whose mode is 000") +def test_a_directory_the_dependents_scan_cannot_read_is_counted_and_holds_nothing(sweep): + """A DIRECTORY THE SCAN CANNOT READ IS COUNTED AND SAID, AND HOLDS NOTHING + (SHOULD 352, by question 2's rule for working directories): another + user's directories are never readable, and holding on them would hold + every checkout on any host that has one. Here a mode-000 directory + beside W is counted once, though two scopes meet it, and the CLOSED + lane's checkout is removed.""" + box = sweep.box + c = box.checkout(CLOSED) + private = box.w.parent / "private" + private.mkdir() + private.chmod(0) + try: + result, rows = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert rows[str(c)][0] == "remove", rows[str(c)] + assert (" (1 director(ies) the dependents scan could not read - another user's, or " + "gone while read - are not looked in)") in result.stdout, result.stdout + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows[str(c)] + finally: + private.chmod(0o755) + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root reads a file whose mode is 000") +@pytest.mark.parametrize("which", [".git file", "alternates file"]) +def test_a_dependents_file_that_cannot_be_read_holds_every_candidate(sweep, which): + """A `.git` OR ALTERNATES FILE THAT CANNOT BE READ HOLDS EVERY CANDIDATE + (lane openRepoTools-3's NIT 356): whose objects it names is unknown. A + `.git` file of mode 000 was skipped before, and the checkout its + directory works in was removed.""" + box = sweep.box + c = box.checkout(CLOSED) + if which == ".git file": + sharer = box.w.parent / "sharer" + sharer.mkdir() + unreadable = sharer / ".git" + unreadable.write_text(f"gitdir: {c}/.git\n", encoding="utf-8") + named = f"the .git file of {sharer} cannot be read" + else: + dep = _borrower(box, box.w.parent / ".lane-worktrees" / "lane-x", c) + unreadable = dep / ".git" / "objects" / "info" / "alternates" + named = f"the alternates file of {dep} cannot be read" + unreadable.chmod(0) + try: + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"what leans on it cannot be told: {named}" in rows[str(c)][1], rows[str(c)] + assert c.is_dir() + finally: + unreadable.chmod(0o644) + + GIT_FILES = [ pytest.param(lambda c, sharer: f"gitdir: {c}/.git\n", id="absolute"), pytest.param(lambda c, sharer: f"gitdir: {c}/.git\r\n", id="CRLF, whose CR git strips"), From 6018b914e3f3eb452a77473bb8693c971a19cb8e Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:05:28 +0000 Subject: [PATCH 32/53] lane-end --sweep-checkouts: the register clones beside W are swept too, under every hold a lane's checkout has and one more, a live process whose `LANES_REPO` names the clone (Brett Heap's ruling on #308's question 1, D37, with lane openRepoTools-3's conditions, #198, slice 4) Brett Heap ruled on #308's question 1 (2026-10-22T12:46:57Z, relayed by lane openRepoTools-3): "yes, have the sweep delete the 139 clones". The per-profile clones beside W, which this build only listed, are now candidates, in one leaves-first order with the checkouts under `-lanes/`, under lane 3's conditions: - 352's wider dependents scan comes first (e04d263), since 47 of the 139 borrow objects through alternates. - Every hold applies: (a)-(c), `.git/lanes-edit/` (351) and the mounts (371). Unpublished content stays until its owner acts. - A NEW HOLD: a live process whose `LANES_REPO` (read from `/proc//environ` by the shell itself, NUL by NUL) names the clone, or a path in it, as spelled or by its real path. A process whose environment cannot be read is counted and said, and holds nothing. With no `/proc`, a clone beside W is held as "cannot tell". `LANE_END_PROC` is the test seam. - The clone the caller's `LANES_REPO` names is kept. A clone beside W is no lane's, so no lane state is asked of it: under its lock, after its fetch, whether it is still a register clone is asked again. It is moved aside beside W, never into `-lanes/`, and an aside left there is named by every later run. `sc_under_home` now takes a path directly in W's parent too, never W or `-lanes/`. The NOTED line names the clones beside W it removed apart from the lane checkouts, each character outside a lane name's written as `?`, and counts those left. The dry run and the act's summaries count both places. The module's box keeps its three clones beside W held by default (an untracked note in the leaf), so every other case is as it was. 13 new cases (leaves first with a name holding a space, a held middle clone, four holds, a mount and a working directory, `LANES_REPO` by path and through a link, no `/proc`, the caller's own `LANES_REPO`, an unfinished removal beside W) and 4 changed ones: red at e04d263, 17 failed; green, 17 passed. With them, the 87 cases of the act, the holds, the dry runs, the locks, the dependents and the NOTED line pass, and hygiene's 253. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit dba02941780592ff253f28151292a1074c57cd1a) --- lane-end | 235 +++++++++++++++++------- tests/test_a23_sweep_checkouts.py | 295 +++++++++++++++++++++++++++--- 2 files changed, 445 insertions(+), 85 deletions(-) diff --git a/lane-end b/lane-end index 9c78d4b..59c1f98 100755 --- a/lane-end +++ b/lane-end @@ -176,17 +176,15 @@ # through its own clone, `-lanes//` (S1), and a # lane that has CLOSED (Amendment 19) leaves its clone # behind, as an old name leaves one beside its new name -# after a rename. Without `--yes` this is a DRY RUN that -# lists every one of them with what holds it, and every -# register clone BESIDE W (a per-profile clone) with what -# it holds; it fetches nothing and changes nothing. -# With `--yes`, run by a PERSON and naming the lane that -# runs it in `LANES_LANE`, it removes each clone under -# `-lanes/` that nothing holds, leaves first, and -# records what it removed in one `NOTED` line of that -# lane's log. A clone beside W is listed and never -# removed by this build, and nothing outside -# `-lanes/` is ever removed +# after a rename; the register clones BESIDE W (the +# per-profile clones) are left from before S1. Without +# `--yes` this is a DRY RUN that lists every one of them +# with what holds it; it fetches nothing and changes +# nothing. With `--yes`, run by a PERSON and naming the +# lane that runs it in `LANES_LANE`, it removes each that +# nothing holds, leaves first, and records what it +# removed in one `NOTED` line of that lane's log. +# Nothing else is ever removed # --reason "" the words the sweep writes into each RETIRED line and # into each row's state cell. `--retire-dormant` only # --yes make the sweep the act. `--retire-dormant` and @@ -293,6 +291,10 @@ # LANE_END_MOUNTINFO the mount table `--sweep-checkouts` reads, in # `/proc/self/mountinfo`'s form (default that file) — the test # seam; one that names no `/` holds every checkout +# LANE_END_PROC the `/proc` whose `/environ` files `--sweep-checkouts` +# reads for the windows' `LANES_REPO` (default /proc) — the +# test seam; one with no `self/environ` holds every clone +# beside W as "cannot tell" # AGENT_PROTOCOL_ROOT dir holding workspace.yaml (default ~/.agents) # OPENREPOTOOLS_BIN_DIR where `--install` put the commands (default # ~/.local/bin) — searched for lanes-edit.sh @@ -945,11 +947,12 @@ fi # lists"*) once that lane is CLOSED too. NEVER W, never a LIVE lane's # checkout, never a lane's that is not CLOSED, never the checkout of the # lane that runs the sweep or of the `LANES_REPO` its caller names, never a -# symbolic link and never anything a link leads to. A CLONE BESIDE W IS LISTED, -# WITH WHAT IT HOLDS, AND NEVER REMOVED BY THIS BUILD: the brief this slice was -# written under says nothing outside `-lanes/` is removed, where S6.2 says -# the per-profile clones are swept too, and the stricter rule is kept until -# lane openRepoTools-3 rules between them. +# symbolic link and never anything a link leads to. AND A REGISTER CLONE +# BESIDE W, which S6.2 sweeps too, under the same holds and one more: a live +# process whose `LANES_REPO` names it. The brief this slice was written under +# said nothing outside `-lanes/` is removed, and Brett Heap ruled between +# the two texts on #308's question 1 (2026-10-22T12:46:57Z): "yes, have the +# sweep delete the 139 clones" (D37). Never W, and never `-lanes/` itself. if [ "$sweep_checkouts" = 1 ]; then [ "$retire" = 0 ] || die "--retire and --sweep-checkouts are two different acts: --retire ends a lane (or names a live fork's act), and --sweep-checkouts removes the register checkouts closed lanes left on disk, ending no lane. Run one of them. Nothing was written." 2 [ "$force" = 0 ] || die "--force ends a lane that still holds something; it has no meaning for the sweep of register checkouts, which removes only what nothing holds. Nothing was removed." 2 @@ -1326,17 +1329,34 @@ SC_IDX done fi - # THE CLONES BESIDE W: listed, never removed by this build (above). - sc_beside=() + # THE CLONES BESIDE W, S6.2's per-profile clones, are candidates since + # Brett Heap's ruling on #308's question 1 (D37). They are no lane's, so no + # lane state is asked of them: what keeps one is every hold a lane's + # checkout has, and a window whose `LANES_REPO` names it (below). The one + # the caller's `LANES_REPO` names is kept, as under `-lanes/`. The aside + # of a removal of one that did not finish stands beside W, and every run + # names it, as it names those under `-lanes/`. Anything beside W that is + # not a register clone is not looked at. + sc_n_beside=0 if ls -A -- "$sc_root" >/dev/null 2>&1; then + for sc_e in "$sc_root"/.[!.]* "$sc_root"/..?*; do + if [ ! -e "$sc_e" ] && [ ! -L "$sc_e" ]; then continue; fi + case "${sc_e##*/}" in .?*.removing-*) sc_leftovers+=("$sc_e") ;; esac + done for sc_e in "$sc_root"/*; do if [ -L "$sc_e" ] || [ ! -d "$sc_e" ]; then continue; fi if [ "$sc_e" = "$sc_w" ] || [ "$sc_e" = "$sc_home" ]; then continue; fi if [ ! -d "$sc_e/.git" ] || [ -L "$sc_e/.git" ]; then continue; fi - if sc_is_register_clone "$sc_e"; then sc_beside+=("$sc_e"); fi + if ! sc_is_register_clone "$sc_e"; then continue; fi + sc_n_beside=$((sc_n_beside + 1)) + if [ -n "$sc_xrepo" ] && [ "$sc_e" = "$sc_xrepo" ]; then + sc_keep "$sc_e" "the clone beside W that LANES_REPO names, which a write of this window may go through" + continue + fi + sc_cand "$sc_e" "" "a clone of the register beside W (S6.2's per-profile clones)" done else - note "$sc_root, the directory W stands in, cannot be listed, so no clone beside W is listed. Nothing outside $sc_home/ is ever removed." + note "$sc_root, the directory W stands in, cannot be listed, so no clone beside W is looked at, and none is removed." fi # ------------------------------------------------------------ the holds @@ -1373,7 +1393,8 @@ SC_IDX # the work; # and a file system mounted at or under it (lane openRepoTools-3's # SHOULD 371), whose files are not the checkout's, or a mount table - # that cannot be read. + # that cannot be read; and a live process whose `LANES_REPO` names + # it, a window that writes through it (D37, below). SC_HOLD=""; SC_IGN="" sc_hold() { SC_HOLD="${SC_HOLD:+$SC_HOLD; }$1"; } sc_first() { # — the count and the first three, as `N (a b c)` @@ -1704,6 +1725,64 @@ $sc_mount_pts SMI } + # A WINDOW THAT WRITES THROUGH IT (lane openRepoTools-3's condition on the + # ruling, D37): a live process whose environment's `LANES_REPO` names the + # clone, or a path in it, as spelled or by its real path. S6.2 honours a + # caller's `LANES_REPO`, and since 10-19 the launcher exports one into each + # lane window, so a window opened on a clone writes the register through it + # while it stands. `/proc//environ` is read: a process's environment + # as it began, which is where the launcher's export is. A process whose + # environment cannot be read (another user's, or one that ended meanwhile) + # is counted and said, and holds nothing, as a working directory that + # cannot be read does not. With no `/proc` a clone beside W, which no + # lane's state keeps, is held as "cannot tell"; a lane's checkout is kept + # by its lane's state, asked again under the lock. `LANE_END_PROC` names + # another `/proc`: the test seam. + sc_repos=""; sc_repo_mode=none; sc_repo_unread=0 + sc_repo_scan() { + sc_repos=""; sc_repo_unread=0; sc_repo_mode=none + srs_proc="${LANE_END_PROC:-/proc}" + if [ ! -d "$srs_proc/self" ] || [ ! -r "$srs_proc/self/environ" ]; then return 0; fi + sc_repo_mode=proc + for srs_d in "$srs_proc"/[0-9]*; do + # Read by the shell itself, NUL by NUL: no process is started per pid. + srs_v=""; srs_ok=0 + { while IFS= read -r -d '' srs_kv; do + case "$srs_kv" in LANES_REPO=*) srs_v="${srs_kv#LANES_REPO=}"; break ;; esac + done; srs_ok=1; } 2>/dev/null < "$srs_d/environ" || : + if [ "$srs_ok" = 0 ]; then + if [ -d "$srs_d" ]; then sc_repo_unread=$((sc_repo_unread + 1)); fi + continue + fi + [ -n "$srs_v" ] || continue + case "$srs_v" in + /*) : ;; + *) srs_c="$(readlink "$srs_d/cwd" 2>/dev/null)" || srs_c="" + if [ -z "$srs_c" ]; then continue; fi + srs_v="$srs_c/$srs_v" ;; + esac + sc_repos="$sc_repos${srs_d##*/}$sc_tab$srs_v +" + srs_r="$(sc_real "$srs_v")" + if [ -n "$srs_r" ] && [ "$srs_r" != "$srs_v" ]; then + sc_repos="$sc_repos${srs_d##*/}$sc_tab$srs_r +" + fi + done + } + sc_repo_inside() { # — the pids whose LANES_REPO is at or under it + sri_out="" + while IFS="$sc_tab" read -r sri_pid sri_v; do + [ -n "$sri_pid" ] || continue + case "$sri_v" in "$1" | "$1"/*) : ;; *) continue ;; esac + case " $sri_out " in *" $sri_pid "*) continue ;; esac + sri_out="${sri_out:+$sri_out }$sri_pid" + done < <1 when this run holds its lock> [] sc_holds "$1" "$2" @@ -1718,6 +1797,11 @@ SMI sjh_pids="$(sc_cwd_inside "$1")" if [ -n "$sjh_pids" ]; then sc_hold "a live process works in it (pid $sjh_pids)"; fi ;; esac + sjh_pids="$(sc_repo_inside "$1")" + if [ -n "$sjh_pids" ]; then sc_hold "the LANES_REPO of a live process names it (pid $sjh_pids): a window writes the register through it"; fi + if [ "$sc_repo_mode" = none ] && [ "${1%/*}" = "$sc_root" ]; then + sc_hold "whether a window's LANES_REPO names it cannot be told: there is no /proc here to read a process's environment from" + fi case "$sc_mount_mode" in none) sc_hold "whether a file system is mounted in it cannot be told: no mount table that names / could be read (${LANE_END_MOUNTINFO:-/proc/self/mountinfo}, or mount)" ;; *) @@ -1729,12 +1813,17 @@ SMI return 0 } - # NEVER A PATH THAT IS NOT DIRECTLY UNDER `-lanes/`, by its real parent, - # asked again at the moment of each act on it. + # NEVER A PATH THAT IS NOT DIRECTLY UNDER `-lanes/` OR DIRECTLY IN W'S + # PARENT, by its real parent, and never W or `-lanes/` itself, asked + # again at the moment of each act on it. sc_under_home() { # - suh_n="${1##*/}" + suh_n="${1##*/}"; suh_p="${1%/*}" case "$suh_n" in '' | . | ..) return 1 ;; esac - [ "${1%/*}" = "$sc_home" ] && [ "$(sc_real "${1%/*}")" = "$sc_home" ] + case "$suh_p" in + "$sc_home") [ "$(sc_real "$suh_p")" = "$sc_home" ] ;; + "$sc_root") [ "$1" != "$sc_w" ] && [ "$1" != "$sc_home" ] && [ "$(sc_real "$suh_p")" = "$sc_root" ] ;; + *) return 1 ;; + esac } # THE LOCK'S PID LINE, in `lanes-edit.sh`'s own form, ` @@ -1889,10 +1978,13 @@ SS_ROW fi printf ' dependents : every repository under %s/ to depth 6, hidden ones too, and under %s/ and %s/ to depth 4; read again under each lock\n' \ "${sc_dep_named[0]}" "${sc_dep_named[1]}" "${sc_dep_named[2]}" - sc_cwd_scan; sc_mount_scan + sc_cwd_scan; sc_mount_scan; sc_repo_scan if [ "$sc_cwd_unread" -gt 0 ]; then printf ' (%s process(es) whose working directory could not be read - another user'"'"'s, or ended while read - do not hold anything)\n' "$sc_cwd_unread" fi + if [ "$sc_repo_unread" -gt 0 ]; then + printf ' (%s process(es) whose environment could not be read - another user'"'"'s, or ended while read - are not asked which LANES_REPO they name)\n' "$sc_repo_unread" + fi sc_dep_scan if [ "$sc_dep_unread" -gt 0 ]; then printf ' (%s director(ies) the dependents scan could not read - another user'"'"'s, or gone while read - are not looked in)\n' "$sc_dep_unread" @@ -1964,28 +2056,48 @@ SLU fi fi - printf '\n under %s/:\n' "$sc_home" - # `sc_gone`: the paths this run removed (a dry run: would remove), one per line. + printf '\n under %s/, and beside W in %s/:\n' "$sc_home" "$sc_root" + # `sc_gone`: the paths this run removed (a dry run: would remove), one per + # line. `sc_n_removed`, `sc_n_partial` and `sc_n_removable` count both + # places, and `sc_nb_*` the clones beside W among them; `sc_n_held` counts + # `-lanes/` alone, and `sc_nb_held` the clones beside W. sc_gone=""; sc_n_removed=0; sc_n_partial=0; sc_n_held=0; sc_n_removable=0; sc_partial="" - sc_removed_names=""; sc_held_names=""; sc_partial_names="" + sc_nb_removed=0; sc_nb_partial=0; sc_nb_held=0; sc_nb_removable=0 + sc_removed_names=""; sc_held_names=""; sc_partial_names=""; sc_bremoved_names="" + sc_is_beside() { [ "${1%/*}" = "$sc_root" ]; } + # A NAME AS THE NOTED LINE SPELLS IT: a clone beside W may be named + # anything, so each character outside letters, digits, `.`, `_` and `-` is + # written as `?`. + sc_said_name() { # + case "$1" in + *[!A-Za-z0-9._-]*) printf '%s' "$1" | LC_ALL=C tr -c 'A-Za-z0-9._-' '?' ;; + *) printf '%s' "$1" ;; + esac + } sc_say_held() { # printf ' hold %s %s; holds: %s\n' "$1" "$2" "$3" + if sc_is_beside "$1"; then sc_nb_held=$((sc_nb_held + 1)); return 0; fi sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${1##*/}" } # THE NOTED LINE'S TEXT, from what this run has done so far, and the two # commands that finish its record by hand. The paths are spelled from the # projects root, `-lanes/`, never from the host's own root, and - # every name in the line is one this sweep already holds to letters, - # digits, `.`, `_` and `-`, so the free text's own rules (no ` — `, no - # newline, no leading arrow) hold for it. + # every name in the line is one this sweep holds to letters, digits, `.`, + # `_` and `-` (a lane's by the lane-name rule, a clone beside W's by + # `sc_said_name`), so the free text's own rules (no ` — `, no newline, no + # leading arrow) hold for it. The clones beside W removed are named apart, + # and those left are counted. sc_rel="${sc_home##*/}" case "$sc_rel" in '' | *[!A-Za-z0-9._-]*) sc_rel="-lanes" ;; esac sc_noted_text() { # [] - snt_t="sweep-checkouts (Amendment 23 S6.2): removed $sc_n_removed register checkout(s) under $sc_rel/ that held nothing unpublished" + snt_t="sweep-checkouts (Amendment 23 S6.2): removed $((sc_n_removed - sc_nb_removed)) register checkout(s) under $sc_rel/ that held nothing unpublished" if [ -n "$sc_removed_names" ]; then snt_t="$snt_t: $sc_removed_names"; fi if [ -n "$sc_partial_names" ]; then snt_t="$snt_t; moved aside and not all removed: $sc_partial_names"; fi if [ -n "$sc_held_names" ]; then snt_t="$snt_t; held and left: $sc_held_names"; fi - snt_t="$snt_t; ${#sc_beside[@]} clone(s) beside W listed and left" + if [ "$sc_nb_removed" -gt 0 ]; then + snt_t="$snt_t; removed $sc_nb_removed clone(s) beside W that held nothing unpublished: $sc_bremoved_names" + fi + snt_t="$snt_t; $((sc_n_beside - sc_nb_removed - sc_nb_partial)) clone(s) beside W left" if [ -n "${1-}" ]; then snt_t="$snt_t; $1"; fi printf '%s' "$snt_t" } @@ -2012,7 +2124,7 @@ SLU sc_say_partial() { if [ -n "$sc_partial" ] && [ "$sc_partial_said" = 0 ]; then sc_partial_said=1 - note "the removal of $sc_partial did not finish (above): what is left is no checkout, under a hidden name, so the next write of that lane clones afresh. Remove it by hand once you have looked at why; every later run names it again, with the command." + note "the removal of $sc_partial did not finish (above): what is left is no checkout, under a hidden name, and the next write of a lane whose checkout it was clones afresh. Remove it by hand once you have looked at why; every later run names it again, with the command." fi } # A SIGNAL NEVER CUTS A REMOVAL IN TWO, AND ONE THAT ENDS THE ACT LEAVES @@ -2053,7 +2165,8 @@ SLU sc_c="${sc_cands[$sc_i]}"; sc_why="${sc_cand_why[$sc_i]}" sc_l="${sc_cand_lane[$sc_i]}"; sc_new="${sc_cand_new[$sc_i]}" if ! sc_under_home "$sc_c"; then - printf ' hold %s %s; it is not directly under %s/, and nothing else is ever removed\n' "$sc_c" "$sc_why" "$sc_home" + printf ' hold %s %s; it is not directly under %s/ or in %s/, and nothing else is ever removed\n' "$sc_c" "$sc_why" "$sc_home" "$sc_root" + if sc_is_beside "$sc_c"; then sc_nb_held=$((sc_nb_held + 1)); continue; fi sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${sc_c##*/}"; continue fi if [ "$sweep_yes" != 1 ]; then @@ -2064,15 +2177,18 @@ SLU printf ' remove %s %s%s\n' "$sc_c" "$sc_why" "${SC_IGN:+; $SC_IGN}" sc_n_removable=$((sc_n_removable + 1)); sc_gone="$sc_gone$sc_c " + if sc_is_beside "$sc_c"; then sc_nb_removable=$((sc_nb_removable + 1)); fi fi continue fi # THE ACT, ONE CHECKOUT AT A TIME, every decision made again under its # own writers' lock after a fetch: the lock first, so no writer of it is - # in it; then its fetch, the lane's state asked again, every hold, what - # leans on it and the working directories read again; then, and only - # then, it is moved aside under `-lanes/` — so its S1 path is absent at - # once and the next write there clones afresh — and the aside removed. + # in it; then its fetch, the lane's state asked again (a clone beside W + # is no lane's: whether it is still a register clone is what is asked), + # every hold, what leans on it, the working directories and the windows' + # `LANES_REPO` read again; then, and only then, it is moved aside in the + # directory it stands in — so its path is absent at once, and the next + # write of a lane there clones afresh — and the aside removed. if ! sc_lock_take "$sc_c"; then sc_say_held "$sc_c" "$sc_why" "$SC_WHY"; continue fi @@ -2080,12 +2196,12 @@ SLU sc_lock_beat if ! sc_fetch "$sc_c"; then sc_act_hold="its fetch of origin $sc_branch failed or timed out, and nothing is decided on an old origin/$sc_branch" - elif ! { sc_lock_beat; sc_still "$sc_c" "$sc_l" "$sc_new"; }; then + elif [ -n "$sc_l" ] && ! { sc_lock_beat; sc_still "$sc_c" "$sc_l" "$sc_new"; }; then sc_act_hold="$SC_WHY" elif ! sc_is_register_clone "$sc_c"; then sc_act_hold="it is no longer what made it a candidate: $SC_WHY" else - sc_dep_scan; sc_cwd_scan; sc_mount_scan + sc_dep_scan; sc_cwd_scan; sc_mount_scan; sc_repo_scan sc_judge_holds "$sc_c" 1 sc_act_hold="$SC_HOLD"; sc_act_ign="$SC_IGN" fi @@ -2100,7 +2216,7 @@ SLU sc_lock_drop sc_say_held "$sc_c" "$sc_why" "$sc_act_hold"; continue fi - sc_aside="$sc_home/.${sc_c##*/}.removing-$sc_utc-$$" + sc_aside="${sc_c%/*}/.${sc_c##*/}.removing-$sc_utc-$$" sc_sig_defer if ! sc_under_home "$sc_c" || [ -e "$sc_aside" ] || [ -L "$sc_aside" ] || ! ( trap '' INT TERM; exec mv -- "$sc_c" "$sc_aside" ) 2>/dev/null; then @@ -2115,12 +2231,18 @@ SLU if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then printf ' PARTLY %s %s; moved aside to %s, which could not all be removed%s\n' "$sc_c" "$sc_why" "$sc_aside" "${sc_act_ign:+; $sc_act_ign}" sc_partial="${sc_partial:+$sc_partial }$sc_aside" - sc_partial_names="${sc_partial_names:+$sc_partial_names }${sc_aside##*/}" + sc_partial_names="${sc_partial_names:+$sc_partial_names }$(sc_said_name "${sc_aside##*/}")" sc_n_partial=$((sc_n_partial + 1)) + if sc_is_beside "$sc_c"; then sc_nb_partial=$((sc_nb_partial + 1)); fi else printf ' removed %s %s%s\n' "$sc_c" "$sc_why" "${sc_act_ign:+; $sc_act_ign}" - sc_removed_names="${sc_removed_names:+$sc_removed_names }${sc_c##*/}" sc_n_removed=$((sc_n_removed + 1)) + if sc_is_beside "$sc_c"; then + sc_nb_removed=$((sc_nb_removed + 1)) + sc_bremoved_names="${sc_bremoved_names:+$sc_bremoved_names }$(sc_said_name "${sc_c##*/}")" + else + sc_removed_names="${sc_removed_names:+$sc_removed_names }${sc_c##*/}" + fi fi sc_gone="$sc_gone$sc_c " @@ -2133,24 +2255,10 @@ SLU done if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ] && [ "${#sc_leftovers[@]}" = 0 ]; then printf ' (nothing)\n'; fi - printf '\n beside W (Amendment 23 S6.2), listed and never removed by this build:\n' - for sc_e in ${sc_beside[@]+"${sc_beside[@]}"}; do - if [ "$sweep_yes" = 1 ]; then - printf ' beside %s left\n' "$sc_e" - continue - fi - sc_judge_holds "$sc_e" 0 "$sc_gone" - if [ -n "$SC_HOLD" ]; then - printf ' beside %s holds: %s\n' "$sc_e" "$SC_HOLD" - else - printf ' beside %s nothing holds it%s\n' "$sc_e" "${SC_IGN:+; $SC_IGN}" - fi - done - if [ "${#sc_beside[@]}" = 0 ]; then printf ' (none)\n'; fi if [ "$sweep_yes" != 1 ]; then - printf '\n %s removable, %s held under %s/; %s clone(s) beside W listed.\n' \ - "$sc_n_removable" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" + printf '\n %s removable, %s held under %s/; %s removable, %s held of the %s clone(s) beside W.\n' \ + "$((sc_n_removable - sc_nb_removable))" "$sc_n_held" "$sc_home" "$sc_nb_removable" "$sc_nb_held" "$sc_n_beside" if [ "$sc_n_removable" -gt 0 ]; then printf '\n DRY RUN — nothing was removed. Make it the act with:\n' printf ' LANES_LANE= %s --sweep-checkouts --yes\n' "$prog" @@ -2162,8 +2270,9 @@ SLU fi exit 0 fi - printf '\n %s removed, %s moved aside and not all removed, %s held under %s/; %s clone(s) beside W left.\n' \ - "$sc_n_removed" "$sc_n_partial" "$sc_n_held" "$sc_home" "${#sc_beside[@]}" + printf '\n %s removed, %s moved aside and not all removed, %s held under %s/; %s removed, %s moved aside and not all removed, %s held of the %s clone(s) beside W.\n' \ + "$((sc_n_removed - sc_nb_removed))" "$((sc_n_partial - sc_nb_partial))" "$sc_n_held" "$sc_home" \ + "$sc_nb_removed" "$sc_nb_partial" "$sc_nb_held" "$sc_n_beside" # THE NOTED LINE (S6.2; S2 lists the sweep among the writers): ONE line in # the runner's own log, naming what was removed and what was held, written # by `lanes-edit.sh log` and so through the runner's own checkout under S3 diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 6e5b592..f8d110e 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -6,9 +6,10 @@ (S1). A lane that has CLOSED (Amendment 19) leaves its clone behind, and so does an old name beside its new one after a rename. The sweep has Amendment 19's shape: a dry run that lists them and removes nothing, then a person's -`--yes`, which needs `LANES_LANE`. The per-profile clones BESIDE W are listed -with what they hold and never removed by this build: nothing outside -`-lanes/` is ever removed. +`--yes`, which needs `LANES_LANE`. The per-profile clones BESIDE W are swept +too, under the same holds and one more, a window whose `LANES_REPO` names +one (Brett Heap's ruling on #308's question 1, D37); nothing else beside W, +and nothing anywhere else, is ever removed. EVERY RUN HERE IS THE REAL `lane-end` AND THE REAL `lanes-edit.sh` IN A SANDBOX: `tests/a23_box.py`'s `RegisterBox` (a bare origin, W cloned from it, @@ -53,6 +54,8 @@ for n, lane in ((1, RUNNER), (2, CLOSED), (3, LIVE), (4, PAUSED), (5, CHAIN[0]), (6, CHAIN[1]), (7, CHAIN[2]), (9, NEW))} PROFILE_CLONES = ("brett-wip-team01-laneA", "brett-wip-team02-laneA", "brett-wip-team03-laneA") +PROFILE_NOTE = "profile-note.txt" # untracked in the leaf clone beside W: it holds the chain +BESIDE_WHY = "a clone of the register beside W (S6.2's per-profile clones)" def _row(lane: str, state: str) -> str: @@ -76,7 +79,9 @@ class Sweep(RegisterBox): ENDED in the register and the logs, and have no checkout until `_chain` makes them. Beside W: three per-profile clones, each borrowing from the one before it and the first - from W (c3 → c2 → c1 → W), and a clone of another repository.""" + from W (c3 → c2 → c1 → W), and a clone of another repository. The leaf, + c3, holds an untracked note, so all three are held unless a case takes + it away (`_free_profile_clones`).""" def __init__(self, root: Path) -> None: lanes = (RUNNER, CLOSED, LIVE, PAUSED, NEW) @@ -110,6 +115,7 @@ def __init__(self, root: Path) -> None: for name in PROFILE_CLONES: previous = self.clone(projects / name, reference=previous) self.profile_clones.append(previous) + self.write(previous, PROFILE_NOTE, "a person's note in this profile's clone\n") other = root / "other.git" self.git("init", "-q", "--bare", "-b", "main", str(other)) self.clone(projects / "otherrepo", origin=other) @@ -445,17 +451,25 @@ def test_a_W_that_is_a_lane_checkout_is_refused(sweep): assert box.everything() == before -def test_clones_beside_W_are_listed_and_never_candidates(sweep): - """S6.2's per-profile clones are listed, and nothing outside `-lanes/` - is a candidate. A clone of another repository is not listed at all.""" +def test_clones_beside_W_are_judged_by_what_they_hold(sweep): + """S6.2's PER-PROFILE CLONES ARE CANDIDATES (Brett Heap's ruling on #308's + question 1, D37), each judged by the holds: the leaf c3 holds an untracked + note, c2 is leaned on by c3, and c1 by c2, so the dry run holds all three + and removes nothing beside W. A clone of another repository is not listed + at all, and neither is W or `-lanes/`.""" box = sweep.box + c1, c2, c3 = box.profile_clones + before = box.everything() result, rows = sweep() - assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones}, rows - assert str(box.w.parent / "otherrepo") not in rows, rows - judged = _tagged(rows, "remove") | _tagged(rows, "hold") - assert judged, rows - assert all(Path(p).parent == box.lanes_root for p in judged), rows - assert "listed and never removed by this build" in result.stdout + assert result.returncode == 3, (result.stdout, result.stderr) + assert rows[str(c3)] == ("hold", f"{BESIDE_WHY}; holds: untracked: 1 ({PROFILE_NOTE})"), rows + assert rows[str(c2)] == ("hold", f"{BESIDE_WHY}; holds: leaned on by {c3} (its alternates file names this clone's objects)"), rows + assert rows[str(c1)] == ("hold", f"{BESIDE_WHY}; holds: leaned on by {c2} (its alternates file names this clone's objects)"), rows + for never in (box.w.parent / "otherrepo", box.w, box.lanes_root): + assert str(never) not in rows, (never, rows) + assert " 1 removable, 0 held under " in result.stdout, result.stdout + assert "; 0 removable, 3 held of the 3 clone(s) beside W." in result.stdout, result.stdout + assert box.everything() == before # ------------------------------------------------------- the act, and holds @@ -493,7 +507,7 @@ def test_closed_lane_checkout_removed_live_kept(sweep, tmp_path): assert box.git("-C", runner, "rev-parse", "HEAD").strip() == box.origin_head(), \ "the runner's checkout moved only to the NOTED commit it pushed" assert after[runner][1:] == before[runner][1:], "and holds nothing new" - assert _tagged(rows, "beside") == {str(c) for c in box.profile_clones} + assert all(rows[str(c)][0] == "hold" for c in box.profile_clones), rows _no_aside(box) assert not list(box.lanes_root.glob("*/lanes/.lanes-edit.lock")), "every lock it took is released" @@ -1577,18 +1591,21 @@ def test_a_register_unreadable_at_the_recheck_keeps_the_checkout(sweep): assert c.is_dir() -def test_nothing_outside_W_lanes_is_removed(sweep): - """The clones beside W are listed with what holds them, and `--yes` removes - none of them: nothing outside `-lanes/` is ever removed.""" +def test_held_clones_beside_W_are_left_by_yes(sweep): + """`--yes` takes each clone beside W under its lock after a fetch, as a + lane's checkout, and leaves every one that something holds exactly as it + was: the leaf by its untracked note, the other two by what leans on them. + Its NOTED line counts them as left.""" box = sweep.box before = {str(c): box.everything()[str(c)] for c in box.profile_clones} - _, rows = sweep() - assert rows[str(box.profile_clones[0])] == ("beside", f"holds: leaned on by {box.profile_clones[1]} (its alternates file names this clone's objects)"), rows - assert rows[str(box.profile_clones[2])][1].startswith("nothing holds it"), rows result, rows = sweep("--yes", LANES_LANE=RUNNER) assert result.returncode == 0, (result.stdout, result.stderr) assert {str(c): box.everything()[str(c)] for c in box.profile_clones} == before - assert all(rows[str(c)] == ("beside", "left") for c in box.profile_clones), rows + assert all(rows[str(c)][0] == "hold" for c in box.profile_clones), rows + assert "; 0 removed, 0 moved aside and not all removed, 3 held of the 3 clone(s) beside W." in result.stdout + assert _noted_on_origin(box)[0].endswith("; 3 clone(s) beside W left"), _noted_on_origin(box) + assert not list(box.w.parent.glob("*/lanes/.lanes-edit.lock")), "every lock it took is released" + assert not list(box.w.parent.glob(".*.removing-*")), "no aside is left beside W" # ------------------------------------------------- leaves first, and chains @@ -1880,7 +1897,7 @@ def test_noted_line_written_through_runner_checkout(sweep): assert noted[0].endswith( "sweep-checkouts (Amendment 23 S6.2): removed 1 register checkout(s) under " f"brett-wip-lanes/ that held nothing unpublished: {CLOSED}; " - "3 clone(s) beside W listed and left"), noted + "3 clone(s) beside W left"), noted assert str(box.root) not in noted[0], "no host path is written into the log" assert box.git("-C", str(runner), "rev-parse", "HEAD").strip() == box.origin_head(), \ "the NOTED commit is the runner's checkout's own" @@ -2319,3 +2336,237 @@ def test_a_directory_sharing_Ws_git_directory_is_kept_as_W(sweep): assert rows[str(d)] == ("keep", "W itself, the readers' copy"), rows[str(d)] assert d.is_dir() assert box.w_state() == w_before + + +# ------------------------------------------------ the clones beside W (D37) + +def _free_profile_clones(box: Sweep) -> None: + """The leaf clone beside W loses its untracked note: nothing holds the + chain c3 → c2 → c1 any more.""" + (box.profile_clones[2] / PROFILE_NOTE).unlink() + + +def test_clones_beside_W_are_removed_leaves_first(sweep, tmp_path): + """BRETT HEAP'S RULING ON #308's QUESTION 1 (D37): "yes, have the sweep + delete the 139 clones". With nothing holding them, the chain beside W + (c3 → c2 → c1 → W) is removable in ONE run, leaves first, in one order + with the lane checkouts', and `--yes` removes it in that order. A clone + whose name holds a character outside a lane name's is removed too, and + the NOTED line writes `?` for that character. W, `-lanes/` and the + clone of another repository stay, and no aside is left beside W.""" + box = sweep.box + _free_profile_clones(box) + odd = box.clone(box.w.parent / "brett-wip team04", reference=box.w) + chain = list(reversed(box.profile_clones)) + result, rows = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert _entries(result.stdout, chain) == [("remove", str(c)) for c in chain], result.stdout + assert rows[str(odd)] == ("remove", BESIDE_WHY), rows + assert "; 4 removable, 0 held of the 4 clone(s) beside W." in result.stdout, result.stdout + other = box.everything()[str(box.w.parent / "otherrepo")] + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert _entries(result.stdout, chain) == [("removed", str(c)) for c in chain], result.stdout + assert rows[str(odd)][0] == "removed", rows + gone = [*chain, odd] + _assert_under(Path(os.path.realpath(tmp_path)), gone) + for c in gone: + assert not os.path.lexists(c), c + box.git("-C", str(box.w), "fsck", "--connectivity-only") # W, which c1 borrowed from, is whole + assert box.w.is_dir() and box.lanes_root.is_dir() + assert box.everything()[str(box.w.parent / "otherrepo")] == other + assert not list(box.w.parent.glob(".*.removing-*")), "no aside is left beside W" + noted = _noted_on_origin(box) + assert len(noted) == 1, noted + said = noted[0].split("; removed 4 clone(s) beside W that held nothing unpublished: ", 1)[1] + names, _, tail = said.partition("; ") + names = names.split(" ") + assert sorted(names) == sorted([c.name for c in chain] + ["brett-wip?team04"]), noted + assert [n for n in names if n in PROFILE_CLONES] == [c.name for c in chain], noted + assert tail == "0 clone(s) beside W left", noted + + +def test_a_held_clone_beside_W_keeps_what_it_borrows_from(sweep): + """S6.2 (b), BESIDE W: the middle clone, c2, holds a commit origin lacks. + The leaf, c3, which borrows from it, goes; c2 is held by its commit, and + c1, which c2 borrows from, by c2 — in the dry run and under `--yes`.""" + box = sweep.box + c1, c2, c3 = box.profile_clones + _free_profile_clones(box) + _commit_unpushed(box, c2) + for args, env, gone in ((), {}, "remove"), (("--yes",), {"LANES_LANE": RUNNER}, "removed"): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c3)][0] == gone, rows[str(c3)] + assert rows[str(c2)][0] == "hold", rows[str(c2)] + assert "commit(s) origin lacks by content" in rows[str(c2)][1], rows[str(c2)] + assert rows[str(c1)][0] == "hold", rows[str(c1)] + assert f"leaned on by {c2} " in rows[str(c1)][1], rows[str(c1)] + assert not os.path.lexists(c3) + assert c2.is_dir() and c1.is_dir() + + +def _kept_edit(box: Sweep, c: Path) -> None: + box.write(c, ".git/lanes-edit/hand-edit-20261022T104947Z-1.diff", "a hand edit kept\n") + + +BESIDE_HOLDS = [ + pytest.param(_commit_unpushed, "commit(s) origin lacks by content", id="a + on main"), + pytest.param(_stash, "a stash of 1 entr(ies)", id="a stash"), + pytest.param(_untracked_elsewhere, "untracked: 1 (notes.txt)", id="untracked"), + pytest.param(_kept_edit, "lanes-edit.sh keeps 1 (hand-edit-20261022T104947Z-1.diff) in " + ".git/lanes-edit/", id="a hand edit lanes-edit.sh kept"), +] + + +@pytest.mark.parametrize("make,said", BESIDE_HOLDS) +def test_holds_beside_W(sweep, make, said): + """EVERY HOLD OF A LANE'S CHECKOUT HOLDS A CLONE BESIDE W (D37, lane + openRepoTools-3's conditions): unpublished content stays until its owner + acts. The leaf clone, freed of its note, is given one hold; the dry run + and `--yes` hold it, naming it, and leave it as it was.""" + box = sweep.box + c3 = box.profile_clones[2] + _free_profile_clones(box) + make(box, c3) + before = box.everything()[str(c3)] + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c3)][0] == "hold", rows[str(c3)] + assert said in rows[str(c3)][1], rows[str(c3)] + assert box.everything()[str(c3)] == before, "the held clone is as it was" + assert not (c3 / "lanes" / ".lanes-edit.lock").exists(), "the sweep's own lock is released" + + +@pytest.mark.parametrize("what", ["a mount", "a working directory"]) +def test_a_mount_or_a_process_in_a_clone_beside_W_holds_it(sweep, what): + """(c) AND SHOULD 371 BESIDE W: a process working in the leaf clone, or + a file system mounted in it (the mount table seam), holds it.""" + box = sweep.box + c3 = box.profile_clones[2] + _free_profile_clones(box) + env, sleeper = {}, None + if what == "a mount": + (c3 / ".git" / "mnt").mkdir() + env["LANE_END_MOUNTINFO"] = _mount_table(box, c3 / ".git" / "mnt") + said = f"a file system is mounted at or under it: 1 ({c3}/.git/mnt)" + else: + sleeper = subprocess.Popen(["sleep", "600"], cwd=str(c3 / "lanes")) + said = f"a live process works in it (pid {sleeper.pid})" + try: + for args, more in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env, **more) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c3)][0] == "hold", rows[str(c3)] + assert said in rows[str(c3)][1], rows[str(c3)] + assert c3.is_dir() + finally: + if sleeper is not None: + sleeper.kill() + sleeper.wait() + + +@pytest.mark.parametrize("spelled", ["its own path", "a link to it"]) +def test_a_window_whose_LANES_REPO_names_a_clone_beside_W_holds_it(sweep, spelled): + """A WINDOW THAT WRITES THROUGH A CLONE HOLDS IT (lane openRepoTools-3's + condition on the ruling, D37): a live process whose environment's + `LANES_REPO` names the leaf clone, by its own path or through a link to + it, holds it, by the dry run and `--yes`, naming the process. The process + is a `sleep` started with that `LANES_REPO`, killed by its PID.""" + box = sweep.box + c3 = box.profile_clones[2] + _free_profile_clones(box) + named = c3 + if spelled == "a link to it": + named = box.root / "link-to-c3" + named.symlink_to(c3) + sleeper = subprocess.Popen(["sleep", "600"], cwd=str(box.root), + env=dict(box.env, LANES_REPO=str(named))) + try: + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c3)][0] == "hold", rows[str(c3)] + assert (f"the LANES_REPO of a live process names it (pid {sleeper.pid}): a window " + "writes the register through it") in rows[str(c3)][1], rows[str(c3)] + assert c3.is_dir() + finally: + sleeper.kill() + sleeper.wait() + + +def test_with_no_proc_whether_a_window_names_a_clone_beside_W_cannot_be_told(sweep): + """WITH NO `/proc`, A CLONE BESIDE W IS HELD AS "CANNOT TELL" (D37): no + lane's state keeps it, and no window's `LANES_REPO` can be read. + `LANE_END_PROC`, the seam, names an empty directory. The CLOSED lane's + checkout, which its lane's state keeps, is still removed.""" + box = sweep.box + _free_profile_clones(box) + empty = box.root / "no-proc" + empty.mkdir() + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, LANE_END_PROC=str(empty), **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(box.checkout(CLOSED))][0] in ("remove", "removed"), rows + for c in box.profile_clones: + assert rows[str(c)][0] == "hold", rows[str(c)] + assert ("whether a window's LANES_REPO names it cannot be told: there is no /proc " + "here to read a process's environment from") in rows[str(c)][1], rows[str(c)] + assert all(c.is_dir() for c in box.profile_clones) + + +def test_the_clone_beside_W_the_callers_LANES_REPO_names_is_kept(sweep): + """THE CLONE A CALLER'S `LANES_REPO` NAMES IS KEPT, beside W as under + `-lanes/`: a write of this window may go through it. The clone it + borrows from is held by it.""" + box = sweep.box + c1, c2, c3 = box.profile_clones + _free_profile_clones(box) + result, rows = sweep(LANES_REPO=str(c3)) + assert result.returncode == 3, (result.stdout, result.stderr) + assert rows[str(c3)] == ("keep", "the clone beside W that LANES_REPO names, which a write " + "of this window may go through"), rows + assert rows[str(c2)][0] == "hold", rows[str(c2)] + assert f"leaned on by {c3} " in rows[str(c2)][1], rows[str(c2)] + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root removes what a read-only directory holds") +def test_a_removal_beside_W_that_does_not_finish_leaves_its_aside_beside_W(sweep): + """A CLONE BESIDE W IS MOVED ASIDE BESIDE W, NEVER INTO `-lanes/`, and + an aside left there is named by every later run (SHOULD 372's rule, + D37). The leaf clone holds an ignored read-only directory: it is moved + aside, its removal stops partway, it is named PARTLY, the run exits 1, + its aside's name is in the NOTED line, and the next dry run lists the + aside as `leftover`, with the command that removes it.""" + box = sweep.box + c3 = box.profile_clones[2] + _free_profile_clones(box) + (c3 / ".git" / "info" / "exclude").write_text("scratch/\n", encoding="utf-8") + box.write(c3, "scratch/kept.txt", "an ignored file in a read-only directory\n") + (c3 / "scratch").chmod(0o555) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + asides = list(box.w.parent.glob(f".{c3.name}.removing-*")) + assert rows[str(c3)][0] == "PARTLY", rows[str(c3)] + assert result.returncode == 1, (result.stdout, result.stderr) + assert len(asides) == 1, asides + assert not list(box.lanes_root.glob(f".{c3.name}.removing-*")) + assert not os.path.lexists(c3) + noted = _noted_on_origin(box) + assert len(noted) == 1, noted + assert f"; moved aside and not all removed: {asides[0].name};" in noted[0], noted + # The aside's own `.git` went before `rm` met the read-only directory, + # so nothing leans on c2 any more, and c2 and c1 go after it. + assert "2 removed, 1 moved aside and not all removed, 0 held of the 3 clone(s) beside W" \ + in result.stdout, result.stdout + result, rows = sweep() + assert rows[str(asides[0])][0] == "leftover", rows + assert rows[str(asides[0])][1].endswith( + f"chmod -R u+rwX -- {asides[0]} && rm -rf -- {asides[0]}"), rows[str(asides[0])] + finally: + for aside in box.w.parent.glob(f".{c3.name}.removing-*"): + (aside / "scratch").chmod(0o755) + if (c3 / "scratch").is_dir(): + (c3 / "scratch").chmod(0o755) From 4511512c06018f291e526d006ffd20e066e22195 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:14:10 +0000 Subject: [PATCH 33/53] lane-end --sweep-checkouts: the holds' comment line at 93 columns is wrapped, and a case holds every comment line of the sweep's section to 80 (lane openRepoTools-3's NIT 394, #198, slice 4) The holds' comment line that ends "No `origin/` to compare" stood at 93 columns, in a block that is otherwise 80 or less. It is wrapped, and the rest of its paragraph with it; no word changes. `test_the_sweeps_comments_are_80_columns_or_less` reads `lane-end` from the section's header, `# ---- AMENDMENT 23 S6.2`, to the `fi` that ends it, and fails on any comment line over 80 characters (characters, not bytes: a dash is one column). Red before the wrap (line 1381, 93), green after; its mutant, one comment line lengthened again, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 433f759beb91dbdf44ece97c410fc64183e5d8bc) --- lane-end | 9 +++++---- tests/test_a23_sweep_checkouts.py | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/lane-end b/lane-end index 59c1f98..aab8fd9 100755 --- a/lane-end +++ b/lane-end @@ -1378,10 +1378,11 @@ SC_IDX # `lanes/`, because no tool can tell a person's work from scratch; an # operation in progress, unmerged entries, a held `index.lock`, the # checkout's own writers' lock, and a hand edit of the register that - # `lanes-edit.sh commit` keeps in `.git/lanes-edit/`. No `origin/` to compare - # with is itself a hold: nothing can be proved published. IGNORED - # files do not hold — the register's own rules keep them out of every - # commit — and they are named, so a person sees them first; + # `lanes-edit.sh commit` keeps in `.git/lanes-edit/`. No + # `origin/` to compare with is itself a hold: nothing can be + # proved published. IGNORED files do not hold — the register's own + # rules keep them out of every commit — and they are named, so a + # person sees them first; # (b) another repository whose alternates file names its objects, looked # for under W's parent, `$TMPDIR` and `$HOME` (SHOULD 352, below), and # its own linked worktrees; a scan that cannot finish holds them all; diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index f8d110e..5f78825 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2570,3 +2570,27 @@ def test_a_removal_beside_W_that_does_not_finish_leaves_its_aside_beside_W(sweep (aside / "scratch").chmod(0o755) if (c3 / "scratch").is_dir(): (c3 / "scratch").chmod(0o755) + + +# ------------------------------------------------------------ the source + +def _sweep_section() -> list: + """`lane-end`'s sweep, from its section's header to the `fi` that ends + it: `(line number, line)` pairs.""" + lines = (REPO / "lane-end").read_text(encoding="utf-8").splitlines() + start = next(i for i, line in enumerate(lines) + if line.startswith("# ---- AMENDMENT 23 S6.2: `--sweep-checkouts`")) + end = lines.index("fi", start) + return list(enumerate(lines[start:end + 1], start + 1)) + + +def test_the_sweeps_comments_are_80_columns_or_less(): + """THE SWEEP'S COMMENTS WRAP AT 80 COLUMNS (lane openRepoTools-3's NIT + 394): one line stood at 93 in a block that is otherwise 80 or less. + Every comment line of the section is held to it, counted in + characters, since a dash is one column and three bytes.""" + section = _sweep_section() + assert len(section) > 1000, "the section was found whole" + wide = [(n, len(line)) for n, line in section + if line.lstrip().startswith("#") and len(line) > 80] + assert not wide, wide From a06b2cbaad686db38d9b1f744ebc2b02ab573cff Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:16:33 +0000 Subject: [PATCH 34/53] lane-end --sweep-checkouts: a `-lanes/` that cannot be entered or listed, or whose real path cannot be read, refuses the run with 1 instead of reading as "nothing to remove" (lane openRepoTools-3's NIT 377, #198, slice 4) With `-lanes/` at mode 000, `sc_real` could not enter it, so its real path read as empty, and the sweep said it "resolves through a link to , which this sweep never follows" and exited 0, in the dry run and under `--yes`: a read that failed, reported as nothing there. It is now asked first whether it can be entered and listed, and whether its real path can be read; when it cannot, the run refuses with 1 (D10: a read that failed) and "Nothing was removed", before anything beside W is looked at either. `test_a_lanes_folder_that_cannot_be_entered_or_listed_is_refused` (skipped as root): the dry run and `--yes` both exit 1 with that reason, and every checkout and clone stays. Red before the fix (the dry run read on and exited 3, the clones beside W being free); green, with the dry runs and the lane-checkout W case, and hygiene. Its mutant, the new test dropped, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 9b19e124e7b183024bf11ddc081c936f78350612) --- lane-end | 9 ++++++++- tests/test_a23_sweep_checkouts.py | 26 ++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/lane-end b/lane-end index aab8fd9..a842af0 100755 --- a/lane-end +++ b/lane-end @@ -261,7 +261,8 @@ # failed (which is NOT the same as the lane having no object log), or the # close-out gate could not be read (no `lane-worktrees`, or an exit that # is none of its three). `--sweep-checkouts`: no W, a read that failed -# (the listing of lanes, or a fetch `--yes` decides on), a removal +# (the listing of lanes, a `-lanes/` that cannot be entered or +# listed, or a fetch `--yes` decides on), a removal # that did not finish, or removals whose NOTED line was not written or # not confirmed on origin (the commands to finish it are printed) # 2 refusal — bad arguments, no row (or two), something still in flight, or @@ -1311,6 +1312,12 @@ SC_IDX sc_home_why="there is no $sc_home/: no lane checkout has been made on this workstation" elif [ ! -d "$sc_home" ]; then sc_home_why="$sc_home is not a directory: nothing in it is looked at" + # ONE THAT CANNOT BE ENTERED OR LISTED IS NO EMPTY ONE (lane + # openRepoTools-3's NIT 377): at mode 000 its real path read as empty, + # and the run said it "resolves through a link to ," and exited 0. It is + # a read that failed, and nothing beside W is looked at either. + elif [ ! -x "$sc_home" ] || [ ! -r "$sc_home" ] || [ -z "$(sc_real "$sc_home")" ]; then + die "$sc_home cannot be entered or listed, or its real path read, so what is in it is unknown — which is not an empty directory, nor one reached through a link. Nothing was removed." 1 elif [ "$(sc_real "$sc_home")" != "$sc_home" ]; then sc_home_why="$sc_home resolves through a link to $(sc_real "$sc_home"), which this sweep never follows: nothing in it is looked at" elif ! ls -A -- "$sc_home" >/dev/null 2>&1; then diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 5f78825..da57e38 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2594,3 +2594,29 @@ def test_the_sweeps_comments_are_80_columns_or_less(): wide = [(n, len(line)) for n, line in section if line.lstrip().startswith("#") and len(line) > 80] assert not wide, wide + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root enters and lists a directory of mode 000") +def test_a_lanes_folder_that_cannot_be_entered_or_listed_is_refused(sweep): + """A `-LANES/` THAT CANNOT BE ENTERED OR LISTED IS NO EMPTY ONE (lane + openRepoTools-3's NIT 377). At mode 000 its real path could not be read, + and the sweep said it "resolves through a link to , which this sweep + never follows" and exited 0, in the dry run and under `--yes`. Both now + refuse with 1, a read that failed, before anything is removed, beside W + too.""" + box = sweep.box + _free_profile_clones(box) + before = box.everything() + box.lanes_root.chmod(0) + try: + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode == 1, (result.stdout, result.stderr) + assert (f"{box.lanes_root} cannot be entered or listed, or its real path read" + in result.stderr), result.stderr + assert "Nothing was removed." in result.stderr, result.stderr + assert not rows, rows + finally: + box.lanes_root.chmod(0o755) + assert box.everything() == before From 1ac76163edd400d0d6565fc17d2524984b2e690c Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:19:45 +0000 Subject: [PATCH 35/53] lane-end --sweep-checkouts: under routing, `--yes` with a `LANES_REPO` that is W refuses with 2 before anything is removed, in S5's words (lane openRepoTools-3's NIT 376, #198, slice 4) Under `LANES_ROUTE_WRITES=1`, `lanes-edit.sh` refuses any write through a `LANES_REPO` that is W, the readers' copy (Amendment 23 S5), by its real toplevel or its git common directory. The sweep removed a checkout first, then its NOTED line was refused, and the by-hand line it printed was refused again in that window. The preflight now asks the same question of the caller's `LANES_REPO` before the first removal and refuses with 2, naming the lane checkout the line would go through. Without routing, `LANES_REPO=W` is the default write path, and is left as it is; the dry run writes nothing and is not refused. `test_yes_with_LANES_REPO_naming_W_is_refused_before_anything_is_removed`, W named by its own path and by a linked worktree of it (one git common directory): `--yes` exits 2 with S5's words, the CLOSED lane's checkout stays, and nothing reaches origin; the dry run still exits 3. Red before the fix (the act ran, exit 1); green after, with the other `LANES_REPO` cases, the preflight refusals and hygiene. Its mutant, the new test dropped, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 0f564e84ad17149df57854e7330eaae3f4dbb49b) --- lane-end | 14 +++++++++++++- tests/test_a23_sweep_checkouts.py | 31 +++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/lane-end b/lane-end index a842af0..ac5b154 100755 --- a/lane-end +++ b/lane-end @@ -269,7 +269,8 @@ # the close-out gate: the lane's trees, branches, scratch, caches or # residue still on disk, or the sweep refusing the lane. # `--sweep-checkouts`: `--yes` with no `LANES_LANE`, or with -# `LANES_NO_FETCH=1`; a W that is itself a lane checkout; a runner lane +# `LANES_NO_FETCH=1`, or under `LANES_ROUTE_WRITES=1` with a +# `LANES_REPO` that is W; a W that is itself a lane checkout; a runner lane # the register does not know, or one with no transcript uuid for the # session field of its NOTED line # 3 `--sweep-checkouts`, the dry run: something under `-lanes/` is @@ -1194,6 +1195,17 @@ SC_IDX if [ -n "$sc_xtop" ]; then sc_xrepo="$(sc_real "$sc_xtop")"; fi fi fi + # A `LANES_REPO` THAT IS W REFUSES `--yes` BEFORE ANYTHING IS REMOVED + # (lane openRepoTools-3's NIT 376). Under routing, `lanes-edit.sh` refuses + # a write through W, the readers' copy (Amendment 23 S5), by its toplevel + # or its git common directory, so the NOTED line this act ends with would + # be refused after the removals, and its by-hand line again in this + # window. Without routing that `LANES_REPO` is the default write path. + if [ "$sweep_yes" = 1 ] && [ -n "$sc_xrepo" ] && [ "${LANES_ROUTE_WRITES:-}" = 1 ]; then + if [ "$sc_xrepo" = "$sc_w" ] || [ "$(sc_git_common "$sc_xrepo")" = "$sc_wgcd" ]; then + die "LANES_REPO names the workspace checkout $sc_w (as $LE_REPO_GIVEN), and that checkout is the readers' copy: no tool writes there (Amendment 23 S5), so the NOTED line this act ends with would be refused after the removals. Unset LANES_REPO and re-run from your lane — LANES_LANE=$sc_runner $prog --sweep-checkouts --yes — and the line goes through $sc_home/$sc_runner/. Nothing was removed." 2 + fi + fi # ------------------------------------------------ the entries, one by one # diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index da57e38..ab813f8 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2620,3 +2620,34 @@ def test_a_lanes_folder_that_cannot_be_entered_or_listed_is_refused(sweep): finally: box.lanes_root.chmod(0o755) assert box.everything() == before + + +@pytest.mark.parametrize("spelled", ["W", "a linked worktree of W"]) +def test_yes_with_LANES_REPO_naming_W_is_refused_before_anything_is_removed(sweep, spelled): + """UNDER ROUTING, A `LANES_REPO` THAT IS W REFUSES `--yes` BEFORE ANYTHING + IS REMOVED (lane openRepoTools-3's NIT 376). `lanes-edit.sh` refuses a + write through W, the readers' copy (Amendment 23 S5), by its toplevel or + its git common directory, so the sweep removed a checkout and then its + NOTED line was refused, and the by-hand line it printed again in that + window. Now it refuses with 2 in S5's words, the CLOSED lane's checkout + stays and nothing reaches origin. The dry run writes nothing, and still + lists. (A `LANES_REPO` naming W's `lanes/` directory is no register + checkout at all, and the listing itself refuses it.)""" + box = sweep.box + c = box.checkout(CLOSED) + named = box.w + if spelled == "a linked worktree of W": + named = box.root / "w-linked" + box.git("-C", str(box.w), "worktree", "add", "-q", "--detach", str(named)) + head = box.origin_head() + result, rows = sweep(LANES_REPO=str(named)) + assert result.returncode == 3, (result.stdout, result.stderr) + assert rows[str(c)][0] == "remove", rows[str(c)] + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_REPO=str(named)) + assert result.returncode == 2, (result.stdout, result.stderr) + assert "that checkout is the readers' copy: no tool writes there (Amendment 23 S5)" \ + in result.stderr, result.stderr + assert f"the line goes through {box.lanes_root}/{RUNNER}/" in result.stderr, result.stderr + assert "Nothing was removed." in result.stderr, result.stderr + assert c.is_dir() + assert box.origin_head() == head, "nothing reached origin" From a90f7ba397382d31e761207a099bbd615f80d7ac Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:23:41 +0000 Subject: [PATCH 36/53] lane-end --sweep-checkouts: the pid line of the lock the sweep takes is cleaned as `lanes-edit.sh`'s `lock_ident` cleans its own, `:` kept and a bad container `unknown` (lane openRepoTools-3's NIT 359, #198, slice 4) A writer of a checkout compares the pid line of the lock it meets with its own `lock_ident` to know a holder in its own container, whose pid it can ask; any other line is aged by the directory's mtime alone. The sweep kept only `[A-Za-z0-9._-]` and mapped anything else, and an empty value, to `none` (container) or `unknown` (host), while `lock_ident` keeps `:` too (`lanes_binding_value_ok`), and maps an empty container to `none` and a bad one to `unknown`. So a container named with a `:` made the sweep's line read as another container's, and a killed sweep's lock was waited out for ten minutes instead of taken back at once. The sweep's line now follows `lock_ident` character for character. `test_the_locks_pid_line_is_lanes_edits_own`: a hook at the CLOSED lane's fetch copies the pid line the sweep wrote; with `LANES_CONTAINER=bench:a` it ends ` bench:a hosta`, with `bench a` it ends ` unknown hosta`, and with `LANES_HOST=host:1` it ends ` bench-a host:1`. Red before the fix (`none`, `none`, `unknown`); green after. Its mutant, `:` dropped from the container's set, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 909dce4c46f481ed5220a21866eb611a8e707d41) --- lane-end | 12 ++++++++++-- tests/test_a23_sweep_checkouts.py | 28 ++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 2 deletions(-) diff --git a/lane-end b/lane-end index ac5b154..5f2e1e2 100755 --- a/lane-end +++ b/lane-end @@ -1849,13 +1849,21 @@ SRI # THE LOCK'S PID LINE, in `lanes-edit.sh`'s own form, ` # `, so a writer of that checkout in this container waits on a live # holder and one in another container ages it by the directory's mtime. + # Each value is cleaned as `lock_ident` cleans its own (lane + # openRepoTools-3's NIT 359): kept where `lanes_binding_value_ok` takes + # it (letters, digits, `.`, `_`, `:`, `-`), the container `none` and the + # host `unknown` where empty, and either `unknown` where it holds anything + # else. A writer compares the line with its own `lock_ident`; cleaned + # another way, it read as another container's, and a killed sweep's lock + # was waited out for ten minutes instead of taken back. sc_ident="none unknown" if [ "$sweep_yes" = 1 ]; then sc_wsl="$("$LANES_EDIT" workstation 2>/dev/null || :)" sc_host="$(printf '%s\n' "$sc_wsl" | cut -f3)" || sc_host="" sc_cont="$(printf '%s\n' "$sc_wsl" | cut -f7)" || sc_cont="" - case "$sc_host" in '' | *[!A-Za-z0-9._-]*) sc_host=unknown ;; esac - case "$sc_cont" in '' | *[!A-Za-z0-9._-]*) sc_cont=none ;; esac + sc_host="${sc_host:-unknown}"; sc_cont="${sc_cont:-none}" + case "$sc_host" in *[!A-Za-z0-9._:-]*) sc_host=unknown ;; esac + case "$sc_cont" in *[!A-Za-z0-9._:-]*) sc_cont=unknown ;; esac sc_ident="$sc_cont $sc_host" fi sc_lock_take() { # — 0 holding it (SC_LOCK), 1 with SC_WHY diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index ab813f8..1456c6d 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2651,3 +2651,31 @@ def test_yes_with_LANES_REPO_naming_W_is_refused_before_anything_is_removed(swee assert "Nothing was removed." in result.stderr, result.stderr assert c.is_dir() assert box.origin_head() == head, "nothing reached origin" + + +@pytest.mark.parametrize("env,ident", [ + ({"LANES_CONTAINER": "bench:a"}, "bench:a hosta"), + ({"LANES_CONTAINER": "bench a"}, "unknown hosta"), + ({"LANES_HOST": "host:1"}, "bench-a host:1"), +], ids=["a container with a colon", "a container with a blank", "a host with a colon"]) +def test_the_locks_pid_line_is_lanes_edits_own(sweep, env, ident): + """THE SWEEP'S PID LINE IS `lanes-edit.sh`'S `lock_ident`, CHARACTER FOR + CHARACTER (lane openRepoTools-3's NIT 359): ` `, + each kept where `lanes_binding_value_ok` takes it (letters, digits, `.`, + `_`, `:` and `-`), an empty container `none` and an empty host + `unknown`, and either `unknown` where it holds anything else. A writer + compares a lock's line with its own `lock_ident` to know a holder in its + own container, whose pid it can ask; the sweep's line, cleaned another + way, read as another container's, and a killed sweep's lock was waited + out for ten minutes. A hook at the CLOSED lane's fetch copies the line + the sweep wrote.""" + box = sweep.box + c = box.checkout(CLOSED) + copy = box.root / "pid-line.txt" + _hook_on_fetch_of(box, c, f"cp -- {box.lock_of(c)}/pid {copy}") + result, rows = sweep("--yes", LANES_LANE=RUNNER, **env) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows[str(c)] + pid, _, said = copy.read_text(encoding="utf-8").rstrip("\n").partition(" ") + assert pid.isdigit(), pid + assert said == ident, said From dd23c572e331f246a37942ad33a3421f829370c7 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:27:18 +0000 Subject: [PATCH 37/53] lane-end --sweep-checkouts: the aside's name keeps the first 48 characters of the checkout's, so a long name is moved aside instead of held in every act (lane openRepoTools-3's NIT 379, #198, slice 4) The aside was `..removing--`, which grows with the name: a 240-character name, with its 35-character suffix, is past NAME_MAX, so its `mv` always failed, and the checkout was `remove` in every dry run and held "could not be moved aside" in every `--yes`. The aside is now `..removing--`, at most 84 characters, still named for a person and still matched by `.?*.removing-*`, so every later run names one that is left. `test_a_long_name_is_moved_aside_under_a_name_that_does_not_grow`: a 240-character clone beside W is `remove` in the dry run and `removed` under `--yes`, through an aside (seen by an `mv` first on PATH) whose name is the first 48 characters and the suffix, and no aside is left. Red before the fix (held, could not be moved aside); green after. `l3-d37-aside` is re-pointed at the new line, and the new mutant, the whole name kept, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 1bd64905f14f7859a5e43272d0a98f0a0b91346a) --- lane-end | 23 ++++++++++++++-------- tests/test_a23_sweep_checkouts.py | 32 +++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 8 deletions(-) diff --git a/lane-end b/lane-end index 5f2e1e2..9691627 100755 --- a/lane-end +++ b/lane-end @@ -1310,13 +1310,14 @@ SC_IDX # would lead the sweep out of it, and nothing in it is then looked at. # # A REMOVAL THAT DID NOT FINISH IS NAMED AGAIN BY EVERY RUN (lane - # openRepoTools-3's SHOULD 372). Its aside, `..removing--`, - # is hidden, so a later run would count it with the creation locks and say - # no more: a checkout moved aside by a sweep that was stopped, or whose - # `rm` met a file it could not remove, would be named nowhere again. Each - # one is listed by name, apart from the hidden count, with the command that - # removes it. The sweep never removes one itself: a person looks at why - # first, and one may be another sweep's, still running. + # openRepoTools-3's SHOULD 372). Its aside, `..removing--` + # (the name's first 48 characters, NIT 379), is hidden, so a later run + # would count it with the creation locks and say no more: a checkout moved + # aside by a sweep that was stopped, or whose `rm` met a file it could not + # remove, would be named nowhere again. Each one is listed by name, apart + # from the hidden count, with the command that removes it. The sweep never + # removes one itself: a person looks at why first, and one may be another + # sweep's, still running. sc_home_why=""; sc_hidden=0; sc_leftovers=() if [ -L "$sc_home" ]; then sc_home_why="$sc_home is a symbolic link, which this sweep never follows: nothing in it is looked at" @@ -2244,7 +2245,13 @@ SLU sc_lock_drop sc_say_held "$sc_c" "$sc_why" "$sc_act_hold"; continue fi - sc_aside="${sc_c%/*}/.${sc_c##*/}.removing-$sc_utc-$$" + # THE ASIDE'S NAME DOES NOT GROW WITH THE CHECKOUT'S (lane + # openRepoTools-3's NIT 379): with its suffix, a name near NAME_MAX could + # never be moved aside, so a 240-character one was `remove` in every dry + # run and held in every act. Its first 48 characters name it for a + # person, and the suffix makes it this run's. + sc_an="${sc_c##*/}" + sc_aside="${sc_c%/*}/.${sc_an:0:48}.removing-$sc_utc-$$" sc_sig_defer if ! sc_under_home "$sc_c" || [ -e "$sc_aside" ] || [ -L "$sc_aside" ] || ! ( trap '' INT TERM; exec mv -- "$sc_c" "$sc_aside" ) 2>/dev/null; then diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 1456c6d..24333ab 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2679,3 +2679,35 @@ def test_the_locks_pid_line_is_lanes_edits_own(sweep, env, ident): pid, _, said = copy.read_text(encoding="utf-8").rstrip("\n").partition(" ") assert pid.isdigit(), pid assert said == ident, said + + +def test_a_long_name_is_moved_aside_under_a_name_that_does_not_grow(sweep): + """THE ASIDE'S NAME DOES NOT GROW WITH THE CHECKOUT'S (lane + openRepoTools-3's NIT 379). `..removing--` is past + NAME_MAX for a 240-character name, so its `mv` always failed: `remove` + in every dry run, held "could not be moved aside" in every `--yes`. A + 240-character clone beside W is now moved aside, through an aside (an + `mv` first on PATH logs it) named by its first 48 characters, and + removed.""" + box = sweep.box + long = box.clone(box.w.parent / ("brett-wip-" + "x" * 230), reference=box.w) + assert len(long.name) == 240 + real = shutil.which("mv") + assert real, "no mv on the test's own PATH" + log = box.root / "mv.log" + shim = box.fakebin / "mv" + shim.write_text(f'#!/usr/bin/env bash\nprintf "%s\\n" "${{@: -1}}" >> {log}\nexec {real} "$@"\n', + encoding="utf-8") + shim.chmod(0o755) + result, rows = sweep() + assert rows[str(long)] == ("remove", BESIDE_WHY), rows.get(str(long)) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(long)][0] == "removed", rows[str(long)] + assert not os.path.lexists(long) + asides = [Path(line) for line in log.read_text(encoding="utf-8").splitlines() + if f"/.{long.name[:48]}." in line] + assert len(asides) == 1, log.read_text(encoding="utf-8") + assert asides[0].parent == box.w.parent, asides + assert re.fullmatch(rf"\.{long.name[:48]}\.removing-\d{{8}}T\d{{6}}Z-\d+", asides[0].name), asides + assert not list(box.w.parent.glob(".*.removing-*")), "no aside is left" From 91b4c2b3bbb9e8ef1a08652a56b3ddaabfca5b06 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:37:41 +0000 Subject: [PATCH 38/53] lane-end --sweep-checkouts: each control character in a path or a reason the report prints is written as `?`, so a name with a newline writes no line of its own (lane openRepoTools-3's NIT 358, #198, slice 4) The report printed names raw: a newline in a name under `-lanes/` wrote a line of its own into it, one that could read as another entry's (`remove FORGED ...`), and a TAB in a dependent's path was printed inside a hold's reason. The decision was always right; only the report could be forged. Every entry line (`keep`, `hold`, `remove`, `removed`, `PARTLY`, `leftover`) now writes each control character of its path and its reason as `?`, by the shell's own `${x//[[:cntrl:]]/?}` (no process per name). The commands the `leftover` line hands out keep `%q`, which a shell reads back exactly. `test_a_name_with_a_control_character_writes_no_line_of_its_own`: a directory under `-lanes/` named `odd`, a newline, and a line that reads as the removal of `FORGED`: in the dry run and under `--yes` there is one line naming it, a `keep` whose path ends `odd? remove ...`, and no `FORGED` entry. Red before the fix; green after. The TAB dependent case of 352 now expects `tab?here`. Its mutant, the `keep` line's mapping dropped, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit ed8576fc3c8b0dc28c2fb8faf0e4a3f4684fe542) --- lane-end | 20 +++++++++++++------- tests/test_a23_sweep_checkouts.py | 23 ++++++++++++++++++++++- 2 files changed, 35 insertions(+), 8 deletions(-) diff --git a/lane-end b/lane-end index 9691627..b1d5bc6 100755 --- a/lane-end +++ b/lane-end @@ -1214,8 +1214,14 @@ SC_IDX # is what holds it, and the holds are asked of candidates alone. sc_cands=(); sc_cand_why=(); sc_cand_lane=(); sc_cand_new=() sc_keep_lines="" + # EACH CONTROL CHARACTER OF A PATH OR A REASON THE REPORT PRINTS IS WRITTEN + # AS `?` (lane openRepoTools-3's NIT 358), by the shell's own pattern + # substitution, so no process is started per name: a newline in a name + # wrote a line of its own into the report, one that could read as another + # entry's. The commands the report hands out keep `%q`, which a shell + # reads back exactly. sc_keep() { # - sc_keep_lines="${sc_keep_lines} keep $1 $2 + sc_keep_lines="${sc_keep_lines} keep ${1//[[:cntrl:]]/?} ${2//[[:cntrl:]]/?} " } sc_cand() { # [] @@ -2104,7 +2110,7 @@ SLU esac } sc_say_held() { # - printf ' hold %s %s; holds: %s\n' "$1" "$2" "$3" + printf ' hold %s %s; holds: %s\n' "${1//[[:cntrl:]]/?}" "${2//[[:cntrl:]]/?}" "${3//[[:cntrl:]]/?}" if sc_is_beside "$1"; then sc_nb_held=$((sc_nb_held + 1)); return 0; fi sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${1##*/}" } @@ -2194,7 +2200,7 @@ SLU sc_c="${sc_cands[$sc_i]}"; sc_why="${sc_cand_why[$sc_i]}" sc_l="${sc_cand_lane[$sc_i]}"; sc_new="${sc_cand_new[$sc_i]}" if ! sc_under_home "$sc_c"; then - printf ' hold %s %s; it is not directly under %s/ or in %s/, and nothing else is ever removed\n' "$sc_c" "$sc_why" "$sc_home" "$sc_root" + printf ' hold %s %s; it is not directly under %s/ or in %s/, and nothing else is ever removed\n' "${sc_c//[[:cntrl:]]/?}" "${sc_why//[[:cntrl:]]/?}" "${sc_home//[[:cntrl:]]/?}" "${sc_root//[[:cntrl:]]/?}" if sc_is_beside "$sc_c"; then sc_nb_held=$((sc_nb_held + 1)); continue; fi sc_n_held=$((sc_n_held + 1)); sc_held_names="${sc_held_names:+$sc_held_names }${sc_c##*/}"; continue fi @@ -2203,7 +2209,7 @@ SLU if [ -n "$SC_HOLD" ]; then sc_say_held "$sc_c" "$sc_why" "$SC_HOLD" else - printf ' remove %s %s%s\n' "$sc_c" "$sc_why" "${SC_IGN:+; $SC_IGN}" + printf ' remove %s %s%s\n' "${sc_c//[[:cntrl:]]/?}" "${sc_why//[[:cntrl:]]/?}" "${SC_IGN:+; ${SC_IGN//[[:cntrl:]]/?}}" sc_n_removable=$((sc_n_removable + 1)); sc_gone="$sc_gone$sc_c " if sc_is_beside "$sc_c"; then sc_nb_removable=$((sc_nb_removable + 1)); fi @@ -2264,13 +2270,13 @@ SLU ( trap '' INT TERM; exec rm -rf ${sc_rm_x:+"$sc_rm_x"} -- "$sc_aside" ) 2>/dev/null || : fi if [ -e "$sc_aside" ] || [ -L "$sc_aside" ]; then - printf ' PARTLY %s %s; moved aside to %s, which could not all be removed%s\n' "$sc_c" "$sc_why" "$sc_aside" "${sc_act_ign:+; $sc_act_ign}" + printf ' PARTLY %s %s; moved aside to %s, which could not all be removed%s\n' "${sc_c//[[:cntrl:]]/?}" "${sc_why//[[:cntrl:]]/?}" "${sc_aside//[[:cntrl:]]/?}" "${sc_act_ign:+; ${sc_act_ign//[[:cntrl:]]/?}}" sc_partial="${sc_partial:+$sc_partial }$sc_aside" sc_partial_names="${sc_partial_names:+$sc_partial_names }$(sc_said_name "${sc_aside##*/}")" sc_n_partial=$((sc_n_partial + 1)) if sc_is_beside "$sc_c"; then sc_nb_partial=$((sc_nb_partial + 1)); fi else - printf ' removed %s %s%s\n' "$sc_c" "$sc_why" "${sc_act_ign:+; $sc_act_ign}" + printf ' removed %s %s%s\n' "${sc_c//[[:cntrl:]]/?}" "${sc_why//[[:cntrl:]]/?}" "${sc_act_ign:+; ${sc_act_ign//[[:cntrl:]]/?}}" sc_n_removed=$((sc_n_removed + 1)) if sc_is_beside "$sc_c"; then sc_nb_removed=$((sc_nb_removed + 1)) @@ -2286,7 +2292,7 @@ SLU printf '%s' "$sc_keep_lines" for sc_e in ${sc_leftovers[@]+"${sc_leftovers[@]}"}; do printf ' leftover %s the aside of a removal that did not finish (a sweep stopped by a signal, or an rm that met what it could not remove; or one still running): no checkout, and no write goes through it. Look at why, then remove it by hand: chmod -R u+rwX -- %q && rm -rf -- %q\n' \ - "$sc_e" "$sc_e" "$sc_e" + "${sc_e//[[:cntrl:]]/?}" "$sc_e" "$sc_e" done if [ "${#sc_cands[@]}" = 0 ] && [ -z "$sc_keep_lines" ] && [ "${#sc_leftovers[@]}" = 0 ]; then printf ' (nothing)\n'; fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 24333ab..7e56204 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2187,7 +2187,8 @@ def test_a_dependent_anywhere_in_the_scope_holds(sweep, where): result, rows = sweep(*args, **env) assert result.returncode in (0, 3), (result.stdout, result.stderr) assert rows[str(c)][0] == "hold", rows[str(c)] - assert f"leaned on by {dep} " in rows[str(c)][1], rows[str(c)] + # A control character in a path is printed as `?` (NIT 358). + assert f"leaned on by {str(dep).replace(chr(9), '?')} " in rows[str(c)][1], rows[str(c)] assert c.is_dir() box.git("-C", str(dep), "fsck", "--connectivity-only") @@ -2711,3 +2712,23 @@ def test_a_long_name_is_moved_aside_under_a_name_that_does_not_grow(sweep): assert asides[0].parent == box.w.parent, asides assert re.fullmatch(rf"\.{long.name[:48]}\.removing-\d{{8}}T\d{{6}}Z-\d+", asides[0].name), asides assert not list(box.w.parent.glob(".*.removing-*")), "no aside is left" + + +def test_a_name_with_a_control_character_writes_no_line_of_its_own(sweep): + """NAMES ARE PRINTED WITH EACH CONTROL CHARACTER AS `?` (lane + openRepoTools-3's NIT 358). A newline in a name under `-lanes/` wrote + a line of its own into the report, here one that reads as the removal + of `FORGED`. The name is kept, and said on its own line, in the dry run + and under `--yes`; the directory stays.""" + box = sweep.box + odd = box.lanes_root / "odd\n remove FORGED lane repoS4-2 is CLOSED (ENDED)" + odd.mkdir() + shown = str(odd).replace("\n", "?") + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert "FORGED" not in rows, rows + lines = [line for line in result.stdout.splitlines() if "FORGED" in line] + assert len(lines) == 1, lines + assert lines[0].startswith(f" keep {shown} "), lines + assert odd.is_dir() From 859e866016d9523043c7e10a3de4f436371f59b9 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:43:59 +0000 Subject: [PATCH 39/53] lane-end --sweep-checkouts: commits only a checkout's reflog reaches are named beside it, in the dry run and the act (lane openRepoTools-3's NIT 354, #198, slice 4) A commit taken off its branch (`reset --keep HEAD~1`) is on no ref, so nothing held the checkout, and the removal took the reflog that could undo a mistaken reset, unsaid. Such commits are now named, as ignored files are: "commit(s) only its reflog reaches, which go with it: N ()", from `git rev-list --abbrev-commit --reflog --not --all`, on the `remove` line of the dry run and the `removed` line of the act. They do not hold: a write replayed after a lost push race leaves one whose patch origin has, so a hold would keep such a checkout for good. Lane openRepoTools-3 offered "hold, or name"; this names. `test_commits_only_the_reflog_reaches_are_named`: a commit in the CLOSED lane's checkout, then `reset --keep HEAD~1`; the dry run's `remove` and the act's `removed` name it by its short sha. Red before the fix; green after. Its mutant, the read dropped, is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 307b9ca518591cfa477be7deddbf093a9450f0f0) --- lane-end | 13 ++++++++++++- tests/test_a23_sweep_checkouts.py | 22 ++++++++++++++++++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/lane-end b/lane-end index b1d5bc6..b439ac1 100755 --- a/lane-end +++ b/lane-end @@ -1424,6 +1424,10 @@ SC_IDX # it, a window that writes through it (D37, below). SC_HOLD=""; SC_IGN="" sc_hold() { SC_HOLD="${SC_HOLD:+$SC_HOLD; }$1"; } + # WHAT GOES WITH A CHECKOUT AND HOLDS NOTHING IS NAMED beside it, on the + # dry run's `remove` line and the act's `removed` line, so a person sees + # it first: its ignored files, and the commits only its reflog reaches. + sc_named() { SC_IGN="${SC_IGN:+$SC_IGN; }$1"; } sc_first() { # — the count and the first three, as `N (a b c)` scf_n="$(printf '%s\n' "$1" | awk 'NF { n++ } END { print n + 0 }')" || scf_n="?" scf_h="$(printf '%s\n' "$1" | awk 'NF && k < 3 { print; k++ }' | tr '\n' ' ')" || scf_h="" @@ -1525,7 +1529,14 @@ SH_REFS if [ -n "$sh_x" ]; then sc_hold "linked worktree(s) of it: $(sc_first "$sh_x") (git -C $sh_p worktree list)"; fi fi sh_x="$(git -C "$sh_p" ls-files --others --ignored --exclude-standard --directory 2>/dev/null)" || sh_x="" - if [ -n "$sh_x" ]; then SC_IGN="ignored, which holds nothing: $(sc_first "$sh_x")"; fi + if [ -n "$sh_x" ]; then sc_named "ignored, which holds nothing: $(sc_first "$sh_x")"; fi + # A COMMIT ONLY THE REFLOG REACHES (lane openRepoTools-3's NIT 354): one + # a reset took off its branch is on no ref, and the removal takes the + # reflog that could undo the reset. It holds nothing, since a write + # replayed after a lost push race leaves one whose patch origin has, so + # a hold would keep such a checkout for good; it is named. + sh_x="$(git -C "$sh_p" rev-list --abbrev-commit --reflog --not --all 2>/dev/null)" || sh_x="" + if [ -n "$sh_x" ]; then sc_named "commit(s) only its reflog reaches, which go with it: $(sc_first "$sh_x")"; fi return 0 } diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 7e56204..aed1c4f 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2732,3 +2732,25 @@ def test_a_name_with_a_control_character_writes_no_line_of_its_own(sweep): assert len(lines) == 1, lines assert lines[0].startswith(f" keep {shown} "), lines assert odd.is_dir() + + +def test_commits_only_the_reflog_reaches_are_named(sweep): + """A COMMIT ONLY THE REFLOG REACHES IS NAMED (lane openRepoTools-3's NIT + 354). A commit taken off the branch by `reset --keep HEAD~1` is on no + ref, so nothing holds the checkout, and its removal takes the reflog + that could undo a mistaken reset. It holds nothing, since every write + that loses a push race leaves such a commit whose patch origin has; but + the dry run and the act name it beside the checkout, so a person sees + it first.""" + box = sweep.box + c = box.checkout(CLOSED) + box.write(c, "handoffs/undone.md", "a commit a reset took off the branch\n") + box.commit(c, "undone", "handoffs/undone.md") + sha = box.git("-C", str(c), "rev-parse", "--short", "HEAD").strip() + box.git("-C", str(c), "reset", "-q", "--keep", "HEAD~1") + said = f"commit(s) only its reflog reaches, which go with it: 1 ({sha})" + for args, env, tag in ((), {}, "remove"), (("--yes",), {"LANES_LANE": RUNNER}, "removed"): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c)][0] == tag, rows[str(c)] + assert said in rows[str(c)][1], rows[str(c)] From e43c084428f1c59e8f060af443c865937bf524ae Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:50:22 +0000 Subject: [PATCH 40/53] lane-end --sweep-checkouts: a repository at an ignored path and an ignored editor's swap file hold the checkout, annotated tags are named, and the comment that called ignored files the register's own rules is put right (lane openRepoTools-3's NIT 355, #198, slice 4) Ignored files hold nothing, and are named. But `--exclude-standard` reads the checkout's `.gitignore` files, its `.git/info/exclude` and the user's `core.excludesFile`, not the register's rules alone, as the holds' comment said. Two things such rules can hide are not disposable: - a repository of its own at an ignored path, which may hold commits nothing else has (at any other path it holds, as untracked): lane 3's probe, one in `.git/info/exclude`'s `vendor/` with a commit of its own, gave `remove`; - an editor's swap file (`*.swp`), which says a person has a file of the checkout open. Each ignored directory is looked in (`find`, `.git` pruned) and each of these holds; an ignored directory that cannot be looked in holds too. An annotated tag on a published commit reaches nothing origin lacks and holds nothing, but its tag object goes with the checkout, so it is named beside it, as ignored files are. `test_what_ignored_rules_hide_that_may_be_work_holds` (a repository at an ignored path, an ignored swap file): each holds, in the dry run and under `--yes`, and the checkout stays as it was. `test_annotated_tags_are_named`: an annotated and a lightweight tag on the published HEAD; the dry run's `remove` and the act's `removed` name the annotated one alone. Red before the fix (3 failed); green after. Its mutants, the repository hold and the swap hold dropped, are KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 9e0cbd1588f36459659688f071df3f6cc7ee4942) --- lane-end | 51 +++++++++++++++++++++++-- tests/test_a23_sweep_checkouts.py | 63 +++++++++++++++++++++++++++++++ 2 files changed, 111 insertions(+), 3 deletions(-) diff --git a/lane-end b/lane-end index b439ac1..f555d55 100755 --- a/lane-end +++ b/lane-end @@ -1406,9 +1406,13 @@ SC_IDX # checkout's own writers' lock, and a hand edit of the register that # `lanes-edit.sh commit` keeps in `.git/lanes-edit/`. No # `origin/` to compare with is itself a hold: nothing can be - # proved published. IGNORED files do not hold — the register's own - # rules keep them out of every commit — and they are named, so a - # person sees them first; + # proved published. IGNORED files do not hold, and they are + # named, so a person sees them first — but `--exclude-standard` + # reads the checkout's `.gitignore` files, its `.git/info/exclude` + # and the user's `core.excludesFile`, not the register's rules + # alone, so a repository of its own at an ignored path and an + # editor's swap file hold (NIT 355), and so does an ignored + # directory that cannot be looked in; # (b) another repository whose alternates file names its objects, looked # for under W's parent, `$TMPDIR` and `$HOME` (SHOULD 352, below), and # its own linked worktrees; a scan that cannot finish holds them all; @@ -1528,6 +1532,41 @@ SH_REFS sh_x="$(ls -A -- "$sh_gd/worktrees" 2>/dev/null)" || sh_x="?" if [ -n "$sh_x" ]; then sc_hold "linked worktree(s) of it: $(sc_first "$sh_x") (git -C $sh_p worktree list)"; fi fi + # WHAT IGNORED RULES HIDE IS NOT ALL DISPOSABLE (lane openRepoTools-3's + # NIT 355). `--exclude-standard` reads the checkout's `.gitignore` files, + # its `.git/info/exclude` and the user's `core.excludesFile`, not the + # register's rules alone. A repository of its own at an ignored path may + # hold commits nothing else has (at any other path it holds, as + # untracked), and an editor's swap file says a person has a file of it + # open: each holds. Each ignored directory is looked in for them; one + # that cannot be is a hold too, since what it keeps is unknown. + sh_rep=""; sh_swp=""; sh_unlooked="" + while IFS= read -r -d '' sh_e; do + case "$sh_e" in + */) + sh_y="$(find "$sh_p/$sh_e" \( -name .git -print -prune \) -o \( -type f -name '*.swp' -print \) 2>/dev/null)" || + sh_unlooked="${sh_unlooked}$sh_e +" + while IFS= read -r sh_f; do + [ -n "$sh_f" ] || continue + sh_f="${sh_f#"$sh_p/"}" + case "$sh_f" in + .git | */.git) sh_rep="${sh_rep}${sh_f%/.git} +" ;; + *) sh_swp="${sh_swp}$sh_f +" ;; + esac + done </dev/null) + if [ -n "$sh_rep" ]; then sc_hold "an ignored directory holds a repository of its own: $(sc_first "$sh_rep")"; fi + if [ -n "$sh_swp" ]; then sc_hold "an editor's swap file, ignored, says a file in it is open: $(sc_first "$sh_swp")"; fi + if [ -n "$sh_unlooked" ]; then sc_hold "an ignored directory could not be looked in: $(sc_first "$sh_unlooked")"; fi sh_x="$(git -C "$sh_p" ls-files --others --ignored --exclude-standard --directory 2>/dev/null)" || sh_x="" if [ -n "$sh_x" ]; then sc_named "ignored, which holds nothing: $(sc_first "$sh_x")"; fi # A COMMIT ONLY THE REFLOG REACHES (lane openRepoTools-3's NIT 354): one @@ -1537,6 +1576,12 @@ SH_REFS # a hold would keep such a checkout for good; it is named. sh_x="$(git -C "$sh_p" rev-list --abbrev-commit --reflog --not --all 2>/dev/null)" || sh_x="" if [ -n "$sh_x" ]; then sc_named "commit(s) only its reflog reaches, which go with it: $(sc_first "$sh_x")"; fi + # AN ANNOTATED TAG (NIT 355) on a published commit reaches nothing + # origin lacks, so it holds nothing; its tag object goes with the + # checkout, so it is named. + sh_x="$(git -C "$sh_p" for-each-ref --format='%(objecttype) %(refname:short)' refs/tags 2>/dev/null | + awk '$1 == "tag" { print $2 }')" || sh_x="" + if [ -n "$sh_x" ]; then sc_named "annotated tag(s), which go with it: $(sc_first "$sh_x")"; fi return 0 } diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index aed1c4f..5f1ae0d 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2754,3 +2754,66 @@ def test_commits_only_the_reflog_reaches_are_named(sweep): assert result.returncode in (0, 3), (result.stdout, result.stderr) assert rows[str(c)][0] == tag, rows[str(c)] assert said in rows[str(c)][1], rows[str(c)] + + +def _repository_at_an_ignored_path(box: Sweep, c: Path) -> None: + """A repository of its own, with a commit only it has, at a path the + checkout's own `.git/info/exclude` ignores.""" + (c / ".git" / "info" / "exclude").write_text("vendor/\n", encoding="utf-8") + nested = c / "vendor" / "tool" + box.git("init", "-q", "-b", "main", str(nested)) + box.write(nested, "work.txt", "a commit only this repository has\n") + box.commit(nested, "work", "work.txt") + + +def _swap_file_ignored(box: Sweep, c: Path) -> None: + """An editor's swap file, at a path the checkout's `.git/info/exclude` ignores.""" + (c / ".git" / "info" / "exclude").write_text("*.swp\n", encoding="utf-8") + box.write(c, "handoffs/.note.md.swp", "an editor has handoffs/note.md open\n") + + +IGNORED_HOLDS = [ + pytest.param(_repository_at_an_ignored_path, + "an ignored directory holds a repository of its own: 1 (vendor/tool)", + id="a repository at an ignored path"), + pytest.param(_swap_file_ignored, + "an editor's swap file, ignored, says a file in it is open: 1 (handoffs/.note.md.swp)", + id="an ignored swap file"), +] + + +@pytest.mark.parametrize("make,said", IGNORED_HOLDS) +def test_what_ignored_rules_hide_that_may_be_work_holds(sweep, make, said): + """IGNORED IS NOT ALWAYS DISPOSABLE (lane openRepoTools-3's NIT 355). + `--exclude-standard` reads the checkout's `.gitignore` files, its + `.git/info/exclude` and the user's `core.excludesFile`, not the + register's rules alone. A repository at an ignored path, whose commit + only it has, gave `remove` where the same repository at any other path + holds as untracked; an editor's swap file says a person has a file of it + open. Both hold, in the dry run and under `--yes`, and the checkout + stays as it was.""" + box = sweep.box + c = box.checkout(CLOSED) + make(box, c) + before = box.everything()[str(c)] + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert said in rows[str(c)][1], rows[str(c)] + assert box.everything()[str(c)] == before + + +def test_annotated_tags_are_named(sweep): + """AN ANNOTATED TAG IS NAMED (NIT 355): on a published commit it reaches + nothing origin lacks, so it holds nothing, but its tag object goes with + the checkout, so the dry run and the act name it.""" + box = sweep.box + c = box.checkout(CLOSED) + box.git("-C", str(c), "tag", "-a", "-m", "a note on a published commit", "v-note", "HEAD") + box.git("-C", str(c), "tag", "light", "HEAD") + for args, env, tag in ((), {}, "remove"), (("--yes",), {"LANES_LANE": RUNNER}, "removed"): + result, rows = sweep(*args, **env) + assert result.returncode in (0, 3), (result.stdout, result.stderr) + assert rows[str(c)][0] == tag, rows[str(c)] + assert "annotated tag(s), which go with it: 1 (v-note)" in rows[str(c)][1], rows[str(c)] From 8b28e3c6dc017dce933a651d397d8bf256f484e8 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 08:57:08 +0000 Subject: [PATCH 41/53] lane-end --sweep-checkouts: a sweep-wide lock refuses a second `--yes` with 2, and a candidate gone at its turn is reported as gone, not held (lane openRepoTools-3's NIT 375, #198, slice 4) Two `--yes` runs at once removed nothing twice, but each one's NOTED line said the other's removals were "held and left", and one candidate was left by both. `--yes` now takes a lock of its own before it reads or removes anything: `/lanes-sweep-checkouts.lock`, made by `mkdir`, with `lanes-edit.sh`'s pid line, and let go on every way out by the exit's `cleanup` unless its line names another holder. A second `--yes` is refused with 2, naming the lock and its pid line, and how to remove one a killed sweep left. The lock is in W's git directory, not `-lanes/` as lane 3 sketched, because since D37 the clones beside W are swept when there is no `-lanes/` at all. A candidate that is no longer there at its turn (a person, or another workstation's sweep, removed or moved it) is reported `gone`, not held for want of a `lanes/` directory, and is neither named removed nor held and left in the NOTED line. `test_a_second_sweep_is_refused_while_one_runs`: a hook at the CLOSED lane's fetch, inside the first sweep's act, runs a second `--yes`; it exits 2 with the lock named and prints nothing, and the first removes the checkout, writes one NOTED line and lets its lock go. `test_a_candidate_gone_at_its_turn_is_said_as_gone`: a hook at the first candidate's fetch moves the second away; it is `gone`. Red before the fix; green after. Its mutants, the lock taken by `mkdir -p` and the gone test dropped, are KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 51ee5ff243fe36261fd5560c94d6dc3918084303) --- lane-end | 82 +++++++++++++++++++++++-------- tests/test_a23_sweep_checkouts.py | 46 +++++++++++++++++ 2 files changed, 107 insertions(+), 21 deletions(-) diff --git a/lane-end b/lane-end index f555d55..6de4079 100755 --- a/lane-end +++ b/lane-end @@ -270,7 +270,8 @@ # residue still on disk, or the sweep refusing the lane. # `--sweep-checkouts`: `--yes` with no `LANES_LANE`, or with # `LANES_NO_FETCH=1`, or under `LANES_ROUTE_WRITES=1` with a -# `LANES_REPO` that is W; a W that is itself a lane checkout; a runner lane +# `LANES_REPO` that is W, or while another `--yes` holds its lock in +# W's git directory; a W that is itself a lane checkout; a runner lane # the register does not know, or one with no transcript uuid for the # session field of its NOTED line # 3 `--sweep-checkouts`, the dry run: something under `-lanes/` is @@ -606,6 +607,9 @@ LE_TMP="" # `SC_LOCK` is a register checkout's `lanes/.lanes-edit.lock` that # `--sweep-checkouts --yes` took and holds, released on every way out. SC_LOCK="" +# `SC_SLOCK` is the sweep-wide lock in W's git directory that +# `--sweep-checkouts --yes` takes first (NIT 375), released the same way. +SC_SLOCK="" cleanup() { [ -n "${LE_TMP:-}" ] && rm -f -- "$LE_TMP" # A LOCK WHOSE PID LINE NAMES ANOTHER HOLDER IS LEFT AS IT IS, as @@ -617,6 +621,12 @@ cleanup() { rm -f -- "$SC_LOCK/pid"; rmdir -- "$SC_LOCK" 2>/dev/null || : fi fi + if [ -n "${SC_SLOCK:-}" ]; then + cl_l=""; IFS= read -r cl_l < "$SC_SLOCK/pid" 2>/dev/null || : + if [ -z "$cl_l" ] || [ "$cl_l" = "$$ ${sc_ident:-}" ]; then + rm -f -- "$SC_SLOCK/pid"; rmdir -- "$SC_SLOCK" 2>/dev/null || : + fi + fi return 0 } # A SIGNAL ENDS THE RUN, AND `cleanup` RUNS ON THE WAY OUT. `cleanup` returns @@ -1028,6 +1038,47 @@ if [ "$sweep_checkouts" = 1 ]; then sc_wgcd="$(sc_git_common "$sc_w")" [ -n "$sc_wgcd" ] || die "W's git directory could not be read ($sc_w), so W cannot be told from a clone of it. Nothing was removed." 1 + # THE LOCK'S PID LINE, in `lanes-edit.sh`'s own form, ` + # `, so a writer of that checkout in this container waits on a live + # holder and one in another container ages it by the directory's mtime. + # Each value is cleaned as `lock_ident` cleans its own (lane + # openRepoTools-3's NIT 359): kept where `lanes_binding_value_ok` takes + # it (letters, digits, `.`, `_`, `:`, `-`), the container `none` and the + # host `unknown` where empty, and either `unknown` where it holds anything + # else. A writer compares the line with its own `lock_ident`; cleaned + # another way, it read as another container's, and a killed sweep's lock + # was waited out for ten minutes instead of taken back. + sc_ident="none unknown" + if [ "$sweep_yes" = 1 ]; then + sc_wsl="$("$LANES_EDIT" workstation 2>/dev/null || :)" + sc_host="$(printf '%s\n' "$sc_wsl" | cut -f3)" || sc_host="" + sc_cont="$(printf '%s\n' "$sc_wsl" | cut -f7)" || sc_cont="" + sc_host="${sc_host:-unknown}"; sc_cont="${sc_cont:-none}" + case "$sc_host" in *[!A-Za-z0-9._:-]*) sc_host=unknown ;; esac + case "$sc_cont" in *[!A-Za-z0-9._:-]*) sc_cont=unknown ;; esac + sc_ident="$sc_cont $sc_host" + fi + + # ONE `--yes` AT A TIME (lane openRepoTools-3's NIT 375). Two at once + # removed nothing twice, but each one's NOTED line called the other's + # removals "held and left", and one candidate was left by both. A lock in + # W's own git directory, there whatever `-lanes/` is (the clones beside + # W are swept without one), refuses a second `--yes` with 2 before + # anything is read or removed. Its pid line is a writer's lock's, and it + # is let go on every way out (`cleanup`); one a killed sweep left is a + # person's to remove, and the refusal says how. + if [ "$sweep_yes" = 1 ]; then + sc_slock="$sc_wgcd/lanes-sweep-checkouts.lock" + if ! mkdir -- "$sc_slock" 2>/dev/null; then + sc_sl=""; IFS= read -r sc_sl < "$sc_slock/pid" 2>/dev/null || : + die "another lane-end --sweep-checkouts --yes holds $sc_slock (${sc_sl:-it names no holder}), and two at once would each record the other's removals as held and left. Wait for it to end, then re-run. Where none is running (a sweep that was killed leaves its lock), remove it by hand: rm -rf -- $(printf '%q' "$sc_slock"). Nothing was removed." 2 + fi + SC_SLOCK="$sc_slock" + if ! printf '%s %s\n' "$$" "$sc_ident" 2>/dev/null > "$sc_slock/pid"; then + die "the sweep-wide lock $sc_slock was taken, and this sweep's pid line could not be written into it, so it was let go again. Nothing was removed." 1 + fi + fi + # A CLONE OF THE REGISTER, BY S1'S OWN TEST: a directory (never a symbolic # link) at its own real path, the root of its checkout, with a `.git` # DIRECTORY of its own (not a linked worktree's file, not a link), whose @@ -1909,26 +1960,6 @@ SRI esac } - # THE LOCK'S PID LINE, in `lanes-edit.sh`'s own form, ` - # `, so a writer of that checkout in this container waits on a live - # holder and one in another container ages it by the directory's mtime. - # Each value is cleaned as `lock_ident` cleans its own (lane - # openRepoTools-3's NIT 359): kept where `lanes_binding_value_ok` takes - # it (letters, digits, `.`, `_`, `:`, `-`), the container `none` and the - # host `unknown` where empty, and either `unknown` where it holds anything - # else. A writer compares the line with its own `lock_ident`; cleaned - # another way, it read as another container's, and a killed sweep's lock - # was waited out for ten minutes instead of taken back. - sc_ident="none unknown" - if [ "$sweep_yes" = 1 ]; then - sc_wsl="$("$LANES_EDIT" workstation 2>/dev/null || :)" - sc_host="$(printf '%s\n' "$sc_wsl" | cut -f3)" || sc_host="" - sc_cont="$(printf '%s\n' "$sc_wsl" | cut -f7)" || sc_cont="" - sc_host="${sc_host:-unknown}"; sc_cont="${sc_cont:-none}" - case "$sc_host" in *[!A-Za-z0-9._:-]*) sc_host=unknown ;; esac - case "$sc_cont" in *[!A-Za-z0-9._:-]*) sc_cont=unknown ;; esac - sc_ident="$sc_cont $sc_host" - fi sc_lock_take() { # — 0 holding it (SC_LOCK), 1 with SC_WHY slt_d="$1/lanes/.lanes-edit.lock" if [ -L "$1/lanes" ] || [ ! -d "$1/lanes" ]; then @@ -2280,6 +2311,15 @@ SLU # `LANES_REPO` read again; then, and only then, it is moved aside in the # directory it stands in — so its path is absent at once, and the next # write of a lane there clones afresh — and the aside removed. + # A CANDIDATE GONE AT ITS TURN IS SAID AS GONE (NIT 375), not held for + # want of a `lanes/` directory: a person, or a sweep on another + # workstation, removed or moved it after the listing. It is neither + # removed by this run nor held, and the NOTED line names it as neither. + if [ ! -e "$sc_c" ] && [ ! -L "$sc_c" ]; then + printf ' gone %s %s; it is no longer there: something removed or moved it after the listing, and this run did not\n' \ + "${sc_c//[[:cntrl:]]/?}" "${sc_why//[[:cntrl:]]/?}" + continue + fi if ! sc_lock_take "$sc_c"; then sc_say_held "$sc_c" "$sc_why" "$SC_WHY"; continue fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 5f1ae0d..d165212 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2817,3 +2817,49 @@ def test_annotated_tags_are_named(sweep): assert result.returncode in (0, 3), (result.stdout, result.stderr) assert rows[str(c)][0] == tag, rows[str(c)] assert "annotated tag(s), which go with it: 1 (v-note)" in rows[str(c)][1], rows[str(c)] + + +def test_a_second_sweep_is_refused_while_one_runs(sweep): + """ONE `--yes` AT A TIME (lane openRepoTools-3's NIT 375). Two at once + removed nothing twice, but each one's NOTED line called the other's + removals "held and left". A sweep-wide lock, taken before the listing, + refuses a second one with 2: here a hook at the CLOSED lane's fetch, in + the first sweep's act, runs a second `--yes`. It is refused, naming the + lock and its pid line, and removes nothing; the first removes the + checkout, writes one NOTED line, and lets its lock go.""" + box = sweep.box + c = box.checkout(CLOSED) + out = box.root / "second" + _hook_on_fetch_of(box, c, ( + f"bash {REPO / 'lane-end'} --sweep-checkouts --yes > {out}.out 2> {out}.err; " + f"echo $? > {out}.rc")) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows[str(c)] + err = Path(f"{out}.err").read_text(encoding="utf-8") + assert Path(f"{out}.rc").read_text(encoding="utf-8") == "2\n", err + lock = box.w / ".git" / "lanes-sweep-checkouts.lock" + assert f"another lane-end --sweep-checkouts --yes holds {lock}" in err, err + assert "Nothing was removed." in err, err + assert Path(f"{out}.out").read_text(encoding="utf-8") == "" + assert len(_noted_on_origin(box)) == 1, _noted_on_origin(box) + assert not os.path.lexists(lock), "the first sweep let its lock go" + + +def test_a_candidate_gone_at_its_turn_is_said_as_gone(sweep): + """A CANDIDATE GONE AT ITS TURN IS SAID AS GONE (NIT 375), not held: a + hook at the CLOSED lane's fetch moves the next candidate away, as a + person or another workstation's sweep might. It is reported `gone`, and + the NOTED line neither names it removed nor held and left.""" + box = sweep.box + c, other = box.checkout(CLOSED), _closed_checkout(box) + _hook_on_fetch_of(box, c, f"mv -- {other} {box.root / 'moved-away'}") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows[str(c)] + assert rows[str(other)][0] == "gone", rows[str(other)] + assert "it is no longer there" in rows[str(other)][1], rows[str(other)] + noted = _noted_on_origin(box) + assert len(noted) == 1, noted + assert CHAIN[0] not in noted[0], noted + assert (box.root / "moved-away" / ".git").is_dir() From 4a356ac900ffe7a096e54aaa556f338669077fa3 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 09:07:34 +0000 Subject: [PATCH 42/53] lane-end --sweep-checkouts: the move aside is the commit point, as `mutex_steal`'s rename is: the lock's pid line, the working directories and what leans on the checkout are asked again of what was moved, and any of them moves it back (lane openRepoTools-3's NIT 353, #198, slice 4) Under the lock, the working directories and the dependents were read before about ten git reads, and the pid line just before the `mv`, so three things could come between a read and the move unseen: a process that came to work in the checkout during the git reads, a `--reference` clone of it made then, and a writer that took the lock over between the pid-line read and the move. Lane 3's probes removed the checkout in all three. The window is short, but a removal is the one act nothing undoes. After the move, before the removal, the sweep now asks of what was moved: the pid line of the lock that went with it, read again; the working directories, read again against the aside's path (a process's working directory follows the rename); and what leans on it, read again, with an alternates or `.git` file whose target is gone kept as written, so a clone that names the old path is matched. Any of them moves the checkout back to its path and holds it, saying so; its lock is let go unless another writer holds it now. The lock is refreshed just before the pid-line read. Three cases, red before the fix (each `removed`), green after: a `git` first on PATH, right after the last read under the lock, starts a process in the checkout, or makes a `--reference` clone of it; an `mv` first on PATH writes another holder's pid line into the lock just before the move. Each is held "... as it was moved aside ...; it was moved back", and no aside is left. Their mutants are KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 811c10fc0fc57affb27a1dfa7fed982ff2d82622) --- lane-end | 44 +++++++++++--- tests/test_a23_sweep_checkouts.py | 97 +++++++++++++++++++++++++++++++ 2 files changed, 134 insertions(+), 7 deletions(-) diff --git a/lane-end b/lane-end index 6de4079..320144f 100755 --- a/lane-end +++ b/lane-end @@ -1716,11 +1716,11 @@ SDS_ERR fi sds_g="$(printf '%s\n' "$sds_g" | sed -n -e '1s/^gitdir: //p')" case "$sds_g" in '') continue ;; /*) : ;; *) sds_g="$sds_e/$sds_g" ;; esac + # A target that is gone is kept as written (NIT 353): after the + # move, a clone that named the old path is still matched. sds_r="$(sc_real "$sds_g")" - if [ -n "$sds_r" ]; then - sds_gl="$sds_gl$sds_r$sc_tab$sds_e -" - fi ;; + sds_gl="$sds_gl${sds_r:-$sds_g}$sc_tab$sds_e +" ;; */objects) sds_a="$sds_f/info/alternates" if [ ! -e "$sds_a" ] && [ ! -L "$sds_a" ]; then continue; fi @@ -1732,10 +1732,8 @@ SDS_ERR case "$sds_l" in '' | '#'*) continue ;; esac case "$sds_l" in /*) sds_t="$sds_l" ;; *) sds_t="$sds_f/$sds_l" ;; esac sds_r="$(sc_real "$sds_t")" - if [ -n "$sds_r" ]; then - sds_dl="$sds_dl$sds_r$sc_tab$sds_e + sds_dl="$sds_dl${sds_r:-$sds_t}$sc_tab$sds_e " - fi done </dev/null || : + if [ "$sc_moved_line" != "$$ $sc_ident" ]; then + sc_post="its writers' lock was taken over as it was moved aside (${sc_moved_line:-its pid line cannot be read}), and it is left to that writer" + fi + sc_cwd_scan + sc_x="$(sc_cwd_inside "$sc_aside")" + if [ -n "$sc_x" ]; then sc_post="${sc_post:+$sc_post; }a live process came to work in it as it was moved aside (pid $sc_x)"; fi + sc_dep_scan + if [ -n "$sc_dep_fail" ]; then sc_post="${sc_post:+$sc_post; }what leans on it cannot be told: $sc_dep_fail"; fi + # What is inside the aside went with it, and leans on nothing here. + sc_x="$(sc_dependents "$sc_c" "$sc_gone$sc_aside +")" + if [ -n "$sc_x" ]; then sc_post="${sc_post:+$sc_post; }leaned on by $sc_x, which came as it was moved aside"; fi + if [ -n "$sc_post" ]; then + if [ ! -e "$sc_c" ] && [ ! -L "$sc_c" ] && ( trap '' INT TERM; exec mv -- "$sc_aside" "$sc_c" ) 2>/dev/null; then + if [ "$sc_moved_line" = "$$ $sc_ident" ]; then SC_LOCK="$sc_c/lanes/.lanes-edit.lock"; sc_lock_drop; fi + sc_say_held "$sc_c" "$sc_why" "$sc_post; it was moved back" + else + sc_say_held "$sc_c" "$sc_why" "$sc_post; it could not be moved back, and is at $sc_aside, which every later run names" + fi + sc_sig_arm; continue + fi if sc_under_home "$sc_aside"; then ( trap '' INT TERM; exec rm -rf ${sc_rm_x:+"$sc_rm_x"} -- "$sc_aside" ) 2>/dev/null || : fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index d165212..0f275de 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2863,3 +2863,100 @@ def test_a_candidate_gone_at_its_turn_is_said_as_gone(sweep): assert len(noted) == 1, noted assert CHAIN[0] not in noted[0], noted assert (box.root / "moved-away" / ".git").is_dir() + + +def _git_after_the_last_read(box: Sweep, c: Path, body: str) -> None: + """A `git` first on PATH that runs `body` once, right after the last read + of `c`'s holds under its lock (its ignored files), before the move.""" + git = shutil.which("git") + assert git, "no git on the test's own PATH" + marker = box.root / "after-the-last-read.marker" + shim = box.fakebin / "git" + shim.write_text("#!/usr/bin/env bash\n" + f'case " $* " in *" -C {c} ls-files --others --ignored --exclude-standard --directory "*)\n' + f' {git} "$@"; rc=$?\n' + f" if [ ! -e {marker} ]; then : > {marker}; {body}; fi\n" + ' exit "$rc" ;;\n' + "esac\n" + f'exec {git} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + + +def test_a_process_that_comes_in_after_the_last_read_moves_it_back(sweep): + """THE MOVE IS THE COMMIT POINT (lane openRepoTools-3's NIT 353), as + `mutex_steal`'s rename is. The working directories were read before + the git reads under the lock, so a process that came in during them + was missed and the checkout removed under it. After the move they are + read again against the aside, where such a process now works; it holds + the checkout, which is moved back, its lock let go.""" + box = sweep.box + c = box.checkout(CLOSED) + pidfile = box.root / "late.pid" + _git_after_the_last_read(box, c, ( + f"(cd {c}/lanes && exec sleep 600) /dev/null 2>&1 & echo $! > {pidfile}")) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + late = pidfile.read_text(encoding="utf-8").strip() + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"a live process came to work in it as it was moved aside (pid {late})" \ + in rows[str(c)][1], rows[str(c)] + assert "it was moved back" in rows[str(c)][1], rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (c / ".git").is_dir() + assert not os.path.lexists(box.lock_of(c)), "its lock is let go" + _no_aside(box) + finally: + if pidfile.exists(): + try: + os.kill(int(pidfile.read_text(encoding="utf-8")), signal.SIGKILL) + except (ProcessLookupError, ValueError): + pass + + +def test_a_clone_made_after_the_last_read_moves_it_back(sweep): + """THE MOVE IS THE COMMIT POINT (NIT 353): a `--reference` clone of the + checkout made after the last read of what leans on it, in `$TMPDIR`, + names the old path, which the move took away, so it is matched as + written. It holds the checkout, which is moved back, and the clone stays + whole.""" + box = sweep.box + c = box.checkout(CLOSED) + dep = Path(box.env["TMPDIR"]) / "late-borrower" + git = shutil.which("git") + _git_after_the_last_read(box, c, ( + f"{git} clone -q --no-local --reference {c} {box.origin} {dep}")) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert (dep / ".git").is_dir(), "the shim made the clone" + assert rows[str(c)][0] == "hold", rows[str(c)] + assert f"leaned on by {dep}" in rows[str(c)][1], rows[str(c)] + assert "it was moved back" in rows[str(c)][1], rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + box.git("-C", str(dep), "fsck", "--connectivity-only") + _no_aside(box) + + +def test_a_lock_taken_over_at_the_move_moves_it_back(sweep): + """THE MOVE IS THE COMMIT POINT (NIT 353): the pid line was read before + the move, and a takeover between that read and the move went unseen. + An `mv` first on PATH writes another holder's line into the lock just + before the move. Read again from the aside, it holds the checkout, + which is moved back, and that writer's lock is left as it is.""" + box = sweep.box + c = box.checkout(CLOSED) + pid = box.lock_of(c) / "pid" + real = shutil.which("mv") + assert real, "no mv on the test's own PATH" + shim = box.fakebin / "mv" + shim.write_text("#!/usr/bin/env bash\n" + f'case "$*" in "-- {c} "*"/.{CLOSED}.removing-"*) printf "1 other-container otherhost\\n" > {pid} ;; esac\n' + f'exec {real} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert ("its writers' lock was taken over as it was moved aside " + "(1 other-container otherhost)") in rows[str(c)][1], rows[str(c)] + assert "it was moved back" in rows[str(c)][1], rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (c / ".git").is_dir() + assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" + _no_aside(box) From b6a517c929541f0b4ff8bd9ab908ef1833beb02d Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 12:16:22 +0000 Subject: [PATCH 43/53] Amendment 23 S6.2 tests: a CRLF `.git` file whose path is relative holds the clone it names, the case a kept CR fails, since NIT 353 keeps a target that does not resolve as written (the one survivor of the battery at 811c10f, #198, slice 4; tests only) The mutant `c3-3-crlf` (`tr -d '\r'` replaced by `cat` in the dependents scan's `.git` file read) was KILLED at e04d263 and at every commit after it up to 51ee5ff, and SURVIVED at 811c10f (NIT 353). Its one case wrote `gitdir: /.git` and a CRLF. Since 353 a target that does not resolve is kept as written, so that the clone a moved checkout leaves behind is still matched; with the CR kept, `/.git` and the CR after it is still under `/`, and the clone was held anyway. The CR handling is not redundant: a relative path, as git writes a submodule's `.git` file, resolves only once the CR is gone. With the CR kept, `

/../-lanes//.git` and its CR does not resolve, is kept as written, is under no candidate, and the clone is removed. `test_a_git_file_naming_a_clones_git_directory_holds_it` takes a fourth parameter, `CRLF and relative, whose path resolves once the CR is gone`; git itself reads that file as the clone's git directory, as the case asserts first. The mutant is re-pointed at it, and is KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 80bd3922ed3cd01b36da8004f0befa9693f55868) --- tests/test_a23_sweep_checkouts.py | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 0f275de..ba7b985 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2293,6 +2293,8 @@ def test_a_dependents_file_that_cannot_be_read_holds_every_candidate(sweep, whic pytest.param(lambda c, sharer: f"gitdir: {c}/.git\r\n", id="CRLF, whose CR git strips"), pytest.param(lambda c, sharer: f"gitdir: {os.path.relpath(c / '.git', sharer)}\n", id="relative to the directory it is in"), + pytest.param(lambda c, sharer: f"gitdir: {os.path.relpath(c / '.git', sharer)}\r\n", + id="CRLF and relative, whose path resolves once the CR is gone"), ] @@ -2304,7 +2306,13 @@ def test_a_git_file_naming_a_clones_git_directory_holds_it(sweep, spelled): clone is held, by the dry run and by `--yes`, naming the directory. The file is read as git reads it, so a CRLF line works in that directory too (rv4-c3's NIT c3-3, P3), and so does a path relative to the directory - the file is in (NIT c3-4, P4).""" + the file is in (NIT c3-4, P4). + + A CRLF line with an absolute path holds even with its CR kept: a target + that does not resolve is kept as written (NIT 353), and `/.git` + with a CR after it is still under the clone. A relative one, as git + writes a submodule's, resolves only once the CR is gone, so that case + is the one a kept CR fails.""" box = sweep.box c = box.checkout(CLOSED) sharer = box.w.parent / "sharer" From 8025e38510f86a099b47b571e5690374dc17b669 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 18:14:52 +0000 Subject: [PATCH 44/53] Amendment 23 S6.2: `--sweep-checkouts --yes` refuses with 2 while `LANES_WORKSPACE_ROOT` names another directory than workspace.yaml's `path:`, and that directory, or a clone beside W with a `-lanes/` of its own, is never a candidate (rv4b's MUST A on 811c10f, D45, #198, slice 4) rv4b's pre-review of 811c10f found that `LANES_WORKSPACE_ROOT` set to a per-profile clone beside the real W made that clone "W", and the real W, the readers' copy that workspace.yaml's `path:` names, "a clone of the register beside W": with nothing borrowing from it, `--yes` removed it, and the second-`--yes` lock moved into the substitute's git directory. - `path:` is read from workspace.yaml as every helper reads it, by `lanes_ws_field`, which no `LANES_WORKSPACE_ROOT` reaches. Where the variable is set and its real path is not `path:`'s, `--yes` exits 2 before the lock and before any listing, naming both and saying to unset the variable or point it at `path:`. Nothing is removed. - The dry run still runs, and its header says that `--yes` refuses. - `sc_is_a_w` keeps the directory `path:` names, and a clone beside W with a `-lanes/` beside it (a W with lane checkouts of its own), at the listing and again under the clone's lock before the act. - The seam stays (D45): the shared resolver is byte-identical in the four lane helpers, and the module and the shell suite point helpers at a sandbox through it. A value whose real path is `path:`'s, as spelled or through a link, is no refusal. The header's text says lane windows export it too, and names the refusal. Seven new cases in `tests/test_a23_sweep_checkouts.py`: 5 failed and 2 passed at 80bd392; the module's MUST A and neighbour selection with `tests/test_repo_hygiene.py`, 291 passed. Six new mutants (`rv4b-A-refuse`, `-real`, `-path`, `-lanes`, `-relock`, `-dry`), all KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit c73a9afc6ebfa9569a3d44947fa5be89ffddee65) --- lane-end | 52 +++++++++++++- tests/test_a23_sweep_checkouts.py | 114 ++++++++++++++++++++++++++++++ 2 files changed, 164 insertions(+), 2 deletions(-) diff --git a/lane-end b/lane-end index 320144f..0cd8ebd 100755 --- a/lane-end +++ b/lane-end @@ -271,7 +271,9 @@ # `--sweep-checkouts`: `--yes` with no `LANES_LANE`, or with # `LANES_NO_FETCH=1`, or under `LANES_ROUTE_WRITES=1` with a # `LANES_REPO` that is W, or while another `--yes` holds its lock in -# W's git directory; a W that is itself a lane checkout; a runner lane +# W's git directory, or with a `LANES_WORKSPACE_ROOT` that is not the +# directory workspace.yaml's `path:` names; a W that is itself a lane +# checkout; a runner lane # the register does not know, or one with no transcript uuid for the # session field of its NOTED line # 3 `--sweep-checkouts`, the dry run: something under `-lanes/` is @@ -290,7 +292,10 @@ # LANE_WORKTREES default lane-worktrees beside this command: the close-out # gate's sweep # LANES_WORKSPACE_ROOT the workspace repository, overriding workspace.yaml's -# `path:` — the test seam +# `path:` — the test seam, which lane windows export too. +# `--sweep-checkouts --yes` refuses while it names another +# directory than `path:` does (the dry run says so), and the +# directory `path:` names is never one of its candidates # LANE_END_MOUNTINFO the mount table `--sweep-checkouts` reads, in # `/proc/self/mountinfo`'s form (default that file) — the test # seam; one that names no `/` holds every checkout @@ -1032,6 +1037,29 @@ if [ "$sweep_checkouts" = 1 ]; then fi die "$(lanes_ws_yaml) names path: $sc_w as W — but it stands in $sc_root/, where a workstation keeps its lane checkouts: it is a lane checkout, not W (Amendment 23 S1, S5). Point that \`path:\` at the workspace checkout itself. Nothing was removed." 2 ;; esac + # W IS THE DIRECTORY WORKSPACE.YAML'S `path:` NAMES (rv4b's MUST A, D45). + # `LANES_WORKSPACE_ROOT` overrides it for every helper: the test seam, + # and lane windows export it too. Set to a register clone beside the real + # W, it made that clone "W", and the real W, with every other clone + # beside it, "a clone of the register beside W", which `--yes` removed + # wherever nothing borrowed from it. So `path:` is read as every helper + # reads it, from the file (no `LANES_WORKSPACE_ROOT` reaches that read), + # and `--yes` refuses with 2, before any lock and any listing, while the + # two differ by real path. The dry run says so. And the directory + # `path:` names is never a candidate, nor is a clone beside W with a + # `-lanes/` of its own beside it (`sc_is_a_w`, below). + sc_yaml="$(lanes_ws_yaml)"; sc_ypath="" + if [ -r "$sc_yaml" ]; then + sc_ypath="$(lanes_ws_field path "$sc_yaml" 2>/dev/null)" || sc_ypath="" + if [ -n "$sc_ypath" ]; then sc_ypath="$(sc_real "$sc_ypath")"; fi + fi + sc_wseam="" + if [ -n "${LANES_WORKSPACE_ROOT:-}" ] && [ "$sc_w" != "$sc_ypath" ]; then + sc_wseam="LANES_WORKSPACE_ROOT is $LANES_WORKSPACE_ROOT, so W resolves to $sc_w, and $sc_yaml's path: names ${sc_ypath:-no directory that can be entered}" + if [ "$sweep_yes" = 1 ]; then + die "$sc_wseam. --yes removes the register clones beside W, and with W taken from that variable, the real W, the readers' copy, would read as one of them. Unset LANES_WORKSPACE_ROOT, or point it at the directory path: names, and re-run. Nothing was removed." 2 + fi + fi sc_home="$sc_w-lanes" sc_worigin="$(git -C "$sc_w" remote get-url origin 2>/dev/null || :)" [ -n "$sc_worigin" ] || die "W ($sc_w) names no origin, so no clone can be told to be the register's. Nothing was removed." 1 @@ -1115,6 +1143,20 @@ if [ "$sweep_checkouts" = 1 ]; then fi return 0 } + # NEVER A CANDIDATE (D45): the directory workspace.yaml's `path:` names, + # wherever it stands, and a clone beside W with a `-lanes/` beside + # it, which is a W with lane checkouts of its own. 0 with SC_WHY for one. + sc_is_a_w() { # + if [ -n "$sc_ypath" ] && [ "$1" = "$sc_ypath" ]; then + SC_WHY="it is the directory $sc_yaml's path: names, which every helper resolves as W where LANES_WORKSPACE_ROOT is unset: the readers' copy" + return 0 + fi + if [ "${1%/*}" = "$sc_root" ] && { [ -e "$1-lanes" ] || [ -L "$1-lanes" ]; }; then + SC_WHY="${1##*/}-lanes/ stands beside it: it is a W with lane checkouts of its own, not a per-profile clone" + return 0 + fi + return 1 + } # THE LANES, AS THE REGISTER AND THE LOGS HAVE THEM: `lanes --closed --all`, # the one read that classes a lane CLOSED (Amendment 19), so this sweep and @@ -1276,6 +1318,7 @@ SC_IDX " } sc_cand() { # [] + if sc_is_a_w "$1"; then sc_keep "$1" "$SC_WHY"; return 0; fi sc_cands+=("$1"); sc_cand_lane+=("$2"); sc_cand_why+=("$3"); sc_cand_new+=("${4-}") } # A lane's checkout, judged by the lane's state: CLOSED is a candidate, @@ -2087,6 +2130,9 @@ SS_ROW printf ' W : %s\n' "$sc_w" printf ' lane checkouts : %s/\n' "$sc_home" printf ' runner : %s\n' "${sc_runner:-none named (LANES_LANE is unset)}" + if [ -n "$sc_wseam" ]; then + printf ' (--yes refuses here: %s, and W is the directory path: names)\n' "$sc_wseam" + fi if [ "$sweep_yes" = 1 ]; then printf ' origin : each checkout is fetched under its own lock before it is decided\n' else @@ -2329,6 +2375,8 @@ SLU sc_act_hold="$SC_WHY" elif ! sc_is_register_clone "$sc_c"; then sc_act_hold="it is no longer what made it a candidate: $SC_WHY" + elif sc_is_a_w "$sc_c"; then + sc_act_hold="it is no longer what made it a candidate: $SC_WHY" else sc_dep_scan; sc_cwd_scan; sc_mount_scan; sc_repo_scan sc_judge_holds "$sc_c" 1 diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index ba7b985..fa1d4d3 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -451,6 +451,120 @@ def test_a_W_that_is_a_lane_checkout_is_refused(sweep): assert box.everything() == before +def _unborrow(box: Sweep, repo: Path) -> None: + """`repo` takes every object it borrows into its own store, and borrows + no more (rv4b's probe P1).""" + alt = repo / ".git" / "objects" / "info" / "alternates" + if alt.exists(): + box.git("-C", str(repo), "repack", "-a", "-d", "-q") + alt.write_text("", encoding="utf-8") + + +def _nothing_borrows_from_W(box: Sweep) -> None: + """No clone under `-lanes/` or beside W borrows objects any more, so + no alternates file holds W: only the rules hold it.""" + for repo in sorted(box.lanes_root.iterdir()): + if not repo.is_symlink() and (repo / ".git").is_dir(): + _unborrow(box, repo) + for repo in box.profile_clones: + _unborrow(box, repo) + + +def _yaml_keep(box: Sweep) -> str: + return (f"it is the directory {box.home / '.agents' / 'workspace.yaml'}'s path: names, " + "which every helper resolves as W where LANES_WORKSPACE_ROOT is unset: the readers' copy") + + +@pytest.mark.parametrize("yes", [False, True], ids=["dry run", "--yes"]) +def test_a_W_override_naming_a_clone_beside_W_never_sweeps_the_real_W(sweep, yes): + """MUST A (rv4b's pre-review of 811c10f, D45). `LANES_WORKSPACE_ROOT` + naming a per-profile clone beside the real W made that clone "W", and the + real W, the directory workspace.yaml's `path:` names, "a clone of the + register beside W": with nothing borrowing from it, `--yes` removed it. + `--yes` now refuses with 2 before anything is listed or locked, and the + dry run keeps the real W and says what `--yes` refuses.""" + box = sweep.box + _nothing_borrows_from_W(box) + fake_w = box.profile_clones[2] + before = box.everything() + if yes: + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_WORKSPACE_ROOT=str(fake_w)) + assert result.returncode == 2, (result.stdout, result.stderr) + assert f"LANES_WORKSPACE_ROOT is {fake_w}" in result.stderr, result.stderr + assert f"path: names {box.w}." in result.stderr, result.stderr + assert "Nothing was removed." in result.stderr, result.stderr + assert rows == {}, rows + else: + result, rows = sweep(LANES_WORKSPACE_ROOT=str(fake_w)) + assert rows.get(str(box.w)) == ("keep", _yaml_keep(box)), rows.get(str(box.w)) + assert f"(--yes refuses here: LANES_WORKSPACE_ROOT is {fake_w}" in result.stdout, \ + result.stdout + assert (box.w / ".git").is_dir(), "the real W is there" + assert box.everything() == before + + +def test_the_directory_workspace_yaml_names_is_never_a_candidate(sweep): + """D45: workspace.yaml's `path:`, read as every helper reads it, names + the first clone beside W, while `LANES_WORKSPACE_ROOT` names W. Nothing + holds that clone, and it is kept for what it is, by the dry run, which + also says that `--yes` refuses.""" + box = sweep.box + c1, c2, c3 = box.profile_clones + _free_profile_clones(box) + (box.home / ".agents" / "workspace.yaml").write_text( + f"repository: {box.origin}\npath: {c1}\n", encoding="utf-8") + result, rows = sweep(LANES_WORKSPACE_ROOT=str(box.w)) + assert rows.get(str(c1)) == ("keep", _yaml_keep(box)), rows.get(str(c1)) + assert rows[str(c2)][0] == "remove", rows[str(c2)] + assert rows[str(c3)][0] == "remove", rows[str(c3)] + assert f"(--yes refuses here: LANES_WORKSPACE_ROOT is {box.w}" in result.stdout, result.stdout + + +@pytest.mark.parametrize("when", ["at the listing", "after the listing"]) +def test_a_clone_beside_W_with_lane_checkouts_of_its_own_is_never_a_candidate(sweep, when): + """D45: a clone beside W with a `-lanes/` beside it is a W, with + lane checkouts of its own, and never a per-profile clone: kept at the + listing, and held when that directory comes after it (asked again under + the clone's lock, after its fetch).""" + box = sweep.box + c1, c2, c3 = box.profile_clones + _free_profile_clones(box) + own = c1.parent / f"{c1.name}-lanes" + why = f"{c1.name}-lanes/ stands beside it: it is a W with lane checkouts of its own, not a per-profile clone" + if when == "at the listing": + own.mkdir() + result, rows = sweep() + assert rows.get(str(c1)) == ("keep", why), rows.get(str(c1)) + assert rows[str(c2)][0] == "remove", rows[str(c2)] + assert rows[str(c3)][0] == "remove", rows[str(c3)] + else: + _hook_on_fetch_of(box, c1, f"mkdir {own}") + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows.get(str(c1)) == ( + "hold", f"{BESIDE_WHY}; holds: it is no longer what made it a candidate: {why}"), \ + rows.get(str(c1)) + assert rows[str(c2)][0] == "removed", rows[str(c2)] + assert rows[str(c3)][0] == "removed", rows[str(c3)] + assert (c1 / ".git").is_dir() + + +@pytest.mark.parametrize("spelled", ["W", "a link to W"]) +def test_a_W_override_naming_W_itself_is_no_refusal(sweep, spelled): + """THE SEAM STAYS (D45): `LANES_WORKSPACE_ROOT` naming the directory + workspace.yaml's `path:` names, as spelled or through a link, is the W + the sweep would find without it, and `--yes` runs.""" + box = sweep.box + seam = box.w + if spelled == "a link to W": + seam = box.root / "w-link" + seam.symlink_to(box.w) + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANES_WORKSPACE_ROOT=str(seam)) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(box.checkout(CLOSED))][0] == "removed", rows + assert "--yes refuses here" not in result.stdout, result.stdout + + def test_clones_beside_W_are_judged_by_what_they_hold(sweep): """S6.2's PER-PROFILE CLONES ARE CANDIDATES (Brett Heap's ruling on #308's question 1, D37), each judged by the holds: the leaf c3 holds an untracked From 92186da6f0ce87924d9ec3a33800a2103d212e30 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 20:47:39 +0000 Subject: [PATCH 45/53] Amendment 23 S6.2: after `--sweep-checkouts --yes` moves a checkout aside, the windows' `LANES_REPO` and the mount table are read again, and a window that came to name the old path, a mount that came to be in the aside, or no table or `/proc` to tell by, moves it back (rv4b's SHOULD D on 811c10f, #198, slice 4) rv4b's pre-review of 811c10f found that NIT 353's re-check after the move asked the lock's pid line, the working directories and what leans on the checkout again, but not D37's `LANES_REPO` hold or the mounts: both were read before the git reads under the lock, so a window opened on the CLOSED lane's checkout during them (probe P4) was missed, and the checkout removed under it. - After the move, `sc_repo_scan` runs again, and a live process whose `LANES_REPO` names the old path, as spelled, holds it; with no `/proc` then, a clone beside W is held as "cannot be told", as D37 holds it at the listing and under the lock. - `sc_mount_scan` runs again, and a mount point at or under the aside (the move carried it) holds it; no mount table that names `/` holds it as "cannot be told". - Each moves the checkout back, its lock let go, as the other three do. Four new cases in `tests/test_a23_sweep_checkouts.py` (a window opened after the last read, a mount made as it is moved aside, no table then, `/proc` gone then beside W): 4 failed at c73a9af (`removed` for `hold`). With the fix, the module's post-move, mount, window and `/proc` selection, 27 passed; `tests/test_repo_hygiene.py`, 253 passed. Seven new mutants (`rv4b-D-repo`, `-repo-scan`, `-reponone`, `-mount`, `-mount-scan`, `-mountnone`, `-aside`), all KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 2bc9b667617dd51cb942fa3a1b11ad6a334bedab) --- lane-end | 20 ++++++- tests/test_a23_sweep_checkouts.py | 96 +++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+), 2 deletions(-) diff --git a/lane-end b/lane-end index 0cd8ebd..d939f86 100755 --- a/lane-end +++ b/lane-end @@ -2414,9 +2414,12 @@ SLU # see, between them and the move, is asked of what was moved: the pid # line of the lock that went with it (a writer that took the lock over # after it was read), the working directories (a process that came in - # during the git reads now works in the aside), and what leans on it (a + # during the git reads now works in the aside), what leans on it (a # clone made meanwhile names the old path, which is gone, so it is - # matched as written). Any of them moves it back, and holds it. + # matched as written), the windows' `LANES_REPO` (D37: a window opened + # meanwhile names the old path as spelled), and the mounts (one made + # meanwhile is in the aside, which carried it) (rv4b's SHOULD D, for + # the last two). Any of them moves it back, and holds it. sc_post=""; sc_moved_line="" IFS= read -r sc_moved_line < "$sc_aside/lanes/.lanes-edit.lock/pid" 2>/dev/null || : if [ "$sc_moved_line" != "$$ $sc_ident" ]; then @@ -2431,6 +2434,19 @@ SLU sc_x="$(sc_dependents "$sc_c" "$sc_gone$sc_aside ")" if [ -n "$sc_x" ]; then sc_post="${sc_post:+$sc_post; }leaned on by $sc_x, which came as it was moved aside"; fi + sc_repo_scan + sc_x="$(sc_repo_inside "$sc_c")" + if [ -n "$sc_x" ]; then sc_post="${sc_post:+$sc_post; }the LANES_REPO of a live process came to name it as it was moved aside (pid $sc_x)"; fi + if [ "$sc_repo_mode" = none ] && [ "${sc_c%/*}" = "$sc_root" ]; then + sc_post="${sc_post:+$sc_post; }whether a window's LANES_REPO names it cannot be told as it was moved aside: there is no /proc here to read a process's environment from" + fi + sc_mount_scan + case "$sc_mount_mode" in + none) sc_post="${sc_post:+$sc_post; }whether a file system is mounted in it cannot be told as it was moved aside: no mount table that names / could be read (${LANE_END_MOUNTINFO:-/proc/self/mountinfo}, or mount)" ;; + *) + sc_x="$(sc_mounts_inside "$sc_aside")" || sc_x="(the mount table could not be matched)" + if [ -n "$sc_x" ]; then sc_post="${sc_post:+$sc_post; }a file system came to be mounted at or under it as it was moved aside: $(sc_first "$sc_x")"; fi ;; + esac if [ -n "$sc_post" ]; then if [ ! -e "$sc_c" ] && [ ! -L "$sc_c" ] && ( trap '' INT TERM; exec mv -- "$sc_aside" "$sc_c" ) 2>/dev/null; then if [ "$sc_moved_line" = "$$ $sc_ident" ]; then SC_LOCK="$sc_c/lanes/.lanes-edit.lock"; sc_lock_drop; fi diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index fa1d4d3..63b5120 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -3082,3 +3082,99 @@ def test_a_lock_taken_over_at_the_move_moves_it_back(sweep): assert (c / ".git").is_dir() assert pid.read_text(encoding="utf-8") == "1 other-container otherhost\n" _no_aside(box) + + +def test_a_window_that_names_it_after_the_last_read_moves_it_back(sweep): + """THE MOVE IS THE COMMIT POINT FOR D37's HOLD TOO (rv4b's SHOULD D, + probe P4). The windows' `LANES_REPO` was read before the git reads + under the lock, so a window opened during them on the CLOSED lane's + checkout was missed, and the checkout removed under it. After the move + the environments are read again, and a window that names the old path + holds it: it is moved back, its lock let go.""" + box = sweep.box + c = box.checkout(CLOSED) + pidfile = box.root / "late.pid" + _git_after_the_last_read(box, c, ( + f"(cd {box.root} && LANES_REPO={c} exec sleep 600) /dev/null 2>&1 & " + f"echo $! > {pidfile}")) + try: + result, rows = sweep("--yes", LANES_LANE=RUNNER) + late = pidfile.read_text(encoding="utf-8").strip() + assert rows[str(c)][0] == "hold", rows[str(c)] + assert (f"the LANES_REPO of a live process came to name it as it was moved aside " + f"(pid {late})") in rows[str(c)][1], rows[str(c)] + assert "it was moved back" in rows[str(c)][1], rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (c / ".git").is_dir() + assert not os.path.lexists(box.lock_of(c)), "its lock is let go" + _no_aside(box) + finally: + if pidfile.exists(): + try: + os.kill(int(pidfile.read_text(encoding="utf-8")), signal.SIGKILL) + except (ProcessLookupError, ValueError): + pass + + +def _mv_then(box: Sweep, c: Path, act: str) -> None: + """An `mv` first on PATH that moves `c` aside, then runs `act` (its + `$3` is the aside), as a change made by another process at that moment + would show; every other `mv` is the real one.""" + real = shutil.which("mv") + assert real, "no mv on the test's own PATH" + shim = box.fakebin / "mv" + shim.write_text("#!/usr/bin/env bash\n" + f'case "$*" in "-- {c} "*"/.{c.name}.removing-"*) {real} "$@" || exit $?; {act}; exit 0 ;; esac\n' + f'exec {real} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + + +@pytest.mark.parametrize("then,said", [ + ("a mount in it", "a file system came to be mounted at or under it as it was moved aside: 1 ("), + ("no table", "whether a file system is mounted in it cannot be told as it was moved aside"), +], ids=["a mount in it", "no table"]) +def test_a_mount_made_as_it_is_moved_aside_moves_it_back(sweep, then, said): + """THE MOVE IS THE COMMIT POINT FOR THE MOUNTS TOO (rv4b's SHOULD D). The + mount table was read before the git reads under the lock. An `mv` first + on PATH moves the checkout aside and then, as the kernel would show a + mount made meanwhile, names a mount point inside the aside in the table + (`LANE_END_MOUNTINFO`, the seam), or leaves no table at all. Read again + after the move, it holds the checkout, which is moved back.""" + box = sweep.box + c = box.checkout(CLOSED) + table = _mount_table(box) + act = ("printf '35 21 0:45 /outside-x %s/.git/mnt rw,relatime - ext4 /dev/x rw\\n' " + f'"$3" >> {table}' if then == "a mount in it" else f": > {table}") + _mv_then(box, c, act) + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANE_END_MOUNTINFO=table) + assert rows[str(c)][0] == "hold", rows[str(c)] + assert said in rows[str(c)][1], rows[str(c)] + assert "it was moved back" in rows[str(c)][1], rows[str(c)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (c / ".git").is_dir() + assert not os.path.lexists(box.lock_of(c)), "its lock is let go" + _no_aside(box) + + +def test_a_proc_gone_as_a_clone_beside_W_is_moved_aside_moves_it_back(sweep): + """WITH NO `/proc` AFTER THE MOVE, A CLONE BESIDE W IS MOVED BACK (rv4b's + SHOULD D, as D37 holds it at the listing and under the lock): whether a + window came to write through it cannot be told. `LANE_END_PROC`, the + seam, is a link to `/proc` that an `mv` first on PATH takes away just + after it moves the leaf clone aside.""" + box = sweep.box + c3 = box.profile_clones[2] + _free_profile_clones(box) + proc = box.root / "proc-link" + proc.symlink_to("/proc") + real = shutil.which("mv") + _mv_then(box, c3, f"{real} -- {proc} {proc}.gone") + result, rows = sweep("--yes", LANES_LANE=RUNNER, LANE_END_PROC=str(proc)) + assert rows[str(c3)][0] == "hold", rows[str(c3)] + assert ("whether a window's LANES_REPO names it cannot be told as it was moved aside: " + "there is no /proc here") in rows[str(c3)][1], rows[str(c3)] + assert "it was moved back" in rows[str(c3)][1], rows[str(c3)] + assert result.returncode == 0, (result.stdout, result.stderr) + assert (c3 / ".git").is_dir() + assert not os.path.lexists(box.lock_of(c3)), "its lock is let go" + assert not list(box.w.parent.glob(".*.removing-*")), "no half-removed aside is left" From a90260a4bf4fa7373912ba9e54d439e703aa2deb Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 21:36:39 +0000 Subject: [PATCH 46/53] Amendment 23 S6.2: a directory of this user's own that the dependents scan cannot read holds every `--sweep-checkouts` candidate, in words that name it and `chmod`; another user's is still counted and holds nothing (rv4b's SHOULD B on 811c10f, #198, slice 4) rv4b's pre-review of 811c10f found that every directory the scan could not read was counted and held nothing, this user's own too: a `--reference` clone with a commit of its own, kept in a mode-000 folder beside W (probe P2), was not seen, `--yes` removed the CLOSED lane's checkout, and the clone's `fsck` then exited 10. - Each "Permission denied" line of `find` names the directory: GNU `find` quotes it in the C locale (`'P'`, a quote as `\'`, a backslash, TAB, newline and non-ASCII bytes as escapes), BSD `find` does not; `sc_find_err_path` undoes the quoting. - Where that directory is this user's own (`[ -O ]`), the scan cannot finish (`sc_dep_fail`), which holds every candidate: the words name the directory and `chmod u+rx -- `. - Another user's directory, or one gone while read, is counted and said, and holds nothing, as decided under Q2. The module's case for a mode-000 directory beside W is changed to match: it now holds both CLOSED lanes' checkouts, with a dependent inside it, a plain name and one with a quote, a blank and a backslash. A new case keeps another user's directory counted (a `find` first on PATH says it could not read one, since a test cannot make one without root). Red at 2bc9b66: 2 failed, 1 passed (the own-directory cases, rc 3 for 0). Green: the module's dependents, scope and unreadable-file selection 26 passed, tests/test_repo_hygiene.py 253 passed; the five new mutants (rv4b-B-own, -any, -quotes, -unquote, -escapes) KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit e12d077630b5017ef7cea9b69bb020e669cabd2b) --- lane-end | 36 +++++++++--- tests/test_a23_sweep_checkouts.py | 94 +++++++++++++++++++++++++------ 2 files changed, 105 insertions(+), 25 deletions(-) diff --git a/lane-end b/lane-end index d939f86..75cdc86 100755 --- a/lane-end +++ b/lane-end @@ -1700,12 +1700,16 @@ SH_Y # # A SCAN THAT CANNOT FINISH HOLDS EVERY CANDIDATE (`sc_dep_fail`), since # what leans on each is then unknown: a scope that cannot be entered, a - # `find` that fails for anything but a directory it cannot read, and an - # alternates or `.git` file that cannot be read (NIT 356). A DIRECTORY - # `find` CANNOT READ — another user's, or one gone while read — is counted - # and said, and holds nothing, by question 2's rule for working - # directories: one never readable would hold every checkout for good. It - # is counted once, though two scopes meet it. + # `find` that fails for anything but a directory it cannot read, an + # alternates or `.git` file that cannot be read (NIT 356), and a + # directory of THIS USER'S OWN that `find` cannot read (rv4b's SHOULD B): + # a clone kept in it is this user's work, and a mode this user set can be + # set back, so the words name it and `chmod`. ANOTHER USER'S DIRECTORY + # `find` cannot read, or one gone while read, is counted and said, and + # holds nothing, by question 2's rule for working directories: one never + # readable would hold every checkout for good. It is counted once, though + # two scopes meet it. `find`'s words name the directory as `find: 'P':`, + # in the C locale's quoting (GNU), or as `find: P:` (BSD). sc_dep_lines=""; sc_gitfile_lines=""; sc_dep_fail=""; sc_dep_unread=0 sc_dep_at=(); sc_dep_to=(); sc_dep_named=() sc_dep_root() { # @@ -1713,6 +1717,15 @@ SH_Y if [ -n "$2" ]; then sdr_r="$(sc_real "$2")"; fi sc_dep_to+=("$1"); sc_dep_at+=("$sdr_r"); sc_dep_named+=("${sdr_r:-${2:-(unset)}}") } + sc_find_err_path() { # "> — the path + sfe_x="${1#*: }" + case "$sfe_x" in + \'*\') + sfe_x="${sfe_x#\'}"; sfe_x="${sfe_x%\'}"; sfe_x="${sfe_x//\\\'/\'}" + case "$sfe_x" in *\\*) sfe_x="$(printf '%b' "$sfe_x")" ;; esac ;; + esac + printf '%s' "$sfe_x" + } sc_dep_root 6 "$sc_root" sc_dep_root 4 "${TMPDIR:-/tmp}" sc_dep_root 4 "${HOME:-}" @@ -1735,7 +1748,16 @@ SH_Y while IFS= read -r sds_l; do case "$sds_l" in '') : ;; - *": Permission denied" | *": No such file or directory") + *": Permission denied") + sds_p="$(sc_find_err_path "${sds_l%: Permission denied}")" + if [ -n "$sds_p" ] && [ -O "$sds_p" ]; then + sds_w="$sds_p, a directory of this user's own, cannot be read by the dependents scan, so what in it leans on a checkout is unknown: make it readable (chmod u+rx -- $(printf '%q' "$sds_p")) and re-run" + sc_dep_fail="${sc_dep_fail:-$sds_w}" + else + sds_unread="$sds_unread$sds_l +"; sds_counted=1 + fi ;; + *": No such file or directory") sds_unread="$sds_unread$sds_l "; sds_counted=1 ;; *) sds_other="${sds_other:-$sds_l}" ;; diff --git a/tests/test_a23_sweep_checkouts.py b/tests/test_a23_sweep_checkouts.py index 63b5120..184a130 100644 --- a/tests/test_a23_sweep_checkouts.py +++ b/tests/test_a23_sweep_checkouts.py @@ -2345,29 +2345,87 @@ def test_a_dependents_scan_that_fails_holds_every_candidate(sweep): @pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, reason="root reads a directory whose mode is 000") -def test_a_directory_the_dependents_scan_cannot_read_is_counted_and_holds_nothing(sweep): - """A DIRECTORY THE SCAN CANNOT READ IS COUNTED AND SAID, AND HOLDS NOTHING - (SHOULD 352, by question 2's rule for working directories): another - user's directories are never readable, and holding on them would hold - every checkout on any host that has one. Here a mode-000 directory - beside W is counted once, though two scopes meet it, and the CLOSED - lane's checkout is removed.""" +@pytest.mark.parametrize("name", ["private", "it's a b\\c"], + ids=["plain", "with a quote, a blank and a backslash"]) +def test_a_directory_of_this_users_own_the_dependents_scan_cannot_read_holds_every_candidate( + sweep, name): + """A DIRECTORY OF THIS USER'S OWN THAT THE SCAN CANNOT READ HOLDS EVERY + CANDIDATE (rv4b's SHOULD B, probe P2): a clone kept in it is this user's + work, and what that clone leans on is unknown. Here a `--reference` + clone of the CLOSED lane's checkout, with a commit of its own, is kept in + a mode-000 directory beside W, so the scan cannot see it. Both CLOSED + lanes' checkouts are held, by the dry run and by `--yes`, in words that + name the directory and `chmod`, and the clone stays whole. GNU `find` + quotes the name in its words, escaping a quote and a backslash; the + directory is told by its own path once that quoting is undone.""" box = sweep.box - c = box.checkout(CLOSED) - private = box.w.parent / "private" - private.mkdir() + c, other = box.checkout(CLOSED), _closed_checkout(box) + private = box.w.parent / name + dep = _borrower(box, private / "rescue", c) private.chmod(0) try: - result, rows = sweep() - assert result.returncode == 3, (result.stdout, result.stderr) - assert rows[str(c)][0] == "remove", rows[str(c)] - assert (" (1 director(ies) the dependents scan could not read - another user's, or " - "gone while read - are not looked in)") in result.stdout, result.stdout - result, rows = sweep("--yes", LANES_LANE=RUNNER) - assert result.returncode == 0, (result.stdout, result.stderr) - assert rows[str(c)][0] == "removed", rows[str(c)] + for args, env in ((), {}), (("--yes",), {"LANES_LANE": RUNNER}): + result, rows = sweep(*args, **env) + assert result.returncode == 0, (result.stdout, result.stderr) + for path in (c, other): + assert rows[str(path)][0] == "hold", rows[str(path)] + assert (f"what leans on it cannot be told: {private}, a directory of this " + "user's own, cannot be read by the dependents scan, so what in it " + "leans on a checkout is unknown: make it readable (chmod u+rx -- " + ) in rows[str(path)][1], rows[str(path)] + if name == "private": + assert f"(chmod u+rx -- {private}) and re-run" in rows[str(c)][1], rows[str(c)] finally: private.chmod(0o755) + assert c.is_dir() + assert other.is_dir() + box.git("-C", str(dep), "fsck", "--connectivity-only") + + +def _a_directory_of_another_user() -> str: + """A directory here that is not this user's own: a test cannot make one + without root.""" + for path in ("/", "/usr", "/etc", "/bin"): + try: + if os.stat(path).st_uid != os.geteuid(): + return path + except OSError: + continue + pytest.skip("no directory here is another user's") + + +@pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, + reason="root owns every directory a test can name") +def test_a_directory_of_another_user_the_dependents_scan_cannot_read_is_counted_and_holds_nothing( + sweep): + """ANOTHER USER'S DIRECTORY THE SCAN CANNOT READ IS COUNTED AND SAID, AND + HOLDS NOTHING (SHOULD 352, by question 2's rule for working directories, + kept by rv4b's SHOULD B): such a directory is never readable, and holding + on it would hold every checkout on any host that has one. No test can + make one without root, so a `find` first on PATH says, after the real + scan and in GNU `find`'s words, that it could not read one that is not + this user's (`/`, or the first of `/usr`, `/etc`, `/bin` that is not). It + is counted once, though every scope meets it, and the CLOSED lane's + checkout is removed.""" + box = sweep.box + c = box.checkout(CLOSED) + theirs = _a_directory_of_another_user() + real = shutil.which("find") + assert real, "no find on the test's own PATH" + shim = box.fakebin / "find" + shim.write_text("#!/usr/bin/env bash\n" + 'case " $* " in *" -name objects "*) ' + f"""{real} "$@"; printf '%s\\n' "find: '{theirs}': Permission denied" >&2; exit 1 ;; esac\n""" + f'exec {real} "$@"\n', encoding="utf-8") + shim.chmod(0o755) + result, rows = sweep() + assert result.returncode == 3, (result.stdout, result.stderr) + assert rows[str(c)][0] == "remove", rows[str(c)] + assert (" (1 director(ies) the dependents scan could not read - another user's, or " + "gone while read - are not looked in)") in result.stdout, result.stdout + result, rows = sweep("--yes", LANES_LANE=RUNNER) + assert result.returncode == 0, (result.stdout, result.stderr) + assert rows[str(c)][0] == "removed", rows[str(c)] @pytest.mark.skipif(getattr(os, "geteuid", lambda: 1)() == 0, From e5063e0db7f1dd1c90c4397401525ff03d7e9009 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 23 Oct 2026 21:57:51 +0000 Subject: [PATCH 47/53] Amendment 23 S6.2: the dependents scan reads `$TMPDIR` (or /tmp) to depth 8, where a Claude Code scratchpad keeps its clones, and the scope line names each root's depth (rv4b's SHOULD C on 811c10f, #198, slice 4) rv4b's pre-review of 811c10f found that `$TMPDIR` was scanned to depth 4 only. A Claude Code scratchpad is `/tmp/claude-///scratchpad`, so a clone made in one stands at depth 5 and its object directory at 7 (probe P3): it was not seen, `--yes` removed the checkout it borrowed from, and its `fsck` exited 10. - `sc_dep_root 8 "${TMPDIR:-/tmp}"`: repositories to depth 8, so a clone in a folder of a scratchpad's own (three levels further) is seen too. - The dry run's scope line takes each root's depth from the scan's own list, and says why `$TMPDIR` goes deeper. `DEPENDENTS_AFAR` takes a clone eight levels under `$TMPDIR` in a scratchpad's shape, and one in `$HOME` (rv4b's unpinned `:1675`). Red at e12d077: 2 failed, 6 passed (the scratchpad clone and the scope line). Green: the module's dependents and scope selection 18 passed, tests/test_repo_hygiene.py 253 passed (the comment names the scratchpad's shape relative to `$TMPDIR`, not a host path); the mutants rv4b-C-depth, rv4b-C-scope, r4b-home-depth (rv4b's) and l3-352-tmpdir (re-pointed at depth 8) KILLED. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 7d194593726d9ccb3e645cc132f10f7209bce383) --- lane-end | 15 ++++++++++----- tests/test_a23_sweep_checkouts.py | 15 +++++++++++---- 2 files changed, 21 insertions(+), 9 deletions(-) diff --git a/lane-end b/lane-end index 75cdc86..54fa69c 100755 --- a/lane-end +++ b/lane-end @@ -1682,8 +1682,13 @@ SH_Y # (b) WHAT LEANS ON WHAT, LOOKED FOR WHERE CLONES ARE MADE (lane # openRepoTools-3's SHOULD 352; S6.2 (b): *"no other clone's alternates # file names it"*): every repository under W's parent to depth 6, hidden - # directories included, and under `$TMPDIR` (or /tmp) and `$HOME` to depth - # 4, by their real paths. `find` lists each object directory, pruned there, + # directories included, under `$TMPDIR` (or /tmp) to depth 8, and under + # `$HOME` to depth 4, by their real paths. `$TMPDIR` goes deeper for + # Claude Code's scratchpads (rv4b's SHOULD C): one is + # `claude-///scratchpad` under it, so a clone made + # in it stands at depth 5, its object directory at 7, and a clone in a + # folder of its own there deeper still. `find` lists each object + # directory, pruned there, # and each `.git` FILE. An object directory with an alternates file is a # repository's store, and each line of that file names an object directory # the repository borrows from: `sc_dep_lines` holds `