From 026280cdcb1694d844a731a3769a0adb45b22587 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Wed, 14 Oct 2026 23:34:56 +0000 Subject: [PATCH 01/25] openRepoShape: `--install` records the ref it runs at in `openRepoShape.ref` beside the command it places, and the installed command then fetches setup.sh at that ref, hands setup.sh `--shape-ref` for it and checks the doctor's clone out at it, so a command installed from a release stays on that release while `$OPENREPOSHAPE_REF` still overrides it for one run (#564) An installed command read `$OPENREPOSHAPE_REF` at every run and fell back to main, so from v1.0.0 on, with main carrying v1.1.0 work, a command installed from a release drifted off it within hours. The ref is now NAMED by the variable or by the record, and both places that honoured only the exported variable (the doctor's checkout and setup.sh's `--shape-ref`) honour either. The record is a sidecar file, never an edit to the command: the placed command stays byte for byte the file it was installed from, so the installer's own `(unchanged)` verdict and any copy vendored and checked by sha256 still compare bytes and never see the record. An install with no ref records nothing and removes any record already there, which is the command exactly as before. The ref is checked before anything is placed, and the record is checked when it is read, naming the file and the two ways out. `--version` prints the ref in force. docs/cli.md is regenerated for the `--help` paragraph that says so. Co-Authored-By: Claude Opus 5.5 --- docs/cli.md | 11 +- openRepoShape | 123 +++++++++++++-- tests/test_openreposhape_command.py | 235 +++++++++++++++++++++++++++- 3 files changed, 355 insertions(+), 14 deletions(-) diff --git a/docs/cli.md b/docs/cli.md index c793a3e0..7e81e1af 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -150,11 +150,20 @@ opensoft/openRepoTools', and its own installer places them: curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoTools/main/openRepoTools | bash -s -- --install $OPENREPOSHAPE_REPO owner/name to fetch from (default opensoft/openRepoShape) - $OPENREPOSHAPE_REF the ref to fetch it at (default main) + $OPENREPOSHAPE_REF the ref to fetch it at (default: the ref --install + recorded, else main) $OPENREPOSHAPE_ORG the organisation, when --org is not given $OPENREPOSHAPE_BIN_DIR where --install puts it (default ~/.local/bin) $OPENREPOSHAPE_SETUP_SH run this setup.sh on disk instead of fetching one +`--install` RECORDS THE REF it runs at, in `openRepoShape.ref` beside the +command it places: $OPENREPOSHAPE_REF, which the one-line installer +(install.sh) sets to the release it installs. The installed command then +stays on that ref: it fetches setup.sh at it, setup.sh clones the standard +at it (`--shape-ref`), and `--doctor` checks its clone out at it. +$OPENREPOSHAPE_REF set for one run still wins, and `--version` prints the +ref in force. Installed with no ref, it records none and follows main. + `--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` over (default `.`) and prints one row per check and ONE verdict: `COMPLIANT`, `COMPLIANT, SHAPE BEHIND`, `DRIFTED`, `MISPLACED`, `INVALID`, diff --git a/openRepoShape b/openRepoShape index 58bbc3f2..246bf217 100755 --- a/openRepoShape +++ b/openRepoShape @@ -3,7 +3,12 @@ # # openRepoShape — the short way in: `openRepoShape Atlas --org `. # -# Install it once, from anywhere, with no checkout and no clone: +# Install it once, from anywhere, with no checkout and no clone. The newest +# release, pinned to it (`install.sh`, #564): +# +# curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +# +# Or this file at main, by hand, which follows main: # # gh api repos/opensoft/openRepoShape/contents/openRepoShape \ # -H 'Accept: application/vnd.github.raw' | bash -s -- --install @@ -28,9 +33,23 @@ set -euo pipefail # setup.sh reads the same variable as something to `git clone` — see # shape_clone_url below: one name, both uses. REPO="${OPENREPOSHAPE_REPO:-opensoft/openRepoShape}" -REF="${OPENREPOSHAPE_REF:-main}" SELF="${BASH_SOURCE[0]:-$0}" +# THE REF, AND WHETHER IT WAS NAMED (#564). A NAMED ref is one somebody +# chose: setup.sh is told it (`--shape-ref`) and `--doctor` checks its clone +# out at it, where an unnamed `main` leaves both to the default branch. Two +# things name one, and the first wins: `$OPENREPOSHAPE_REF`, set for a run, +# and the ref `--install` RECORDED beside this file (`$SELF.ref`, read by +# `read_recorded_ref` below, once `checked_ref` exists to check it). With +# neither, it is `main`, unnamed, exactly as before there was a record. +REF="${OPENREPOSHAPE_REF:-main}" +REF_NAMED=0 +[ -z "${OPENREPOSHAPE_REF:-}" ] || REF_NAMED=1 + +#: The record's name is the command's own plus this, so the record of +#: `~/.local/bin/openRepoShape` is `~/.local/bin/openRepoShape.ref`. +RECORD_SUFFIX=".ref" + PROJECT="" ORG="" MODE="run" @@ -89,11 +108,20 @@ opensoft/openRepoTools', and its own installer places them: curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoTools/main/openRepoTools | bash -s -- --install $OPENREPOSHAPE_REPO owner/name to fetch from (default opensoft/openRepoShape) - $OPENREPOSHAPE_REF the ref to fetch it at (default main) + $OPENREPOSHAPE_REF the ref to fetch it at (default: the ref --install + recorded, else main) $OPENREPOSHAPE_ORG the organisation, when --org is not given $OPENREPOSHAPE_BIN_DIR where --install puts it (default ~/.local/bin) $OPENREPOSHAPE_SETUP_SH run this setup.sh on disk instead of fetching one +`--install` RECORDS THE REF it runs at, in `openRepoShape.ref` beside the +command it places: $OPENREPOSHAPE_REF, which the one-line installer +(install.sh) sets to the release it installs. The installed command then +stays on that ref: it fetches setup.sh at it, setup.sh clones the standard +at it (`--shape-ref`), and `--doctor` checks its clone out at it. +$OPENREPOSHAPE_REF set for one run still wins, and `--version` prints the +ref in force. Installed with no ref, it records none and follows main. + `--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` over (default `.`) and prints one row per check and ONE verdict: `COMPLIANT`, `COMPLIANT, SHAPE BEHIND`, `DRIFTED`, `MISPLACED`, `INVALID`, @@ -288,9 +316,54 @@ machine_report() { say " run: openRepoShape --preflight (offers what is missing; each install asks first)" } +# THE REF THIS INSTALL RAN AT, WRITTEN BESIDE THE COMMAND IT PLACED (#564). +# Without it an installed command read `$OPENREPOSHAPE_REF` at every run and +# fell back to `main`, so a command installed from a release fetched the next +# day's main: the release it was installed from lasted until main moved. +# +# A FILE BESIDE THE COMMAND, NEVER AN EDIT TO IT. The placed command stays +# byte for byte the file it was installed from, so `cmp` against the release's +# own `openRepoShape` still says whether it IS that release: the `(unchanged)` +# verdict `install_commands` gives, and any copy vendored and checked by +# sha256 elsewhere, compare the command's bytes and never see the record. A +# command copied away without its record follows `main` again, and says so in +# `--version`. +# +# THE REF RECORDED IS THE ONE IN FORCE: `$OPENREPOSHAPE_REF`, else the record +# beside the copy being run, so a pinned command re-installing itself stays +# pinned. With neither, any record already there is REMOVED, which is what +# makes an install at main follow main however the command was installed +# before. Every outcome is one line, and no ref and no record is no line at +# all. The record's old value is echoed only when it passes the same shape +# check, because a hand-edited file is not this command's to print. +record_ref() { + local record="$1$RECORD_SUFFIX" was="" + if [ -f "$record" ]; then + was="$(< "$record")" || was="" + case "$was" in + '' | [!A-Za-z0-9]* | *..* | *[!A-Za-z0-9._/-]*) was="another ref" ;; + esac + [ "${#was}" -le 255 ] || was="another ref" + fi + if [ "$REF_NAMED" -eq 0 ]; then + [ -e "$record" ] || return 0 + rm -f -- "$record" + say "openRepoShape: no longer pinned to ${was:-another ref}; it follows main, or \$OPENREPOSHAPE_REF where set (removed $record)" + elif [ "$was" = "$REF" ]; then + say "openRepoShape: already pinned to $REF in $record (unchanged)" + else + printf '%s\n' "$REF" >"$record" + say "openRepoShape: pinned to $REF${was:+ (was $was)}, recorded in $record" + fi +} + install_commands() { local dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" local name target source verb placed=0 + # THE REF IS CHECKED BEFORE ANYTHING IS COLLECTED OR PLACED: it is about + # to be written down and read back by every later run, and a refusal + # after the copy would leave a new command beside an old record. + [ "$REF_NAMED" -eq 0 ] || checked_ref "$REF" collect_commands mkdir -p "$dir" for name in "${INSTALLABLES[@]}"; do @@ -307,6 +380,9 @@ install_commands() { placed=$((placed + 1)) say "$name: $verb" done + # The record belongs to the one command that reads it, and is written + # only once that command is in place. + record_ref "$dir/openRepoShape" # THE COUNT, out loud, and kept at one installable (#92): it costs a line # and is what keeps $INSTALLABLES honest — a person reading N lines cannot # tell whether an N+1th was meant to be there. The sentence that made @@ -354,7 +430,8 @@ find_python() { # A CHECKOUT OF THE STANDARD, made the way `setup.sh` self-bootstraps one: # `git clone` of `shape_clone_url`, into the temporary directory `workdir` -# owns, checked out at $OPENREPOSHAPE_REF when one was named. Sets $SHAPE_DIR. +# owns, checked out at $REF when one was named: by $OPENREPOSHAPE_REF, or by +# the record `--install` left beside this file (#564). Sets $SHAPE_DIR. # # WITHOUT `--depth 1`, WHICH IS THE ONE DEVIATION FROM setup.sh AND IS FORCED. # A shallow clone has exactly ONE commit, and the doctor's shape-currency row @@ -378,7 +455,7 @@ shape_checkout() { die "could not clone $REPO into a temporary directory, so there is no standard to check against. \`--doctor\` needs one: run it from a checkout of openRepoShape, or make the clone work." - if [ -n "${OPENREPOSHAPE_REF:-}" ]; then + if [ "$REF_NAMED" -eq 1 ]; then checked_ref "$REF" # ` --`, never `-- `: a `--` BEFORE the argument makes git # read it as a PATHSPEC and try to restore a file of that name from @@ -397,19 +474,44 @@ shape_checkout() { # CHARACTER next, because a newline in it could forge a line of output before # this script has said a word; and only then the SHAPE, which is safe to echo # back because it has passed both. +# +# WHOSE REF IT IS, AND THE WAY OUT, ARE ARGUMENTS (#564): `$OPENREPOSHAPE_REF` +# by default, or the record `--install` wrote, whose refusal also says how to +# replace it. An EMPTY ref is refused too, which no caller could reach before +# the record: an empty variable is no ref at all, while an empty file is one. checked_ref() { - local ref="$1" + local ref="$1" what="${2:-\$OPENREPOSHAPE_REF}" out="${3:+ $3}" + [ -n "$ref" ] || + die "$what is empty, which is not a branch, tag or commit this tool will pass to \`git checkout\`.$out" if [ "${#ref}" -gt 255 ]; then - die "\$OPENREPOSHAPE_REF is ${#ref} characters long; a branch, tag or commit this tool will pass to \`git checkout\` is at most 255 characters." + die "$what is ${#ref} characters long; a branch, tag or commit this tool will pass to \`git checkout\` is at most 255 characters.$out" fi case "$ref" in - *[![:print:]]*) die "\$OPENREPOSHAPE_REF carries a control character, which is not a branch, tag or commit this tool will pass to \`git checkout\`." ;; + *[![:print:]]*) die "$what carries a control character, which is not a branch, tag or commit this tool will pass to \`git checkout\`.$out" ;; esac case "$ref" in - [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) die "\$OPENREPOSHAPE_REF is '$ref', which is not a branch, tag or commit this tool will pass to \`git checkout\`: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'." ;; + [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) die "$what is '$ref', which is not a branch, tag or commit this tool will pass to \`git checkout\`: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'.$out" ;; esac } +# THE RECORD `--install` LEFT BESIDE THIS FILE, READ ONCE, BEFORE ANY RUN +# (#564). Only when no `$OPENREPOSHAPE_REF` was set, which is what lets that +# variable win for a run, and only beside a FILE: a script arriving on stdin +# has no place of its own to have been installed at. CHECKED HERE, AT ONCE, +# rather than where the ref is used: it is a file on disk that every run +# reads, `--version` prints it, and a value nobody typed is not echoed until +# it has passed. `$(< file)` is bash's own read, so no `cat` is needed. +read_recorded_ref() { + local record="$SELF$RECORD_SUFFIX" + [ "$REF_NAMED" -eq 0 ] && [ -f "$SELF" ] && [ -f "$record" ] || return 0 + REF="$(< "$record")" || + die "could not read $record, the ref \`--install\` recorded. Run the installer again, or delete that file to follow main." + checked_ref "$REF" "the ref recorded in $record" \ + "Run the installer again to record a release, or delete that file to follow main." + REF_NAMED=1 +} +read_recorded_ref + # `--doctor`: RUN THE STANDARD'S OWN SCRIPT, never a reimplementation here. # Beside a checkout -- this file sitting in one, which is how a developer runs # it -- that checkout answers, and nothing is fetched at all. Installed, or @@ -671,7 +773,8 @@ ARGS=() [ -z "$PROJECT" ] || ARGS+=(--project "$PROJECT") # Fetched at $REF, so setup.sh self-bootstraps its checkout at $REF too: # otherwise the script that runs and the standard it clones are two commits. -[ -z "${OPENREPOSHAPE_REF:-}" ] || ARGS+=(--shape-ref "$REF") +# Whenever the ref was NAMED, by the variable or by the record (#564). +[ "$REF_NAMED" -eq 0 ] || ARGS+=(--shape-ref "$REF") ARGS+=(${PASSTHROUGH[@]+"${PASSTHROUGH[@]}"}) # $OPENREPOSHAPE_SETUP_SH is the OFFLINE TEST PATH: a setup.sh already on disk diff --git a/tests/test_openreposhape_command.py b/tests/test_openreposhape_command.py index d6b94f82..59cb0c00 100644 --- a/tests/test_openreposhape_command.py +++ b/tests/test_openreposhape_command.py @@ -34,7 +34,7 @@ import pytest -from conftest import REPO, WINDOWS_SKIP +from conftest import REPO, WINDOWS_SKIP, git COMMAND = REPO / "openRepoShape" SETUP = REPO / "setup.sh" @@ -113,7 +113,8 @@ def command_env(home: Path | None = None, env: dict | None = None, def run_cmd(*args: str, home: Path | None = None, env: dict | None = None, - setup_sh: Path | None = SETUP) -> subprocess.CompletedProcess: + setup_sh: Path | None = SETUP, + command: Path = COMMAND) -> subprocess.CompletedProcess: """The command, run from its file, with that environment. `input=""` means stdin is a pipe rather than a terminal, which is what the @@ -121,8 +122,11 @@ def run_cmd(*args: str, home: Path | None = None, env: dict | None = None, controlling terminal either (#492): setup asks at that terminal when stdin is not one, and the terminal pytest was started from is nobody's answer. + + `command` is this checkout's file unless a test names an INSTALLED copy, + which is how the record `--install` leaves beside one is read (#564). """ - return subprocess.run(["bash", str(COMMAND), *args], capture_output=True, + return subprocess.run(["bash", str(command), *args], capture_output=True, text=True, check=False, input="", env=command_env(home, env, setup_sh), start_new_session=True) @@ -1002,3 +1006,228 @@ def test_install_from_stdin_fetches_itself_into_a_live_workdir(offline_github, assert stat.S_IMODE(target.stat().st_mode) == 0o755, name assert target.read_bytes() == (REPO / name).read_bytes(), name assert f"{name}: installed at" in result.stdout + + +# --- the ref `--install` records, and the installed command runs (#564) ----- + +#: A ref for an install to record. Nothing fetches it: every install below +#: runs from this checkout's file, which copies its own bytes, and every run +#: after it is offline. `OTHER` is what `$OPENREPOSHAPE_REF` names for one +#: run, over the record. +PINNED = "v9.9.9" +OTHER = "other" + + +def install_at(tmp_path, ref=None, command=COMMAND): + """`--install` from `command` into `/bin`, with + `$OPENREPOSHAPE_REF` set to `ref` when one is given; the result, and the + path of the command it places.""" + bin_dir = tmp_path / "bin" + env = {"OPENREPOSHAPE_BIN_DIR": str(bin_dir)} + if ref is not None: + env["OPENREPOSHAPE_REF"] = ref + return (run_cmd("--install", home=tmp_path, env=env, command=command), + bin_dir / "openRepoShape") + + +def record_of(installed: Path) -> Path: + """Where `--install` records the ref of the command at `installed`.""" + return installed.with_name(installed.name + ".ref") + + +def version_of(installed: Path, env: dict | None = None) -> str: + result = run_cmd("--version", command=installed, env=env) + assert result.returncode == 0, result.stderr + return result.stdout.strip() + + +def test_install_records_the_ref_beside_a_command_it_leaves_unchanged( + tmp_path): + """THE RECORD IS A FILE BESIDE THE COMMAND, AND THE COMMAND IS THE FILE. + + The placed bytes must still be the bytes it was installed from, because + everything that asks whether an installed command IS a release compares + bytes: this installer's own `(unchanged)` verdict, and a copy vendored + elsewhere and checked by sha256 against the release's own file. So the + ref goes in `openRepoShape.ref`, said on its own line between the + command's line and the count, and `--version` then names it with no + variable set, while the variable still wins for the one run it is set + for. + """ + result, installed = install_at(tmp_path, PINNED) + assert result.returncode == 0, result.stderr + record = record_of(installed) + assert record.read_bytes() == f"{PINNED}\n".encode("ascii") + assert installed.read_bytes() == COMMAND.read_bytes(), ( + "the command must stay byte for byte the file it was installed from") + assert result.stdout.splitlines()[:3] == [ + f"openRepoShape: installed at {installed}", + f"openRepoShape: pinned to {PINNED}, recorded in {record}", + f"openRepoShape: 1 of 1 placed in {installed.parent}"] + assert version_of(installed) == \ + f"openRepoShape (opensoft/openRepoShape @ {PINNED})" + assert version_of(installed, {"OPENREPOSHAPE_REF": OTHER}) == \ + f"openRepoShape (opensoft/openRepoShape @ {OTHER})" + + +def test_an_install_with_no_ref_records_nothing_and_follows_main(tmp_path): + """No ref at install time is the install as it always was: the same + lines, no record, and `main`, unnamed, in `--version`.""" + result, installed = install_at(tmp_path) + assert result.returncode == 0, result.stderr + assert not record_of(installed).exists() + assert "pinned" not in result.stdout + assert version_of(installed) == \ + "openRepoShape (opensoft/openRepoShape @ main)" + + +def test_installing_again_keeps_moves_or_drops_the_record(tmp_path): + """Idempotent at the same ref, moved by a new one, dropped by none. + + A second install at the same ref changes neither file and says so twice. + A newer ref replaces the record and names the one it replaced, which is + what an upgrade looks like. The installed command re-installing itself + with no variable set keeps ITS ref, because that is the ref it runs at. + And an install with no ref at all removes the record, so a command put + back at main follows main however it was installed before. + """ + install_at(tmp_path, PINNED) + same, installed = install_at(tmp_path, PINNED) + record = record_of(installed) + assert same.returncode == 0, same.stderr + assert f"openRepoShape: already pinned to {PINNED} in {record} " \ + "(unchanged)" in same.stdout + assert same.stdout.count("(unchanged)") == 2, same.stdout + + newer, _ = install_at(tmp_path, "v10.0.0") + assert f"openRepoShape: pinned to v10.0.0 (was {PINNED}), recorded in " \ + f"{record}" in newer.stdout + itself, _ = install_at(tmp_path, command=installed) + assert "openRepoShape: already pinned to v10.0.0" in itself.stdout + assert record.read_text(encoding="utf-8") == "v10.0.0\n" + + dropped, _ = install_at(tmp_path) + assert dropped.returncode == 0, dropped.stderr + assert "openRepoShape: no longer pinned to v10.0.0; it follows main" \ + in dropped.stdout + assert not record.exists() + assert version_of(installed).endswith("@ main)") + + +def test_a_ref_that_is_not_one_is_refused_before_anything_is_placed( + tmp_path): + """The ref is about to be written down and read by every later run, so + it is checked first, the way the doctor checks it before a checkout.""" + result, installed = install_at(tmp_path, "bad..ref") + assert result.returncode == 2, result.stdout + result.stderr + assert "$OPENREPOSHAPE_REF is 'bad..ref'" in result.stderr + assert not installed.parent.exists(), "a refused install placed something" + + +@pytest.mark.parametrize("text, shown", [ + pytest.param(b"", "is empty", id="empty"), + pytest.param(b"-x\n", "is '-x'", id="an-option"), + pytest.param(b"v1..v2\n", "is 'v1..v2'", id="a-range"), + pytest.param(b"v1\nforged\n", "carries a control character", + id="two-lines"), + pytest.param(b"a" * 256 + b"\n", "is 256 characters long", + id="over-long"), +]) +def test_a_record_that_is_not_a_ref_is_refused_by_name(tmp_path, text, shown): + """A record is a file on disk that every run reads, so it is checked + when it is read, before `--version` can print it, and the refusal names + the file and the two ways out. The control character and the over-long + value are refused without being echoed. `$OPENREPOSHAPE_REF`, set, is + the way past it for one run, because then the record is not read.""" + _result, installed = install_at(tmp_path, PINNED) + record_of(installed).write_bytes(text) + result = run_cmd("--version", command=installed) + assert result.returncode == 2, result.stdout + result.stderr + assert f"the ref recorded in {record_of(installed)} {shown}" \ + in result.stderr + assert "Run the installer again to record a release, or delete that " \ + "file to follow main." in result.stderr + assert "forged" not in result.stderr and "a" * 256 not in result.stderr + assert result.stdout == "" + assert version_of(installed, {"OPENREPOSHAPE_REF": OTHER}).endswith( + f"@ {OTHER})") + + +#: WHICH REF EACH LATER RUN USES: recorded at install, then overridden by the +#: variable for one run, then neither. The third column is what the run must +#: be handed; `None` is `main`, unnamed, which nothing is told. +RECORDED_RUNS = [ + pytest.param(PINNED, None, PINNED, id="the-record"), + pytest.param(PINNED, OTHER, OTHER, id="the-variable-over-the-record"), + pytest.param(None, None, None, id="no-record"), +] + + +@pytest.mark.parametrize("recorded, variable, expected", RECORDED_RUNS) +def test_setup_sh_clones_the_standard_at_the_ref_in_force( + tmp_path, recorded, variable, expected): + """`--shape-ref` reaches setup.sh whenever a ref was NAMED, and the + record names one: the setup.sh this command fetched at that ref then + clones the standard at the same ref, not at main.""" + _result, installed = install_at(tmp_path, recorded) + stub = tmp_path / "setup.sh" + stub.write_text(STUB_SETUP_SH, encoding="utf-8") + result = run_cmd("--preflight", command=installed, setup_sh=stub, + env=None if variable is None + else {"OPENREPOSHAPE_REF": variable}) + assert result.returncode == 0, result.stdout + result.stderr + match = re.search(rf"{STUB_MARKER} in .+ args: (.*)", result.stdout) + assert match, result.stdout + result.stderr + forwarded = match.group(1).split() + if expected is None: + assert "--shape-ref" not in forwarded, forwarded + else: + at = forwarded.index("--shape-ref") + assert forwarded[at + 1] == expected, forwarded + + +def standard_at_three_refs(tmp_path) -> Path: + """A bare repository whose `shape-doctor.py` says which ref it is. + + Three commits: the `PINNED` tag, the `OTHER` branch and `main`, each + with a doctor that prints its own name and nothing else, so a run + reports which ref the command checked its clone out at. + """ + seed = tmp_path / "seed" + seed.mkdir() + git("init", "-q", "-b", "main", cwd=seed) + + def commit(name): + (seed / "shape-doctor.py").write_text( + f"print('STUB DOCTOR AT {name}')\n", encoding="utf-8") + git("add", "-A", cwd=seed) + git("commit", "-q", "-m", name, cwd=seed) + + commit(PINNED) + git("tag", PINNED, cwd=seed) + git("switch", "-q", "-c", OTHER, cwd=seed) + commit(OTHER) + git("switch", "-q", "main", cwd=seed) + commit("main") + bare = tmp_path / "standard.git" + git("clone", "-q", "--bare", str(seed), str(bare), cwd=tmp_path) + return bare + + +@pytest.mark.parametrize("recorded, variable, expected", RECORDED_RUNS) +def test_the_doctor_checks_its_clone_out_at_the_ref_in_force( + tmp_path, recorded, variable, expected): + """The installed `--doctor` clones the standard, and checks the clone + out at the ref in force: the record, the variable over it, or neither, + which leaves the clone on its default branch.""" + _result, installed = install_at(tmp_path, recorded) + temporary = tmp_path / "tmp" + temporary.mkdir() + env = {"OPENREPOSHAPE_REPO": str(standard_at_three_refs(tmp_path)), + "TMPDIR": str(temporary)} + if variable is not None: + env["OPENREPOSHAPE_REF"] = variable + result = run_cmd("--doctor", str(tmp_path), command=installed, env=env) + assert result.returncode == 0, result.stdout + result.stderr + assert f"STUB DOCTOR AT {expected or 'main'}" in result.stdout, ( + result.stdout + result.stderr) From d0cdbf4c7a500fa912d5e2554e799cf8dfeba1e5 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Wed, 14 Oct 2026 23:35:06 +0000 Subject: [PATCH 02/25] install.sh: one line, `curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash`, installs the newest release's openRepoShape pinned to that release, installs git-filter-repo with pipx where pipx is and it is not or prints the one command that would, and ends with `openRepoShape --preflight`; README, the handbook and the v1.0.1 release notes lead with it (#564) Brett Heap, 2026-10-14: "we need it to be more simple to install ... now and after, we need a simple install process", and "v1.0.1 soon". install.sh is attached to every release, so `releases/latest/download/` is always the newest. It reads the newest tag from github.com's `releases/latest` redirect with no gh and no login, asking `gh api` only when the redirect names none; takes a named release as its one argument; fetches that release's `openRepoShape`, checks it is the command, and runs its `--install` with `$OPENREPOSHAPE_REF` set to the release, then asks the installed command's `--version` that it stayed there. Every step says what it did in one line. It never runs sudo. Exit 0 installed and ready, 1 installed but the preflight or the pin needs attention, 2 nothing installed; git-filter-repo never moves it. Everything runs from a `main` called on the last line, so a cut-off download runs nothing. README.md: step 3 and the worked example lead with the one line, the gh api and raw-URL lines stay as the by-hand way at main, and Layout gains install.sh; the cap moves 1422 -> 1450, with the reason in test_readme_is_short_enough_to_be_read. docs/handbook.html says the same, names the new README blob and adds the (#564) clause. install.sh joins SHIPPED_BASH and the macOS job's `/bin/bash -n` step, and one test holds the one line byte for byte in every file that carries it. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/tests.yml | 4 + README.md | 42 ++- docs/handbook.html | 47 ++- docs/release-notes/v1.0.1.md | 65 ++++ install.sh | 361 +++++++++++++++++++++ tests/test_install_sh.py | 596 +++++++++++++++++++++++++++++++++++ tests/test_repo_hygiene.py | 57 +++- 7 files changed, 1143 insertions(+), 29 deletions(-) create mode 100644 docs/release-notes/v1.0.1.md create mode 100755 install.sh create mode 100644 tests/test_install_sh.py diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e5e47255..08a57826 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -282,11 +282,15 @@ jobs: # for the reason this one exists — those two are what a stock Mac's bash # 3.2 has to parse, and the repository that ships a file is the one that # can say so. + # + # `install.sh` joined the list in v1.0.1 (#564): the one-line install, + # `curl … | bash` on a Mac, hands it to this same `/bin/bash`. - name: /bin/bash parses every bash entry point run: | /bin/bash --version /bin/bash -n setup.sh /bin/bash -n openRepoShape + /bin/bash -n install.sh - name: tests run: python3 -m pytest tests -q diff --git a/README.md b/README.md index f3b378dc..147b3cb5 100644 --- a/README.md +++ b/README.md @@ -102,10 +102,10 @@ is on no `PATH` by default, and its installer finishes by printing the line that puts it there — run that line, or the next shell cannot find `brew`. Then `brew install gh`, and `brew install git-filter-repo` only if you are ADOPTING an existing repository rather than scaffolding a new one. The stock `/bin/bash` is -3.2 and is enough: `setup.sh` and the `openRepoShape` command use no syntax -newer than that, and CI runs the whole suite on `macos-latest`, parsing both -there with `/bin/bash` itself. That the login shell is zsh changes nothing — -both entry points are run under `bash` explicitly. +3.2 and is enough: `setup.sh`, `install.sh` and the `openRepoShape` command use +no syntax newer than that, and CI runs the whole suite on `macos-latest`, +parsing all three there with `/bin/bash` itself. That the login shell is zsh +changes nothing — all three entry points are run under `bash` explicitly. @@ -225,7 +225,27 @@ offer to configure git's credential helper, or run `gh auth setup-git`: the first commits go up with a plain `git push` over HTTPS, and without it the three repositories are created and the push fails. -**3. Install the `openRepoShape` command**, once: +**3. Install the `openRepoShape` command**, once, in one line: + +```sh +curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +``` + +`install.sh` is attached to every release, so that line installs the newest: +it reads which one that is from github.com's `releases/latest` redirect (no +`gh`, no login), installs that release's command PINNED to it, installs +`git-filter-repo` with `pipx` when `pipx` is there and it is not, or prints +the one command that would, and ends with `openRepoShape --preflight`. It +never runs `sudo`. `| bash -s -- v1.0.1` installs a named release instead. +Exit 0 is installed and ready, 1 installed but the preflight or the pin needs +you, 2 nothing installed. Run the same line again to upgrade. + +**Pinned** means the command stays on its release while main moves on: +`--install` records the ref in `openRepoShape.ref` beside the command, and the +command fetches `setup.sh`, clones the standard and runs `--doctor` at it. +`$OPENREPOSHAPE_REF` overrides it for one run; `--version` prints the ref. + +By hand, the same command at main, which records no ref and follows main: ```sh gh api repos/opensoft/openRepoShape/contents/openRepoShape \ @@ -239,7 +259,7 @@ curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRep | bash -s -- --install ``` -It installs ONE command into `~/.local/bin`, idempotently — a second run +Each way installs ONE command into `~/.local/bin`, idempotently — a second run prints `unchanged` — and prints the `export PATH=…` line if that directory is not on `PATH`. The estate verbs `park` and `resume`, for [carrying in-flight work](#carrying-in-flight-work-to-another-workstation), are @@ -380,7 +400,14 @@ records as whose act the election was, and it defaults to the `gh` login. #### Install the command -Two ways in, the short one first. Install `openRepoShape` as a command: +Two ways in, the short one first. Install `openRepoShape` as a command, the +newest release and pinned to it, as step 3 above does: + +```sh +curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +``` + +or by hand, at main, through the API: ```sh gh api repos/opensoft/openRepoShape/contents/openRepoShape \ @@ -1389,6 +1416,7 @@ scripts/path_classify.py the classifier over the path policy scripts/shape_materialize.py the ONE materializer, used by both tools scripts/validate-repository-naming.py openRepoShape installs itself as a command; runs setup.sh +install.sh the one-line install: a release, pinned setup.sh the shim: find an interpreter, hand over setup-project.py THE FLOW, and the Windows way in (no bash there) scaffold-project.py creates the three repositories and the pins diff --git a/docs/handbook.html b/docs/handbook.html index 4e01a52b..377f934a 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -485,11 +485,11 @@

openRepoShape

convention, as data and as running code.

The fastest path — macOS, Linux, WSL2

-
curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRepoShape \
-    | bash -s -- --install
-openRepoShape --preflight
+          
curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash
 openRepoShape <Project> --org <your-org>
-

That first line installs ONE command into ~/.local/bin. The +

That first line installs ONE command into ~/.local/bin: the + newest release, pinned to it, with git-filter-repo where pipx can + install it, and it ends with openRepoShape --preflight. The estate verbs park and resume, for carrying in-flight work to another workstation, are opensoft/openRepoTools’ and have a one-liner of @@ -672,11 +672,11 @@

What this machine needs

brew install gh    # macOS, or Linux with Homebrew

Then brew install git-filter-repo only if you are ADOPTING an existing repository rather than scaffolding a new one.

-

The stock /bin/bash is 3.2 and is enough: setup.sh and the - openRepoShape command use no syntax newer than that, and CI runs the whole - suite on macos-latest, parsing both there with /bin/bash itself. - That the login shell is zsh changes nothing — both entry points are run under - bash explicitly.

+

The stock /bin/bash is 3.2 and is enough: setup.sh, + install.sh and the openRepoShape command use no syntax newer than + that, and CI runs the whole suite on macos-latest, parsing all three there with + /bin/bash itself. That the login shell is zsh changes nothing — all three + entry points are run under bash explicitly.

The tool offers the rest

The preflight names what is missing and, for gh, its login and its credential helper, prints the command for THIS machine and asks once — Type yes to continue: — @@ -755,11 +755,29 @@

Make one machine setting first

Install the openRepoShape command, once

Install the openRepoShape command, once, inside WSL

Fetch the flow, in the directory the new project should land in

+
curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash
+

install.sh is attached to every release, so + that line installs the newest: it reads which one that is from github.com’s + releases/latest redirect (no gh, no login), installs that + release’s command PINNED to it, installs git-filter-repo with + pipx when pipx is there and it is not, or prints the one command + that would, and ends with openRepoShape --preflight. It never runs + sudo. | bash -s -- v1.0.1 installs a named release instead. Exit + 0 is installed and ready, 1 installed but the preflight or the pin needs you, 2 nothing + installed. Run the same line again to upgrade.

+

Pinned means the command stays on its + release while main moves on: --install records the ref in + openRepoShape.ref beside the command, and the command fetches + setup.sh, clones the standard and runs --doctor at it. + $OPENREPOSHAPE_REF overrides it for one run; --version prints the + ref.

+

By hand, the same command at main, which records no ref and follows main:

gh api repos/opensoft/openRepoShape/contents/openRepoShape \
     -H 'Accept: application/vnd.github.raw' | bash -s -- --install

Or, where raw downloads are not blocked, the same bytes over HTTPS:

curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRepoShape \
     | bash -s -- --install
+

By hand, the same command at main, which records no ref and follows main:

curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRepoShape \
     | bash -s -- --install

Or, where raw downloads are blocked, the same bytes through the API:

@@ -774,13 +792,13 @@

Fetch the flow, in the directory the new project should an exhausted stdin, so it cannot ask for the one yes before three repositories exist — it refuses instead. Downloading the file first leaves stdin attached to the terminal you are typing at.

-

It installs ONE command into ~/.local/bin, +

Each way installs ONE command into ~/.local/bin, idempotently — a second run prints unchanged — and prints the export PATH=… line if that directory is not on PATH. The estate verbs park and resume, for carrying in-flight work, are opensoft/openRepoTools’ and have an install line of their own.

-

It installs ONE command into ~/.local/bin, +

Each way installs ONE command into ~/.local/bin, idempotently — a second run prints unchanged — and prints the export PATH=… line if that directory is not on PATH. It is one file at the repository root and implements nothing: @@ -1750,6 +1768,7 @@

Layout of this repository

scripts/shape_materialize.py
the ONE materializer, used by both tools
scripts/validate-repository-naming.py
answers the naming question for one name
openRepoShape
installs itself as a command; runs setup.sh
+
install.sh
the one-line install: a release, pinned
setup.sh
the shim: find an interpreter, hand over
setup-project.py
THE FLOW, and the Windows way in (no bash there)
scaffold-project.py
creates the three repositories and the pins
@@ -1886,9 +1905,11 @@

For AI assistants

adoption’s --project as the source’s own name, added (#499); the merged-away verdict, whose row apply drops, added (#502); the line-ending block an adoption appends to a root .gitattributes the source keeps, - and its verification as the source’s bytes plus that block, added (#471) — a page + and its verification as the source’s bytes plus that block, added (#471); the one-line + install, install.sh, which installs the newest release pinned to it, and the + command’s openRepoShape.ref record, added (#564) — a page cannot name the commit that contains it, and the blob below is the fact a test checks. - README.md blob f3b378dc5704de79ee63972d0ea0be14dc693209. Every command, flag, exit code + README.md blob 147b3cb5c1d91c96196d6cef09ba984cfa0f6576. Every command, flag, exit code and file name on this page is quoted from that README and from AGENTS.md.

diff --git a/docs/release-notes/v1.0.1.md b/docs/release-notes/v1.0.1.md new file mode 100644 index 00000000..a5c5e3cb --- /dev/null +++ b/docs/release-notes/v1.0.1.md @@ -0,0 +1,65 @@ +# openRepoShape v1.0.1 + +v1.0.1 is v1.0.0 plus one change: openRepoShape installs in one line, and the +command it installs stays on the release it was installed from (#564). + +## What 1.0.1 adds + +- **The one-line install.** `install.sh` is attached to every release, so + this line always installs the newest one: + + ```sh + curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash + ``` + + It reads which release is the newest from github.com's `releases/latest` + redirect, with no `gh` and no login. It fetches that release's + `openRepoShape` and installs it into `~/.local/bin`, or + `$OPENREPOSHAPE_BIN_DIR`, pinned to the release. It installs + `git-filter-repo` 2.47.0 with `pipx` when `pipx` is there and + `git-filter-repo` is not, and otherwise prints the one command that + installs it. It ends with `openRepoShape --preflight`. It never runs `sudo` + and installs nothing else. It exits 0 when the command is installed and the + preflight passed, 1 when the command is installed but the preflight did not + pass or the command does not stay on the release, and 2 when it installed + nothing. +- **An installed command that stays on its release.** `openRepoShape + --install` records the ref it runs at, `$OPENREPOSHAPE_REF`, in + `openRepoShape.ref` beside the command it places. The installed command then + fetches `setup.sh` at that ref, `setup.sh` clones the standard at it + (`--shape-ref`), and `--doctor` checks its clone out at it. In v1.0.0 an + installed command fetched main on every run unless `$OPENREPOSHAPE_REF` was + exported. `$OPENREPOSHAPE_REF` set for one run still overrides the record, + and `openRepoShape --version` prints the ref in force. An install with no + ref records nothing, removes any record already there, and follows main. The + installed command is byte for byte the release's own file. + +## Install and upgrade + +- **The newest release:** the line above. Run it again to upgrade to the + newest release. +- **A named release:** + `curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash -s -- v1.0.1`, + or `bash install.sh v1.0.1` from a checkout or a download. +- **What is installed:** `openRepoShape --version` prints + `openRepoShape (opensoft/openRepoShape @ v1.0.1)`. +- **By hand, at main:** the README's + `gh api repos/opensoft/openRepoShape/contents/openRepoShape … | bash -s -- --install` + line still installs main's command, which records no ref and follows main. +- **Windows:** `install.sh` is bash, for macOS, Linux and WSL2. Native + Windows still starts with `py setup-project.py`, as the README says. +- **A project's pin** on the standard works as before. Follow + [How to pin 1.0.0](v1.0.0.md#how-to-pin-100) with the v1.0.1 release + commit. + +## A script that runs `--install` + +A script that runs `openRepoShape --install` with `$OPENREPOSHAPE_REF` +exported now pins the installed command to that value. If the value is not a +ref the command should run, unset it for the install. To keep the command on a +commit or a tag, set it to that commit or tag. + +## Known limits + +v1.0.1 changes nothing else, so its known limits are v1.0.0's: +[Known limits in v1.0.0](v1.0.0.md#known-limits). diff --git a/install.sh b/install.sh new file mode 100755 index 00000000..8713a67a --- /dev/null +++ b/install.sh @@ -0,0 +1,361 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 +# +# install.sh — openRepoShape in one line, pinned to a release (#564): +# +# curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +# +# A named release instead of the newest: `… | bash -s -- v1.0.1`. From a +# checkout or a download, the same: `bash install.sh []`. Running the +# same line again upgrades to the newest release. This file is attached to +# every GitHub release, which is what makes that URL always the newest one. +# +# FOUR STEPS, IN THIS ORDER, EACH SAYING IN A LINE WHAT IT DID: +# +# 1. THE RELEASE: the one named, or the newest, read from the redirect +# github.com answers `releases/latest` with. That needs no `gh`, no token +# and no login, because the repository is public; `gh api` is asked only +# when the redirect names no release and `gh` is installed. +# 2. THAT RELEASE'S `openRepoShape`, fetched into a temporary directory, +# checked to be the command, and installed by its own `--install` with +# `$OPENREPOSHAPE_REF` set to the release. `--install` records the ref +# beside the command it places, and the command then stays on it. +# 3. GIT-FILTER-REPO, which `adopt-project.py` needs and nothing else does: +# said when it is there; installed with `pipx` when pipx is there and it +# is not; otherwise the one command that installs it, printed. +# 4. THE INSTALLED COMMAND'S `--preflight`, LAST, and its verdict. +# +# IT NEVER RUNS sudo AND INSTALLS NOTHING ELSE. The command and, through pipx, +# git-filter-repo are the whole of what it installs. The preflight it ends +# with offers what is missing behind a typed `yes`, at the person's own +# terminal, exactly as `openRepoShape --preflight` always has. +# +# EXIT STATUS: +# 0 installed, pinned to the release, and the preflight passed; +# 1 installed, but the preflight did not pass, or the installed command +# does not stay on the release (one older than v1.0.1 cannot record +# it): the lines above say which; +# 2 refused before anything was installed: a version that is not a ref, no +# release found, a fetch that failed, or a file that is not the command. +# git-filter-repo never changes it: only adopting needs it, and +# `adopt-project.py` refuses with its own install command where it is missing. +# +# EVERYTHING RUNS FROM `main`, CALLED ON THE LAST LINE. Under `curl … | bash` +# bash reads this script from the pipe as it goes, so a download cut off +# part-way would run whatever had arrived. Inside a function nothing runs +# until the last line has arrived, and by then the pipe holds nothing a child +# could read in its place. +# +# BASH 3.2, which is /bin/bash on every Mac: no `${x,,}`, no `declare -A`, +# no `mapfile`; an empty array never meets `set -u`. + +set -euo pipefail + +REPO="${OPENREPOSHAPE_REPO:-opensoft/openRepoShape}" + +#: The version CI installs (`.github/workflows/tests.yml`), so what this +#: installs is what the adoption suites ran against. +FILTER_REPO_PIN="git-filter-repo==2.47.0" + +WORKDIR="" + +#: What `latest_from_redirect` and `latest_from_gh` found, and when they did +#: not, why. Globals rather than output, because a function run inside +#: `$(…)` is a subshell and could say only one of the two. +FOUND="" +FAULT="" + +say() { printf '%s\n' "$*"; } +die() { printf '\nREFUSED: %s\n' "$1" >&2; exit "${2:-2}"; } + +usage() { + cat <<'USAGE' +install.sh: install openRepoShape, pinned to a release. + + curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash + curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash -s -- v1.0.1 + bash install.sh [] + +With no it installs the newest release. It fetches that release's +`openRepoShape`, installs it into ~/.local/bin pinned to the release, installs +git-filter-repo with pipx when pipx is there and git-filter-repo is not (or +prints the command that installs it), and ends with `openRepoShape +--preflight`. It never runs sudo. Run the same line again to upgrade. + +Exit 0: installed, pinned, and the preflight passed. 1: installed, but the +preflight did not pass, or the command does not stay on the release. 2: +refused, and nothing was installed. + + $OPENREPOSHAPE_REPO owner/name to install from (default opensoft/openRepoShape) + $OPENREPOSHAPE_BIN_DIR where the command goes (default ~/.local/bin) +USAGE +} + +# THE REPOSITORY, AS AN OWNER/NAME, because every URL below is built from it. +# The launcher also takes a URL or a path in this variable, for the clone +# setup.sh makes; there is no release at either, so this refuses them by +# name. A control character is refused before anything is echoed. +checked_repo() { + case "$REPO" in + *[![:print:]]*) die "\$OPENREPOSHAPE_REPO carries a control character; it must be an owner/name on github.com, such as opensoft/openRepoShape." ;; + esac + case "$REPO" in + [A-Za-z0-9]*/[A-Za-z0-9._-]*) ;; + *) die "\$OPENREPOSHAPE_REPO is '$REPO', which is not an owner/name on github.com, such as opensoft/openRepoShape: this installer builds github.com URLs from it." ;; + esac + case "$REPO" in + */*/* | *[!A-Za-z0-9._/-]* | */. | */..) die "\$OPENREPOSHAPE_REPO is '$REPO', which is not an owner/name on github.com, such as opensoft/openRepoShape: this installer builds github.com URLs from it." ;; + esac +} + +# THE LAUNCHER'S `checked_ref`, TRANSCRIBED: the version reaches a URL here +# and `git checkout` once the command runs, so it is held to the same three +# checks in the same order (length, then a control character, then the +# shape, which is the only one that echoes it), plus the empty value. Not +# shared, because this file arrives alone down a pipe. +checked_version() { + local ref="$1" what="$2" + [ -n "$ref" ] || die "$what is empty. Name a release, such as v1.0.1, or none for the newest." + if [ "${#ref}" -gt 255 ]; then + die "$what is ${#ref} characters long; a release tag, branch or commit is at most 255 characters." + fi + case "$ref" in + *[![:print:]]*) die "$what carries a control character, which is not a release tag, branch or commit." ;; + esac + case "$ref" in + [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) die "$what is '$ref', which is not a release tag, branch or commit: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'." ;; + esac +} + +# THE NEWEST RELEASE, FROM THE REDIRECT. `releases/latest` answers 302 with +# `location: …/releases/tag/`; a repository with no release yet answers +# with `…/releases`, which names no tag and is no answer. The header name is +# matched in any case (HTTP/2 sends it in lower case, HTTP/1.1 need not), +# without `${x,,}`, which bash 3.2 does not have. +latest_from_redirect() { + local url="$1" headers line value status=0 + FOUND="" + command -v curl >/dev/null 2>&1 || { + FAULT="curl is not installed" + return 1 + } + headers="$(curl -fsSI "$url" 2>/dev/null)" || status=$? + if [ "$status" -ne 0 ]; then + FAULT="curl failed, exit $status" + return 1 + fi + while IFS= read -r line; do + line="${line%$'\r'}" + case "$line" in + [Ll][Oo][Cc][Aa][Tt][Ii][Oo][Nn]:*) + value="${line#*:}" + while :; do + case "$value" in + ' '* | $'\t'*) value="${value#?}" ;; + *) break ;; + esac + done + case "$value" in + */releases/tag/?*) + FOUND="${value##*/releases/tag/}" + return 0 + ;; + esac + case "$value" in + *[![:print:]]*) FAULT="it redirects to a location that names no release" ;; + *) FAULT="it redirects to $value, which names no release" ;; + esac + return 1 + ;; + esac + done <<<"$headers" + FAULT="it answered with no location" + return 1 +} + +# THE FALLBACK, AND ONLY WHERE `gh` IS INSTALLED: `gh api` needs a login even +# for a public repository, which is why it is not the first way. +latest_from_gh() { + local status=0 + FOUND="" + command -v gh >/dev/null 2>&1 || { + FAULT="gh is not installed" + return 1 + } + FOUND="$(gh api "repos/$REPO/releases/latest" --jq .tag_name 2>/dev/null)" || status=$? + if [ "$status" -ne 0 ] || [ -z "$FOUND" ]; then + FAULT="gh failed, exit $status" + return 1 + fi +} + +# THE COMMAND, AND NOT A PAGE THAT SAYS 200. A captive portal, a proxy's +# error page or a renamed file is refused here, before anything runs it: +# the launcher's first line, and the `INSTALLABLES=(` line that +# `tests/test_repo_hygiene.py` holds it to carrying. +looks_like_the_launcher() { + local line first=1 + while IFS= read -r line || [ -n "$line" ]; do + if [ "$first" -eq 1 ]; then + [ "$line" = "#!/usr/bin/env bash" ] || return 1 + first=0 + fi + case "$line" in "INSTALLABLES=("*) return 0 ;; esac + done <"$1" + return 1 +} + +# THE ONE COMMAND THAT INSTALLS IT HERE, for a machine with no pipx. POSIX +# shell, no sudo: Homebrew's where `brew` is, and otherwise pip, under +# whichever Python name this machine has, at the version CI runs. +filter_repo_command() { + local candidate + if command -v brew >/dev/null 2>&1; then + printf '%s' "brew install git-filter-repo" + return 0 + fi + for candidate in python3 python; do + if command -v "$candidate" >/dev/null 2>&1; then + printf '%s' "$candidate -m pip install --user $FILTER_REPO_PIN" + return 0 + fi + done + printf '%s' "python3 -m pip install --user $FILTER_REPO_PIN" +} + +filter_repo_step() { + local found status=0 + if found="$(command -v git-filter-repo 2>/dev/null)" && [ -n "$found" ]; then + say "git-filter-repo: already installed ($found)" + return 0 + fi + if command -v git >/dev/null 2>&1 && + git filter-repo --version >/dev/null 2>&1; then + say "git-filter-repo: already installed (git filter-repo --version answers)" + return 0 + fi + if command -v pipx >/dev/null 2>&1; then + say "git-filter-repo: not installed; installing it with pipx, because adopt-project.py needs it: pipx install $FILTER_REPO_PIN" + pipx install "$FILTER_REPO_PIN" || status=$? + if [ "$status" -ne 0 ]; then + say "git-filter-repo: pipx could not install it (exit $status). Before adopting a repository, install it: pipx install $FILTER_REPO_PIN" + elif command -v git-filter-repo >/dev/null 2>&1; then + say "git-filter-repo: installed with pipx" + else + say "git-filter-repo: installed with pipx, into a directory that is not on PATH yet: run \`pipx ensurepath\` and open a new shell" + fi + return 0 + fi + say "git-filter-repo: not installed, and there is no pipx to install it with. Only adopt-project.py needs it; before adopting a repository, run:" + say " $(filter_repo_command)" +} + +main() { + local version="" asked launcher_url bin_dir target shown status=0 pinned=1 + + # NOTHING A CHILD READS IS THIS SCRIPT. Piped, stdin is the pipe the + # script came down; a terminal stays a terminal, which is where the + # preflight asks before it offers anything. + [ -t 0 ] || exec /dev/null 2>&1 || + die "curl is not installed, and this installer fetches with it. Install curl, or install by hand with: gh api repos/$REPO/contents/openRepoShape -H 'Accept: application/vnd.github.raw' | bash -s -- --install" + + # 1. THE RELEASE. + if [ -n "$version" ]; then + checked_version "$version" "the version" + say "release: $version, as named" + else + asked="https://github.com/$REPO/releases/latest" + if latest_from_redirect "$asked"; then + version="$FOUND" + else + local redirect_fault="$FAULT" + latest_from_gh || die "could not find the newest release of $REPO, so nothing was installed. Both ways failed: + curl -fsSI $asked ($redirect_fault) + gh api repos/$REPO/releases/latest ($FAULT) + Name a release instead, as bash install.sh v1.0.1 or as: + curl -fsSL https://github.com/$REPO/releases/latest/download/install.sh | bash -s -- v1.0.1" + version="$FOUND" + asked="gh api repos/$REPO/releases/latest" + fi + checked_version "$version" "the newest release's tag" + say "release: $version, the newest release of $REPO ($asked)" + fi + + # 2. THAT RELEASE'S COMMAND, FETCHED, CHECKED AND INSTALLED BY ITSELF. + WORKDIR="$(mktemp -d "${TMPDIR:-/tmp}/openRepoShape-install.XXXXXXXX")" || + die "could not make a temporary directory under ${TMPDIR:-/tmp}. Set \$TMPDIR to a directory you can write to." + trap 'rm -rf -- "$WORKDIR"' EXIT + launcher_url="https://raw.githubusercontent.com/$REPO/$version/openRepoShape" + curl -fsSL "$launcher_url" -o "$WORKDIR/openRepoShape" || status=$? + [ "$status" -eq 0 ] && [ -s "$WORKDIR/openRepoShape" ] || + die "could not fetch $launcher_url (curl exit $status), so nothing was installed. Is $version a release of $REPO? https://github.com/$REPO/releases" + looks_like_the_launcher "$WORKDIR/openRepoShape" || + die "$launcher_url is not the openRepoShape command (its first line is not #!/usr/bin/env bash, or it has no INSTALLABLES line), so nothing was installed." + say "fetch: $launcher_url" + + bin_dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" + target="$bin_dir/openRepoShape" + say "install: OPENREPOSHAPE_REF=$version openRepoShape --install" + OPENREPOSHAPE_REF="$version" "${BASH:-bash}" "$WORKDIR/openRepoShape" --install || + die "the $version openRepoShape's --install refused (exit $?); its own refusal above says why." + + # THE PIN, CHECKED BY ASKING THE INSTALLED COMMAND, with the variable + # unset: a release older than v1.0.1 installs without recording a ref and + # would follow main, which is the drift this installer exists to end. + shown="$(unset OPENREPOSHAPE_REF; "${BASH:-bash}" "$target" --version 2>/dev/null)" || shown="" + case "$shown" in + *"@ $version)") ;; + *) + pinned=0 + say "install: $target does not stay on $version: it says '${shown:-nothing}' to --version. A release older than v1.0.1 cannot record the ref it was installed at; export OPENREPOSHAPE_REF=$version to keep it there." + ;; + esac + if [ -n "${OPENREPOSHAPE_REF:-}" ] && [ "$OPENREPOSHAPE_REF" != "$version" ]; then + say "install: \$OPENREPOSHAPE_REF is set in this shell, and while it is, every run uses it instead of $version. Unset it to run $version." + fi + + # 3. GIT-FILTER-REPO. + filter_repo_step + + # 4. THE PREFLIGHT, LAST, THROUGH THE INSTALLED COMMAND AND ITS RECORD. + say "preflight: $target --preflight" + status=0 + ( + unset OPENREPOSHAPE_REF + "${BASH:-bash}" "$target" --preflight + ) || status=$? + + if [ "$status" -ne 0 ]; then + say "done: openRepoShape $version is installed at $target, but the preflight did not pass (exit $status); it names what is missing above." + exit 1 + fi + if [ "$pinned" -eq 0 ]; then + say "done: openRepoShape $version is installed at $target and this machine is ready, but the command does not stay on $version (see above)." + exit 1 + fi + say "done: openRepoShape $version is installed at $target, and this machine is ready. Run the same line again to upgrade to the newest release." + exit 0 +} + +# `${1+"$@"}`, setup.sh's own spelling: no arguments is the one-line run, and +# that has to survive `set -u` under every bash this has to run on. +main ${1+"$@"} diff --git a/tests/test_install_sh.py b/tests/test_install_sh.py new file mode 100644 index 00000000..7ced9afc --- /dev/null +++ b/tests/test_install_sh.py @@ -0,0 +1,596 @@ +# SPDX-License-Identifier: Apache-2.0 +"""`install.sh`: openRepoShape in one line, pinned to a release (#564). + +OFFLINE, ON A `$PATH` THE TEST OWNS ENTIRELY. Each run's `$PATH` holds the +real programs the installer and the command it installs need (`bash`, +`mktemp`, `cp` and the rest of `INSTALL_NEEDS`), and FAKES for everything +that would reach the network or install something: `curl`, `gh`, `pipx`, +and, where a test asks for them, `git-filter-repo`, `git` and `brew`. +Restricted rather than prepended, because the machine running this may have +git-filter-repo and pipx installed for real, and which of the three +git-filter-repo branches runs must be the test's choice, not the machine's. + +EVERY FAKE IS ONE PROGRAM. A two-line shell launcher per name hands its +arguments to `FAKE` below, which appends the call to `calls.log` and answers +from `net.json`: the redirect github.com answers `releases/latest` with, the +files raw.githubusercontent.com serves, the tag `gh api` reports, and where +`pipx` puts what it installs. The `setup.sh` it serves is another launcher +into the same program, so the preflight the installer ends with is LOGGED, +with the arguments the installed command handed it: `--shape-ref ` +is where the pin shows. + +THE COMMAND SERVED IS THIS CHECKOUT'S `openRepoShape`, so the install the +installer runs is the real one. Every run sets `GH_TOKEN=bogus` and an empty +`GH_CONFIG_DIR`, and its `TMPDIR` is a directory each test checks is empty +again afterwards. + +WINDOWS_SKIP, for the reason `tests/test_openreposhape_command.py` gives: the +installer and the command are bash, run where `bash` is a bash, and on +Windows the way in is `setup-project.py`. +""" + +from __future__ import annotations + +import json +import re +import shlex +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +from conftest import REPO, WINDOWS_SKIP +from test_openreposhape_command import COMMAND, INSTALL_NEEDS, command_env + +INSTALLER = REPO / "install.sh" + +#: Bash by ABSOLUTE path: `subprocess` resolves a bare argv[0] against the +#: child's `$PATH`, and the one each run builds holds only what it chose. +BASH = shutil.which("bash") + +pytestmark = [pytest.mark.skipif(BASH is None, reason="install.sh is bash"), + WINDOWS_SKIP] + +SLUG = "opensoft/openRepoShape" +RELEASE = "v9.9.9" +FILTER_REPO = "git-filter-repo==2.47.0" + + +def latest(slug: str = SLUG) -> str: + return f"https://github.com/{slug}/releases/latest" + + +def tag_page(tag: str, slug: str = SLUG) -> str: + return f"https://github.com/{slug}/releases/tag/{tag}" + + +def raw(name: str, ref: str = RELEASE, slug: str = SLUG) -> str: + return f"https://raw.githubusercontent.com/{slug}/{ref}/{name}" + + +#: THE ONE PROGRAM BEHIND EVERY FAKE. `curl -fsSI` answers a configured +#: redirect with the headers github.com sends, CRLF and all, and `curl +#: -fsSL` a configured file, to `-o` or to stdout; anything else is curl's +#: own exit 22. `gh` reports a release only when one is configured. `pipx` +#: places a `git-filter-repo` launcher where it is told to, or fails. Every +#: call is logged first, so what was NOT asked is as plain as what was. +FAKE = r''' +import json +import shlex +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +NET = json.loads((HERE / "net.json").read_text(encoding="utf-8")) +NAME, ARGS = sys.argv[1], sys.argv[2:] +with open(HERE / "calls.log", "a", encoding="utf-8") as log: + log.write(" ".join([NAME] + ARGS) + "\n") + + +def refuse(message): + sys.stderr.write(message + "\n") + return 22 if NAME == "curl" else 1 + + +def launcher(directory, name): + path = Path(directory) / name + path.write_text("#!/bin/sh\nexec %s %s %s \"$@\"\n" % ( + shlex.quote(sys.executable), shlex.quote(str(Path(__file__))), name), + encoding="utf-8") + path.chmod(0o755) + + +def curl(): + url = next((arg for arg in ARGS if arg.startswith("https://")), "") + if "-fsSI" in ARGS: + location = NET["redirects"].get(url) + if location is None: + return refuse("curl: (22) The requested URL returned error: 404") + sys.stdout.write("HTTP/2 302 \r\nlocation: %s\r\n" + "content-length: 0\r\n\r\n" % location) + return 0 + served = NET["files"].get(url) + if served is None: + return refuse("curl: (22) The requested URL returned error: 404") + data = Path(served).read_bytes() + if "-o" in ARGS: + Path(ARGS[ARGS.index("-o") + 1]).write_bytes(data) + else: + sys.stdout.buffer.write(data) + return 0 + + +def gh(): + if ARGS[:1] == ["api"] and ARGS[1].endswith("/releases/latest") \ + and NET.get("gh_latest"): + print(NET["gh_latest"]) + return 0 + return refuse("fake gh: no answer for: " + " ".join(ARGS)) + + +def pipx(): + if NET.get("pipx_installs_into") is None: + return refuse("fake pipx: could not install " + " ".join(ARGS)) + launcher(NET["pipx_installs_into"], "git-filter-repo") + return 0 + + +def git(): + if ARGS == ["filter-repo", "--version"] and NET.get("git_filter_repo"): + print("a40bce548d2c") + return 0 + return refuse("fake git: no answer for: " + " ".join(ARGS)) + + +def setup_sh(): + print("fake preflight: " + " ".join(ARGS)) + return NET.get("preflight_exit", 0) + + +sys.exit({"curl": curl, "gh": gh, "pipx": pipx, "git": git, + "setup.sh": setup_sh}.get(NAME, lambda: 0)()) +''' + +#: A RELEASE OLDER THAN v1.0.1, as far as the installer can tell: the +#: launcher's first line and its `INSTALLABLES` line, an `--install` that +#: copies itself and records nothing, and a `--version` that therefore says +#: `main`. It is all the installer asks of a command. +OLD_LAUNCHER = """#!/usr/bin/env bash +INSTALLABLES=(openRepoShape) +case "$1" in +--install) mkdir -p "$OPENREPOSHAPE_BIN_DIR" && cp "$0" "$OPENREPOSHAPE_BIN_DIR/openRepoShape" ;; +--version) printf 'openRepoShape (%s @ %s)\\n' "$OPENREPOSHAPE_REPO" "${OPENREPOSHAPE_REF:-main}" ;; +esac +""" + + +class Net: + """The fakes, what they answer, the `$PATH` that holds them, and a run. + + `fakes` names the programs on that `$PATH` that answer through `FAKE`; + `serve` adds a release's two files; `config` is written to `net.json` + at every `run`, so a test changes an answer between two runs by + changing it here. + """ + + def __init__(self, tmp_path: Path, fakes=("curl", "gh", "pipx")): + self.tmp_path = tmp_path + self.root = tmp_path / "net" + self.root.mkdir() + (self.root / "fake.py").write_text(FAKE, encoding="utf-8") + self.path = tmp_path / "path" + self.path.mkdir() + for name in (*INSTALL_NEEDS, "bash"): + real = shutil.which(name) + assert real, f"this machine has no {name}, which the install needs" + (self.path / name).symlink_to(real) + for name in fakes: + self.fake(name) + self.temporary = tmp_path / "tmp" + self.temporary.mkdir() + self.bin = tmp_path / ".local" / "bin" + self.config = {"redirects": {latest(): tag_page(RELEASE)}, + "files": {}, "pipx_installs_into": str(self.path)} + self.serve(RELEASE) + + def fake(self, name: str, directory: Path | None = None) -> Path: + """A launcher for `name` into `FAKE`, on this `$PATH` by default.""" + path = (directory or self.path) / name + path.write_text( + f"#!/bin/sh\nexec {shlex.quote(sys.executable)} " + f"{shlex.quote(str(self.root / 'fake.py'))} {name} \"$@\"\n", + encoding="utf-8") + path.chmod(0o755) + return path + + def serve(self, ref: str, slug: str = SLUG, + launcher: Path = COMMAND) -> None: + """`openRepoShape` and `setup.sh` at `ref`, as raw URLs answer.""" + served = self.root / "served" / ref + served.mkdir(parents=True, exist_ok=True) + self.config["files"][raw("openRepoShape", ref, slug)] = str(launcher) + self.config["files"][raw("setup.sh", ref, slug)] = str( + self.fake("setup.sh", served)) + + def run(self, *args: str, piped: bool = False, + env: dict | None = None) -> subprocess.CompletedProcess: + """`bash install.sh `, or `… | bash -s -- ` when piped.""" + (self.root / "net.json").write_text(json.dumps(self.config), + encoding="utf-8") + gh_config = self.tmp_path / "gh-config" + gh_config.mkdir(exist_ok=True) + argv = [BASH, "-s", "--", *args] if piped \ + else [BASH, str(INSTALLER), *args] + return subprocess.run( + argv, capture_output=True, text=True, check=False, + input=INSTALLER.read_text(encoding="utf-8") if piped else "", + cwd=str(self.tmp_path), start_new_session=True, + env=command_env(home=self.tmp_path, setup_sh=None, env={ + "PATH": str(self.path), "GH_TOKEN": "bogus", + "GH_CONFIG_DIR": str(gh_config), + "TMPDIR": str(self.temporary), **(env or {})})) + + def calls(self) -> list: + log = self.root / "calls.log" + return log.read_text(encoding="utf-8").splitlines() \ + if log.exists() else [] + + def installed(self, result, ref: str = RELEASE) -> None: + """The command is placed, byte for byte, with `ref` recorded beside + it, and the run left nothing in `$TMPDIR`.""" + target = self.bin / "openRepoShape" + assert target.read_bytes() == COMMAND.read_bytes(), ( + result.stdout + result.stderr) + assert (self.bin / "openRepoShape.ref").read_text( + encoding="utf-8") == f"{ref}\n" + assert list(self.temporary.iterdir()) == [] + + def refused(self, result) -> None: + """Exit 2, nothing placed, no preflight run, nothing left behind.""" + assert result.returncode == 2, result.stdout + result.stderr + assert not self.bin.exists(), "a refused install placed something" + assert not any(call.startswith("setup.sh") for call in self.calls()) + assert list(self.temporary.iterdir()) == [] + + +@pytest.fixture +def net(tmp_path): + return Net(tmp_path) + + +def preflight_call(ref: str = RELEASE) -> str: + """How the preflight reaches setup.sh through a command pinned to `ref`.""" + return f"setup.sh --shape-ref {ref} --preflight" + + +# --- the one line ------------------------------------------------------------ + +def test_the_one_line_installs_the_newest_release_pinned_to_it(net): + """`curl … | bash`: the newest release, from the redirect alone. + + One line per step, in order, and the calls behind them in the same + order: the redirect, the release's command, then the preflight, LAST, + through the installed command, whose `setup.sh` is fetched at the + release and handed `--shape-ref` for it. `gh` is never asked for a + release when the redirect named one. + """ + result = net.run(piped=True) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + target = net.bin / "openRepoShape" + steps = [ + f"release: {RELEASE}, the newest release of {SLUG} ({latest()})", + f"fetch: {raw('openRepoShape')}", + f"install: OPENREPOSHAPE_REF={RELEASE} openRepoShape --install", + f"openRepoShape: pinned to {RELEASE}, recorded in {target}.ref", + "git-filter-repo: ", + f"preflight: {target} --preflight", + f"done: openRepoShape {RELEASE} is installed at {target}, and this " + "machine is ready. Run the same line again to upgrade to the newest " + "release.", + ] + lines = result.stdout.splitlines() + at = [next(i for i, line in enumerate(lines) if line.startswith(step)) + for step in steps] + assert at == sorted(at), result.stdout + calls = net.calls() + assert calls[0] == f"curl -fsSI {latest()}" + assert calls[1].startswith(f"curl -fsSL {raw('openRepoShape')} -o ") + assert f"curl -fsSL {raw('setup.sh')}" in calls + assert calls[-1] == preflight_call(), calls + assert not any("releases/latest" in call for call in calls[1:]), calls + + +@pytest.mark.parametrize("piped", [False, True], ids=["file", "piped"]) +def test_a_named_release_is_installed_and_no_newest_is_asked_for(net, piped): + """`bash install.sh v9.9.8` and `… | bash -s -- v9.9.8`: that release, + and no question about which one is newest.""" + net.serve("v9.9.8") + result = net.run("v9.9.8", piped=piped) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result, "v9.9.8") + assert "release: v9.9.8, as named" in result.stdout.splitlines() + assert not any("-fsSI" in call for call in net.calls()), net.calls() + assert net.calls()[-1] == preflight_call("v9.9.8") + + +def test_gh_answers_when_the_redirect_names_no_release(net): + """A redirect to `/releases` names no tag (what github.com answers before + a repository's first release); `gh api` is asked next, where installed.""" + net.config["redirects"][latest()] = f"https://github.com/{SLUG}/releases" + net.config["gh_latest"] = RELEASE + result = net.run() + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"release: {RELEASE}, the newest release of {SLUG} (gh api " \ + f"repos/{SLUG}/releases/latest)" in result.stdout + + +@pytest.mark.parametrize("location, fakes, redirect_fault, gh_fault", [ + pytest.param(f"https://github.com/{SLUG}/releases", ("curl",), + f"it redirects to https://github.com/{SLUG}/releases, which " + "names no release", "gh is not installed", id="no-release"), + pytest.param(None, ("curl", "gh"), "curl failed, exit 22", + "gh failed, exit 1", id="both-failed"), +]) +def test_no_release_found_names_each_way_that_failed( + tmp_path, location, fakes, redirect_fault, gh_fault): + """Each way names its URL or command and why it gave no answer, and the + way out is to name a release; nothing is fetched or installed.""" + net = Net(tmp_path, fakes) + if location is None: + del net.config["redirects"][latest()] + else: + net.config["redirects"][latest()] = location + result = net.run() + net.refused(result) + assert f"could not find the newest release of {SLUG}" in result.stderr + assert f"curl -fsSI {latest()} ({redirect_fault})" in result.stderr + assert f"gh api repos/{SLUG}/releases/latest ({gh_fault})" \ + in result.stderr + assert "bash install.sh v1.0.1" in result.stderr + assert not any("-fsSL" in call for call in net.calls()) + + +@pytest.mark.parametrize("args, said", [ + pytest.param(("v1..v2",), "the version is 'v1..v2'", id="a-range"), + pytest.param(("x.lock",), "the version is 'x.lock'", id="a-lock-file"), + pytest.param(("/etc",), "the version is '/etc'", id="a-path"), + pytest.param(("v1\nforged",), "the version carries a control character", + id="two-lines"), + pytest.param(("a" * 256,), "the version is 256 characters long", + id="over-long"), + pytest.param(("-x",), "unknown option '-x'", id="an-option"), + pytest.param(("-x\nforged",), "an option carries a control character", + id="an-option-of-two-lines"), + pytest.param(("v1.0.1", "v1.0.2"), "two versions given", id="two"), +]) +def test_a_version_that_is_not_a_ref_is_refused_before_any_request( + net, args, said): + """Checked the way the launcher checks a ref, before any URL is built + from it; a control character and the over-long value never echoed.""" + result = net.run(*args) + net.refused(result) + assert said in result.stderr + assert net.calls() == [] + assert "forged" not in result.stderr and "a" * 256 not in result.stderr + + +def test_a_failed_fetch_names_its_url_and_installs_nothing(net): + del net.config["files"][raw("openRepoShape")] + result = net.run() + net.refused(result) + assert f"could not fetch {raw('openRepoShape')} (curl exit 22), so " \ + "nothing was installed" in result.stderr + + +def test_a_page_that_is_not_the_command_is_refused(net): + """A proxy's or a portal's page can arrive with a 200; it is refused + before anything runs it.""" + page = net.root / "page.html" + page.write_text("Sign in\n", + encoding="utf-8") + net.config["files"][raw("openRepoShape")] = str(page) + result = net.run() + net.refused(result) + assert f"{raw('openRepoShape')} is not the openRepoShape command" \ + in result.stderr + + +# --- git-filter-repo --------------------------------------------------------- + +def present_on_path(net): + net.fake("git-filter-repo") + return f"git-filter-repo: already installed ({net.path}/git-filter-repo)" + + +def answering_git(net): + net.fake("git") + net.config["git_filter_repo"] = True + return "git-filter-repo: already installed (git filter-repo --version " \ + "answers)" + + +def pipx_onto_path(net): + return "git-filter-repo: installed with pipx" + + +def pipx_off_path(net): + elsewhere = net.tmp_path / "pipx-bin" + elsewhere.mkdir() + net.config["pipx_installs_into"] = str(elsewhere) + return "git-filter-repo: installed with pipx, into a directory that is " \ + "not on PATH yet: run `pipx ensurepath` and open a new shell" + + +def pipx_failing(net): + net.config["pipx_installs_into"] = None + return f"git-filter-repo: pipx could not install it (exit 1). Before " \ + f"adopting a repository, install it: pipx install {FILTER_REPO}" + + +def no_pipx(net): + (net.path / "pipx").unlink() + return f" python3 -m pip install --user {FILTER_REPO}" + + +def no_pipx_but_brew(net): + no_pipx(net) + net.fake("brew") + return " brew install git-filter-repo" + + +@pytest.mark.parametrize("arrange", [ + present_on_path, answering_git, pipx_onto_path, pipx_off_path, + pipx_failing, no_pipx, no_pipx_but_brew]) +def test_git_filter_repo_is_said_installed_or_printed(net, arrange): + """The three branches, and the ways each can end. + + PRESENT, on `$PATH` or as `git filter-repo`: said, and pipx is never + run. NOT PRESENT AND PIPX IS THERE: said first, then `pipx install` at + CI's version, then what came of it. NEITHER: the one command for this + machine, printed and never run. None changes the exit status, and the + step sits between the install and the preflight. + """ + expected = arrange(net) + result = net.run() + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + lines = result.stdout.splitlines() + assert expected in lines, result.stdout + ran_pipx = [call for call in net.calls() if call.startswith("pipx")] + if arrange in (pipx_onto_path, pipx_off_path, pipx_failing): + assert ran_pipx == [f"pipx install {FILTER_REPO}"], net.calls() + assert "git-filter-repo: not installed; installing it with pipx, "\ + "because adopt-project.py needs it: pipx install " \ + f"{FILTER_REPO}" in lines + assert net.calls().index(ran_pipx[0]) < \ + net.calls().index(preflight_call()) + else: + assert ran_pipx == [], net.calls() + if arrange in (no_pipx, no_pipx_but_brew): + assert lines[lines.index(expected) - 1] == ( + "git-filter-repo: not installed, and there is no pipx to install " + "it with. Only adopt-project.py needs it; before adopting a " + "repository, run:") + assert lines.index(expected) < lines.index( + f"preflight: {net.bin}/openRepoShape --preflight") + + +# --- the preflight, the exit status, and running it again -------------------- + +def test_a_preflight_that_does_not_pass_exits_one_with_the_command_placed( + net): + net.config["preflight_exit"] = 1 + result = net.run() + assert result.returncode == 1, result.stdout + result.stderr + net.installed(result) + assert f"done: openRepoShape {RELEASE} is installed at " \ + f"{net.bin}/openRepoShape, but the preflight did not pass (exit 1); " \ + "it names what is missing above." in result.stdout + + +def test_a_variable_in_the_shell_is_named_and_the_preflight_runs_the_pin(net): + """`$OPENREPOSHAPE_REF` exported in the person's shell overrides the pin + on every run while it stays set, so the installer says so; the release + is still what is recorded, and the preflight still runs through it.""" + result = net.run(env={"OPENREPOSHAPE_REF": "v0.0.1"}) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"install: $OPENREPOSHAPE_REF is set in this shell, and while it " \ + f"is, every run uses it instead of {RELEASE}. Unset it to run " \ + f"{RELEASE}." in result.stdout + assert net.calls()[-1] == preflight_call() + + +def test_the_same_line_again_changes_nothing_and_then_upgrades(net): + """Idempotent while the newest release is the same one; the next + release, once it is the newest, replaces the record and says which it + replaced.""" + assert net.run().returncode == 0 + again = net.run() + assert again.returncode == 0, again.stdout + again.stderr + assert f"openRepoShape: already installed at {net.bin}/openRepoShape " \ + "(unchanged)" in again.stdout + assert f"openRepoShape: already pinned to {RELEASE} in " \ + f"{net.bin}/openRepoShape.ref (unchanged)" in again.stdout + + net.config["redirects"][latest()] = tag_page("v10.0.0") + net.serve("v10.0.0") + upgrade = net.run() + assert upgrade.returncode == 0, upgrade.stdout + upgrade.stderr + net.installed(upgrade, "v10.0.0") + assert f"openRepoShape: pinned to v10.0.0 (was {RELEASE}), recorded in " \ + f"{net.bin}/openRepoShape.ref" in upgrade.stdout + + +def test_a_release_whose_command_records_no_ref_exits_one(net): + """A release before v1.0.1 installs a command that follows main. The + installer asks the installed command, says so, and exits 1.""" + old = net.root / "old-openRepoShape" + old.write_text(OLD_LAUNCHER, encoding="utf-8") + net.serve(RELEASE, launcher=old) + result = net.run(env={"OPENREPOSHAPE_BIN_DIR": str(net.bin), + "OPENREPOSHAPE_REPO": SLUG}) + assert result.returncode == 1, result.stdout + result.stderr + assert f"install: {net.bin}/openRepoShape does not stay on {RELEASE}: " \ + f"it says 'openRepoShape ({SLUG} @ main)' to --version." \ + in result.stdout + assert f"export OPENREPOSHAPE_REF={RELEASE}" in result.stdout + assert result.stdout.splitlines()[-1].endswith( + f"but the command does not stay on {RELEASE} (see above).") + + +# --- the repository it installs from ---------------------------------------- + +def test_openreposhape_repo_names_where_everything_comes_from(net): + """A fork named once is where the release, the command and the + preflight's setup.sh all come from, and the installed command says so.""" + fork = "acme/openRepoShape" + net.config["redirects"] = {latest(fork): tag_page(RELEASE, fork)} + net.config["files"] = {} + net.serve(RELEASE, fork) + result = net.run(env={"OPENREPOSHAPE_REPO": fork}) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert all(SLUG not in call for call in net.calls()), net.calls() + assert f"curl -fsSL {raw('setup.sh', RELEASE, fork)}" in net.calls() + + +@pytest.mark.parametrize("value", [ + "https://github.com/acme/openRepoShape.git", "acme", "acme/x/y", + "/srv/openRepoShape"]) +def test_a_repository_that_is_not_an_owner_and_name_is_refused(net, value): + result = net.run(env={"OPENREPOSHAPE_REPO": value}) + net.refused(result) + assert f"$OPENREPOSHAPE_REPO is '{value}', which is not an owner/name " \ + "on github.com" in result.stderr + assert net.calls() == [] + + +# --- what the file says about itself ------------------------------------------ + +def test_help_prints_the_one_line_and_exits_zero(tmp_path): + result = subprocess.run( + [BASH, str(INSTALLER), "--help"], capture_output=True, text=True, + check=False, input="", + env=command_env(home=tmp_path, setup_sh=None, + env={"GH_TOKEN": "bogus", + "GH_CONFIG_DIR": str(tmp_path)})) + assert result.returncode == 0, result.stderr + assert "curl -fsSL https://github.com/opensoft/openRepoShape/releases/" \ + "latest/download/install.sh | bash" in result.stdout + + +def test_it_installs_the_git_filter_repo_ci_runs(): + """What the installer puts on a machine is what the adoption suites ran + against: the pin it names is every pin `.github/workflows/tests.yml` + installs.""" + [pinned] = re.findall(r'^FILTER_REPO_PIN="(git-filter-repo==[^"]+)"$', + INSTALLER.read_text(encoding="utf-8"), re.M) + ci = re.findall(r"git-filter-repo==[0-9.]+", ( + REPO / ".github" / "workflows" / "tests.yml").read_text( + encoding="utf-8")) + assert ci and set(ci) == {pinned}, (pinned, ci) diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index 3761eb76..ff065cf7 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -66,12 +66,13 @@ "shape_advisory", "conftest", "bootstrap"} #: The bash scripts a person runs BEFORE they have a checkout, or with no -#: checkout in sight at all: the front door itself and the command that -#: fetches it. `park` and `resume` were here from #82 until the carve (#92) -#: and are opensoft/openRepoTools' now, held to these same rules by that -#: repository's own copy of this test. Every one is a shipped executable and -#: is held to the same shebang, mode bit and `set -euo pipefail` rule. -SHIPPED_BASH = ["setup.sh", "openRepoShape"] +#: checkout in sight at all: the front door itself, the command that +#: fetches it, and the one-line installer that places that command pinned to +#: a release (#564). `park` and `resume` were here from #82 until the carve +#: (#92) and are opensoft/openRepoTools' now, held to these same rules by +#: that repository's own copy of this test. Every one is a shipped executable +#: and is held to the same shebang, mode bit and `set -euo pipefail` rule. +SHIPPED_BASH = ["setup.sh", "openRepoShape", "install.sh"] #: The entry point a person runs BEFORE they have a checkout on a machine with #: no bash: the same front door, on an interpreter alone. Held to the same @@ -1183,8 +1184,25 @@ def test_readme_is_short_enough_to_be_read(): # shim's answer there changed from a clone of the upstream to a few # lines of its own, and a README that still promised the flag list # would send a reader to a command that no longer prints it. - assert len(lines) <= 1422, ( - f"README.md is {len(lines)} lines; the cap is 1422") + # 2026-10-14: 1422 -> 1450 — the one-line install, v1.0.1 (#564). Brett + # Heap: "we need it to be more simple to install ... now and after, we + # need a simple install process". Step 3 of the quick start now LEADS + # with `curl -fsSL …/releases/latest/download/install.sh | bash`, and + # the worked example's "Install the command" with the same line: + # eleven lines for the two blocks and their lead-ins. Nine say what + # that line does that the old two did not, because each is a thing a + # reader would otherwise meet unannounced on their own machine: it + # reads the newest release from a redirect with no `gh`, it installs + # git-filter-repo with pipx or prints the command, it ends in the + # preflight, it never runs `sudo`, how to name a release, and its three + # exit statuses. Five say what PINNED means — the record beside the + # command, what reads it, and the variable that overrides it — because + # a command that quietly stops following main is a surprise to anyone + # who expected it to. Two re-introduce the existing lines as the + # by-hand way, at main, so neither is lost, and one is the Layout row + # for `install.sh`. + assert len(lines) <= 1450, ( + f"README.md is {len(lines)} lines; the cap is 1450") #: #476: what each document that carries the first rule says an assistant does @@ -1247,7 +1265,7 @@ def test_shipped_bash_is_executable_and_fails_loudly(name): @WINDOWS_SKIP @pytest.mark.parametrize("name", SHIPPED_BASH) def test_shipped_bash_parses_under_bash(name): - """`bash -n` on the two shipped bash scripts, everywhere there is a bash. + """`bash -n` on the shipped bash scripts, everywhere there is a bash. SKIPPED ON WINDOWS, AND `shutil.which` IS NOT ENOUGH TO SEE WHY. The `bash` a stock Windows install puts on PATH is @@ -1341,6 +1359,27 @@ def test_the_openrepotools_install_line_is_the_same_everywhere(name): f" {OPENREPOTOOLS_INSTALL}") +#: THE ONE LINE THAT INSTALLS THIS STANDARD (#564), and every file that +#: carries it: the installer it fetches, the command it installs (in its +#: header), the README and its designed reading, and the release notes that +#: introduced it. Pasted by people who cannot check it, like the two lines +#: above, so the copies are held byte for byte. `releases/latest/download/` +#: is what makes it the NEWEST release's `install.sh` every time. +ONE_LINE_INSTALL = ( + "curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/" + "download/install.sh | bash") +ONE_LINE_INSTALL_IN = ["install.sh", "openRepoShape", "README.md", + "docs/handbook.html", "docs/release-notes/v1.0.1.md"] + + +@pytest.mark.parametrize("name", ONE_LINE_INSTALL_IN) +def test_the_one_line_install_is_the_same_everywhere(name): + text = (REPO / name).read_text(encoding="utf-8") + assert ONE_LINE_INSTALL in text, ( + f"{name} must carry the one-line install byte for byte:\n" + f" {ONE_LINE_INSTALL}") + + def test_nothing_here_installs_park_or_resume(): """The carve, asserted rather than remembered (#92). From 319b005da6d315451a8ef4e31e6f28cfed699186 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 11:21:46 +0000 Subject: [PATCH 03/25] openRepoShape and install.sh, the fix round on #577: `--install` records a ref only with `OPENREPOSHAPE_RECORD_REF=1`, which install.sh sets, so a script that exports `$OPENREPOSHAPE_REF` around it gets v1.0.0's install; every curl that runs is HTTPS only, redirects included, at TLS 1.2 or newer; a link to the command reads its record; `--help` answers past a broken record; and install.sh says which failure it met (#564) The review on #577 (RV577, FIX-ROUND) and SonarCloud shell:S6506. openRepoShape: - The ref is recorded only when asked, or when it came from the command's own record (a re-install of itself stays pinned). workBenches exports `OPENREPOSHAPE_REF=pinned-by-workBenches-no-fetch` around `--install`; that no longer becomes a record every later run would fail on. Any other install records nothing and removes a record already there. - The raw-URL fallback curl carries `--proto '=https' --proto-redir '=https' --tlsv1.2`, written on the line. - The record is found beside the FILE, symlinks followed with a portable `readlink` loop (no `-f`, at most 40 links). - A broken record is kept as a fault and refused after the arguments are read, so `--help` still answers; `--version` and every run refuse. A record that is a directory is refused in the usual form, by a run and by an install, before anything is placed. install.sh: - Both curls HTTPS only, redirects included, TLS 1.2 or newer. - Sets `OPENREPOSHAPE_RECORD_REF=1` for the install. - A fetch the server refused (curl exit 22) asks whether the release exists; no answer at all points at the network instead. A preflight that exits 1 names what is missing; any other status says it did not run to the end. - With no pipx, `apt-get` machines are told `sudo apt-get install -y pipx && pipx install git-filter-repo==2.47.0` (printed, never run), because Debian 12 and Ubuntu 23.04+ refuse `pip install --user`. - A command that did not record the tag gets its preflight run with the tag named, not main's setup.sh; the shell-profile notes say so. - The last line is `{ main ${1+"$@"}; }`: a download cut inside it runs nothing. install.sh itself never runs sudo; the wording now says the preflight's offers, some of them sudo, each wait for a typed yes. Tests: HTTPS_ONLY in the call asserts and a guard on every curl that runs in both scripts; the opt-in, links, a directory record and `--help` past a bad record; an upper-case and a relative `location`; a non-default OPENREPOSHAPE_BIN_DIR; the apt-get branch; the network failure; the preflight's two failures; three truncated downloads; S9073 composite asserts split. docs/cli.md regenerated for the `--help` change. Co-Authored-By: Claude Opus 5.5 --- docs/cli.md | 19 +-- install.sh | 130 ++++++++++++------ openRepoShape | 198 ++++++++++++++++++++-------- tests/test_install_sh.py | 188 +++++++++++++++++++++----- tests/test_openreposhape_command.py | 80 ++++++++++- 5 files changed, 474 insertions(+), 141 deletions(-) diff --git a/docs/cli.md b/docs/cli.md index 7e81e1af..9f2a8f82 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -155,14 +155,17 @@ opensoft/openRepoTools', and its own installer places them: $OPENREPOSHAPE_ORG the organisation, when --org is not given $OPENREPOSHAPE_BIN_DIR where --install puts it (default ~/.local/bin) $OPENREPOSHAPE_SETUP_SH run this setup.sh on disk instead of fetching one - -`--install` RECORDS THE REF it runs at, in `openRepoShape.ref` beside the -command it places: $OPENREPOSHAPE_REF, which the one-line installer -(install.sh) sets to the release it installs. The installed command then -stays on that ref: it fetches setup.sh at it, setup.sh clones the standard -at it (`--shape-ref`), and `--doctor` checks its clone out at it. -$OPENREPOSHAPE_REF set for one run still wins, and `--version` prints the -ref in force. Installed with no ref, it records none and follows main. + $OPENREPOSHAPE_RECORD_REF 1: --install records $OPENREPOSHAPE_REF (below) + +`--install` RECORDS A REF ONLY WHEN ASKED: with $OPENREPOSHAPE_RECORD_REF=1 +it writes $OPENREPOSHAPE_REF to `openRepoShape.ref` beside the command it +places, which is what the one-line installer (install.sh) does with the +release tag it installs. The installed command then stays on that ref: it +fetches setup.sh at it, setup.sh clones the standard at it (`--shape-ref`), +and `--doctor` checks its clone out at it. $OPENREPOSHAPE_REF set for one run +still wins, and `--version` prints the ref in force. A command re-installing +itself keeps its own record; any other install records nothing, removes a +record already there, and follows main. `--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` over (default `.`) and prints one row per check and ONE verdict: diff --git a/install.sh b/install.sh index 8713a67a..453507bb 100755 --- a/install.sh +++ b/install.sh @@ -18,33 +18,36 @@ # when the redirect names no release and `gh` is installed. # 2. THAT RELEASE'S `openRepoShape`, fetched into a temporary directory, # checked to be the command, and installed by its own `--install` with -# `$OPENREPOSHAPE_REF` set to the release. `--install` records the ref -# beside the command it places, and the command then stays on it. +# `$OPENREPOSHAPE_REF` set to the release tag and +# `$OPENREPOSHAPE_RECORD_REF=1`, which asks it to record that tag beside +# the command it places; the command then stays on it. # 3. GIT-FILTER-REPO, which `adopt-project.py` needs and nothing else does: # said when it is there; installed with `pipx` when pipx is there and it # is not; otherwise the one command that installs it, printed. # 4. THE INSTALLED COMMAND'S `--preflight`, LAST, and its verdict. # -# IT NEVER RUNS sudo AND INSTALLS NOTHING ELSE. The command and, through pipx, -# git-filter-repo are the whole of what it installs. The preflight it ends -# with offers what is missing behind a typed `yes`, at the person's own -# terminal, exactly as `openRepoShape --preflight` always has. +# install.sh ITSELF NEVER RUNS sudo, AND INSTALLS NOTHING BUT THESE TWO: the +# command and, through pipx, git-filter-repo. The preflight it ends with +# offers what is missing, each behind a typed `yes` at the person's own +# terminal, exactly as `openRepoShape --preflight` always has, and some of +# those offers are `sudo` commands: that is the preflight's, asked first. # # EXIT STATUS: -# 0 installed, pinned to the release, and the preflight passed; +# 0 installed, pinned to the release tag, and the preflight passed; # 1 installed, but the preflight did not pass, or the installed command -# does not stay on the release (one older than v1.0.1 cannot record +# does not stay on the release tag (one older than v1.0.1 cannot record # it): the lines above say which; # 2 refused before anything was installed: a version that is not a ref, no # release found, a fetch that failed, or a file that is not the command. # git-filter-repo never changes it: only adopting needs it, and # `adopt-project.py` refuses with its own install command where it is missing. # -# EVERYTHING RUNS FROM `main`, CALLED ON THE LAST LINE. Under `curl … | bash` -# bash reads this script from the pipe as it goes, so a download cut off -# part-way would run whatever had arrived. Inside a function nothing runs -# until the last line has arrived, and by then the pipe holds nothing a child -# could read in its place. +# EVERYTHING RUNS FROM `main`, CALLED ON THE LAST LINE, INSIDE BRACES. Under +# `curl … | bash` bash reads this script from the pipe as it goes, so a +# download cut off part-way would run whatever had arrived. Inside a function +# nothing runs until the last line has arrived; the braces make that line one +# command that bash parses whole, so a download cut inside it runs nothing +# either; and by then the pipe holds nothing a child could read in its place. # # BASH 3.2, which is /bin/bash on every Mac: no `${x,,}`, no `declare -A`, # no `mapfile`; an empty array never meets `set -u`. @@ -77,13 +80,15 @@ install.sh: install openRepoShape, pinned to a release. bash install.sh [] With no it installs the newest release. It fetches that release's -`openRepoShape`, installs it into ~/.local/bin pinned to the release, installs -git-filter-repo with pipx when pipx is there and git-filter-repo is not (or -prints the command that installs it), and ends with `openRepoShape ---preflight`. It never runs sudo. Run the same line again to upgrade. +`openRepoShape`, installs it into ~/.local/bin pinned to the release tag, +installs git-filter-repo with pipx when pipx is there and git-filter-repo is +not (or prints the command that installs it), and ends with `openRepoShape +--preflight`. install.sh itself never runs sudo; the preflight it ends with +offers what is missing, each behind a typed `yes` at your terminal. Run the +same line again to upgrade. Exit 0: installed, pinned, and the preflight passed. 1: installed, but the -preflight did not pass, or the command does not stay on the release. 2: +preflight did not pass, or the command does not stay on the release tag. 2: refused, and nothing was installed. $OPENREPOSHAPE_REPO owner/name to install from (default opensoft/openRepoShape) @@ -131,7 +136,13 @@ checked_version() { # `location: …/releases/tag/`; a repository with no release yet answers # with `…/releases`, which names no tag and is no answer. The header name is # matched in any case (HTTP/2 sends it in lower case, HTTP/1.1 need not), -# without `${x,,}`, which bash 3.2 does not have. +# without `${x,,}`, which bash 3.2 does not have, and a relative location is +# read the same way as a whole URL. +# +# EVERY curl HERE IS HTTPS ONLY, REDIRECTS INCLUDED, AT TLS 1.2 OR NEWER +# (SonarCloud shell:S6506), with the flags written out on the curl line +# itself, never gathered into an array, so a reader and the analyser both +# see them there. `tests/test_install_sh.py` holds every curl line to it. latest_from_redirect() { local url="$1" headers line value status=0 FOUND="" @@ -139,7 +150,7 @@ latest_from_redirect() { FAULT="curl is not installed" return 1 } - headers="$(curl -fsSI "$url" 2>/dev/null)" || status=$? + headers="$(curl -fsSI --proto '=https' --proto-redir '=https' --tlsv1.2 "$url" 2>/dev/null)" || status=$? if [ "$status" -ne 0 ]; then FAULT="curl failed, exit $status" return 1 @@ -205,15 +216,22 @@ looks_like_the_launcher() { return 1 } -# THE ONE COMMAND THAT INSTALLS IT HERE, for a machine with no pipx. POSIX -# shell, no sudo: Homebrew's where `brew` is, and otherwise pip, under -# whichever Python name this machine has, at the version CI runs. +# THE ONE COMMAND THAT INSTALLS IT HERE, for a machine with no pipx, PRINTED +# AND NEVER RUN, in POSIX shell: Homebrew's where `brew` is; where `apt-get` +# is, pipx from apt and then git-filter-repo through it, because Debian 12 +# and Ubuntu 23.04 on refuse `pip install --user` into the system Python +# (externally-managed-environment); and otherwise pip, under whichever Python +# name this machine has. Always at the version CI runs, except brew's. filter_repo_command() { local candidate if command -v brew >/dev/null 2>&1; then printf '%s' "brew install git-filter-repo" return 0 fi + if command -v apt-get >/dev/null 2>&1; then + printf '%s' "sudo apt-get install -y pipx && pipx install $FILTER_REPO_PIN" + return 0 + fi for candidate in python3 python; do if command -v "$candidate" >/dev/null 2>&1; then printf '%s' "$candidate -m pip install --user $FILTER_REPO_PIN" @@ -305,17 +323,27 @@ main() { die "could not make a temporary directory under ${TMPDIR:-/tmp}. Set \$TMPDIR to a directory you can write to." trap 'rm -rf -- "$WORKDIR"' EXIT launcher_url="https://raw.githubusercontent.com/$REPO/$version/openRepoShape" - curl -fsSL "$launcher_url" -o "$WORKDIR/openRepoShape" || status=$? - [ "$status" -eq 0 ] && [ -s "$WORKDIR/openRepoShape" ] || - die "could not fetch $launcher_url (curl exit $status), so nothing was installed. Is $version a release of $REPO? https://github.com/$REPO/releases" + curl -fsSL --proto '=https' --proto-redir '=https' --tlsv1.2 "$launcher_url" -o "$WORKDIR/openRepoShape" || status=$? + # WHICH FAILURE IT WAS DECIDES WHAT TO SAY. curl's exit 22 is the server + # answering with an error, which for a raw URL means no such file at that + # ref; anything else is curl getting no answer at all (DNS, a proxy, TLS, + # a timeout), where asking whether the release exists would send the + # person looking in the wrong place. + case "$status" in + 0) [ -s "$WORKDIR/openRepoShape" ] || + die "fetched $launcher_url, but it was empty, so nothing was installed." ;; + 22) die "could not fetch $launcher_url: the server answered with an error (curl exit 22), so nothing was installed. Is $version a release of $REPO? https://github.com/$REPO/releases" ;; + *) die "could not fetch $launcher_url: curl got no answer (curl exit $status), so nothing was installed. Check this machine's network, DNS and proxy, then run the line again." ;; + esac looks_like_the_launcher "$WORKDIR/openRepoShape" || die "$launcher_url is not the openRepoShape command (its first line is not #!/usr/bin/env bash, or it has no INSTALLABLES line), so nothing was installed." say "fetch: $launcher_url" bin_dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" target="$bin_dir/openRepoShape" - say "install: OPENREPOSHAPE_REF=$version openRepoShape --install" - OPENREPOSHAPE_REF="$version" "${BASH:-bash}" "$WORKDIR/openRepoShape" --install || + say "install: OPENREPOSHAPE_REF=$version OPENREPOSHAPE_RECORD_REF=1 openRepoShape --install" + OPENREPOSHAPE_REF="$version" OPENREPOSHAPE_RECORD_REF=1 \ + "${BASH:-bash}" "$WORKDIR/openRepoShape" --install || die "the $version openRepoShape's --install refused (exit $?); its own refusal above says why." # THE PIN, CHECKED BY ASKING THE INSTALLED COMMAND, with the variable @@ -326,36 +354,56 @@ main() { *"@ $version)") ;; *) pinned=0 - say "install: $target does not stay on $version: it says '${shown:-nothing}' to --version. A release older than v1.0.1 cannot record the ref it was installed at; export OPENREPOSHAPE_REF=$version to keep it there." + say "install: $target does not stay on $version: it says '${shown:-nothing}' to --version. A release older than v1.0.1 cannot record the tag it was installed at: install v1.0.1 or newer, or export OPENREPOSHAPE_REF=$version in your shell profile to keep it there." ;; esac if [ -n "${OPENREPOSHAPE_REF:-}" ] && [ "$OPENREPOSHAPE_REF" != "$version" ]; then - say "install: \$OPENREPOSHAPE_REF is set in this shell, and while it is, every run uses it instead of $version. Unset it to run $version." + say "install: \$OPENREPOSHAPE_REF is set in this shell, by your shell profile or by hand, and while it is, every run uses it instead of $version. Remove it from your shell profile to run $version." fi # 3. GIT-FILTER-REPO. filter_repo_step - # 4. THE PREFLIGHT, LAST, THROUGH THE INSTALLED COMMAND AND ITS RECORD. + # 4. THE PREFLIGHT, LAST, THROUGH THE INSTALLED COMMAND AND ITS RECORD, so + # its setup.sh is the release's. Where the command did not record the tag, + # the tag is handed to this one run instead, so that a command which + # follows main is still not checked with main's setup.sh. say "preflight: $target --preflight" status=0 - ( - unset OPENREPOSHAPE_REF - "${BASH:-bash}" "$target" --preflight - ) || status=$? + if [ "$pinned" -eq 1 ]; then + ( + unset OPENREPOSHAPE_REF + "${BASH:-bash}" "$target" --preflight + ) || status=$? + else + OPENREPOSHAPE_REF="$version" "${BASH:-bash}" "$target" --preflight || status=$? + fi - if [ "$status" -ne 0 ]; then - say "done: openRepoShape $version is installed at $target, but the preflight did not pass (exit $status); it names what is missing above." + # EXIT 1 IS THE PREFLIGHT'S "not ready", which names what is missing; any + # other status is a run that did not get that far (setup.sh could not be + # fetched, or there is no Python to run it), and its REFUSED line says so. + case "$status" in + 0) ;; + 1) + say "done: openRepoShape $version is installed at $target, but the preflight did not pass (exit 1); it names what is missing above." exit 1 - fi + ;; + *) + say "done: openRepoShape $version is installed at $target, but the preflight did not run to the end (exit $status); the REFUSED line above says why." + exit 1 + ;; + esac if [ "$pinned" -eq 0 ]; then say "done: openRepoShape $version is installed at $target and this machine is ready, but the command does not stay on $version (see above)." exit 1 fi - say "done: openRepoShape $version is installed at $target, and this machine is ready. Run the same line again to upgrade to the newest release." + say "done: openRepoShape $version is installed at $target, pinned to the release tag, and this machine is ready. Run the same line again to upgrade to the newest release." exit 0 } # `${1+"$@"}`, setup.sh's own spelling: no arguments is the one-line run, and -# that has to survive `set -u` under every bash this has to run on. -main ${1+"$@"} +# that has to survive `set -u` under every bash this has to run on. In +# BRACES, so the call is one compound command that bash reads to its closing +# brace before running any of it: a download cut anywhere inside this line +# is a syntax error, and runs nothing. +{ main ${1+"$@"}; } diff --git a/openRepoShape b/openRepoShape index 246bf217..bee32cde 100755 --- a/openRepoShape +++ b/openRepoShape @@ -4,7 +4,7 @@ # openRepoShape — the short way in: `openRepoShape Atlas --org `. # # Install it once, from anywhere, with no checkout and no clone. The newest -# release, pinned to it (`install.sh`, #564): +# release, pinned to its tag (`install.sh`, #564): # # curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash # @@ -39,13 +39,21 @@ SELF="${BASH_SOURCE[0]:-$0}" # chose: setup.sh is told it (`--shape-ref`) and `--doctor` checks its clone # out at it, where an unnamed `main` leaves both to the default branch. Two # things name one, and the first wins: `$OPENREPOSHAPE_REF`, set for a run, -# and the ref `--install` RECORDED beside this file (`$SELF.ref`, read by -# `read_recorded_ref` below, once `checked_ref` exists to check it). With +# and the ref `--install` RECORDED beside this file (`.ref`, read by +# `read_recorded_ref` below, once `ref_fault` exists to check it). With # neither, it is `main`, unnamed, exactly as before there was a record. REF="${OPENREPOSHAPE_REF:-main}" REF_NAMED=0 [ -z "${OPENREPOSHAPE_REF:-}" ] || REF_NAMED=1 +#: 1 when the ref in force came from the record, which is what lets a +#: command re-installing itself keep it with no opt-in (see `record_ref`). +REF_FROM_RECORD=0 + +#: Why the record beside this file was refused, said once the arguments are +#: read, so that `--help` still answers past a broken record. +RECORD_FAULT="" + #: The record's name is the command's own plus this, so the record of #: `~/.local/bin/openRepoShape` is `~/.local/bin/openRepoShape.ref`. RECORD_SUFFIX=".ref" @@ -113,14 +121,17 @@ opensoft/openRepoTools', and its own installer places them: $OPENREPOSHAPE_ORG the organisation, when --org is not given $OPENREPOSHAPE_BIN_DIR where --install puts it (default ~/.local/bin) $OPENREPOSHAPE_SETUP_SH run this setup.sh on disk instead of fetching one - -`--install` RECORDS THE REF it runs at, in `openRepoShape.ref` beside the -command it places: $OPENREPOSHAPE_REF, which the one-line installer -(install.sh) sets to the release it installs. The installed command then -stays on that ref: it fetches setup.sh at it, setup.sh clones the standard -at it (`--shape-ref`), and `--doctor` checks its clone out at it. -$OPENREPOSHAPE_REF set for one run still wins, and `--version` prints the -ref in force. Installed with no ref, it records none and follows main. + $OPENREPOSHAPE_RECORD_REF 1: --install records $OPENREPOSHAPE_REF (below) + +`--install` RECORDS A REF ONLY WHEN ASKED: with $OPENREPOSHAPE_RECORD_REF=1 +it writes $OPENREPOSHAPE_REF to `openRepoShape.ref` beside the command it +places, which is what the one-line installer (install.sh) does with the +release tag it installs. The installed command then stays on that ref: it +fetches setup.sh at it, setup.sh clones the standard at it (`--shape-ref`), +and `--doctor` checks its clone out at it. $OPENREPOSHAPE_REF set for one run +still wins, and `--version` prints the ref in force. A command re-installing +itself keeps its own record; any other install records nothing, removes a +record already there, and follows main. `--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` over (default `.`) and prints one row per check and ONE verdict: @@ -189,6 +200,10 @@ workdir() { # curl is the fallback for a machine with no `gh`. Each attempt writes to a # file and is checked before its bytes are used, so a half-written first # attempt is never mistaken for a fetch. +# +# HTTPS ONLY, REDIRECTS INCLUDED, TLS 1.2 OR NEWER (#564, SonarCloud +# shell:S6506): the flags are written out on the curl line itself, never +# gathered into an array, so a reader and the analyser both see them there. fetch_from_repo() { local path="$1" dest="$2" if command -v gh >/dev/null 2>&1 && @@ -198,7 +213,8 @@ fetch_from_repo() { return 0 fi if command -v curl >/dev/null 2>&1 && - curl -fsSL "https://raw.githubusercontent.com/$REPO/$REF/$path" \ + curl -fsSL --proto '=https' --proto-redir '=https' --tlsv1.2 \ + "https://raw.githubusercontent.com/$REPO/$REF/$path" \ >"$dest" 2>/dev/null && [ -s "$dest" ]; then return 0 fi @@ -329,23 +345,27 @@ machine_report() { # command copied away without its record follows `main` again, and says so in # `--version`. # -# THE REF RECORDED IS THE ONE IN FORCE: `$OPENREPOSHAPE_REF`, else the record -# beside the copy being run, so a pinned command re-installing itself stays -# pinned. With neither, any record already there is REMOVED, which is what -# makes an install at main follow main however the command was installed -# before. Every outcome is one line, and no ref and no record is no line at -# all. The record's old value is echoed only when it passes the same shape -# check, because a hand-edited file is not this command's to print. +# RECORDED ONLY WHEN ASKED (#564, the review on #577). `$OPENREPOSHAPE_REF` +# alone records nothing, because a script that sets it around `--install` +# for its own reasons is not asking for a pin: workBenches sets it to a value +# that names no ref at all, to make any fetch fail loudly, and a record of +# that would break every later run on every machine it installs to. So the +# ref in force is written only when `$OPENREPOSHAPE_RECORD_REF` is 1, as the +# one-line installer sets it, or when it came from the record beside the copy +# being run, so a pinned command re-installing itself stays pinned. Any other +# install is an install with no ref: it records nothing, and REMOVES a record +# already there, which makes an install at main follow main however the +# command was installed before. Every outcome is one line, and nothing to +# record and no record is no line at all. The record's old value is echoed +# only when it is a ref this command would accept, because a hand-edited +# file is not this command's to print. record_ref() { - local record="$1$RECORD_SUFFIX" was="" + local record="$1" recording="$2" was="" if [ -f "$record" ]; then was="$(< "$record")" || was="" - case "$was" in - '' | [!A-Za-z0-9]* | *..* | *[!A-Za-z0-9._/-]*) was="another ref" ;; - esac - [ "${#was}" -le 255 ] || was="another ref" + [ -z "$(ref_fault "$was" "it")" ] || was="another ref" fi - if [ "$REF_NAMED" -eq 0 ]; then + if [ "$recording" -eq 0 ]; then [ -e "$record" ] || return 0 rm -f -- "$record" say "openRepoShape: no longer pinned to ${was:-another ref}; it follows main, or \$OPENREPOSHAPE_REF where set (removed $record)" @@ -359,11 +379,20 @@ record_ref() { install_commands() { local dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" - local name target source verb placed=0 - # THE REF IS CHECKED BEFORE ANYTHING IS COLLECTED OR PLACED: it is about - # to be written down and read back by every later run, and a refusal - # after the copy would leave a new command beside an old record. - [ "$REF_NAMED" -eq 0 ] || checked_ref "$REF" + local name target source verb placed=0 recording=0 record + if [ "$REF_FROM_RECORD" -eq 1 ] || + { [ "$REF_NAMED" -eq 1 ] && [ "${OPENREPOSHAPE_RECORD_REF:-}" = 1 ]; }; then + recording=1 + fi + # THE RECORD SITS BESIDE THE FILE, symlinks followed, as it is read. + record="$(resolved "$dir/openRepoShape")$RECORD_SUFFIX" + # BOTH CHECKED BEFORE ANYTHING IS COLLECTED OR PLACED: a ref about to be + # written down and read back by every later run, and a record path this + # install could neither write nor remove. A refusal after the copy would + # leave a new command beside an old record. + [ "$recording" -eq 0 ] || checked_ref "$REF" + [ ! -e "$record" ] || [ -f "$record" ] || + die "$record is not a file, so this install could neither record its ref there nor remove it. Move it away, then run this again." collect_commands mkdir -p "$dir" for name in "${INSTALLABLES[@]}"; do @@ -382,7 +411,7 @@ install_commands() { done # The record belongs to the one command that reads it, and is written # only once that command is in place. - record_ref "$dir/openRepoShape" + record_ref "$record" "$recording" # THE COUNT, out loud, and kept at one installable (#92): it costs a line # and is what keeps $INSTALLABLES honest — a person reading N lines cannot # tell whether an N+1th was meant to be there. The sentence that made @@ -475,43 +504,94 @@ shape_checkout() { # this script has said a word; and only then the SHAPE, which is safe to echo # back because it has passed both. # -# WHOSE REF IT IS, AND THE WAY OUT, ARE ARGUMENTS (#564): `$OPENREPOSHAPE_REF` -# by default, or the record `--install` wrote, whose refusal also says how to -# replace it. An EMPTY ref is refused too, which no caller could reach before -# the record: an empty variable is no ref at all, while an empty file is one. -checked_ref() { - local ref="$1" what="${2:-\$OPENREPOSHAPE_REF}" out="${3:+ $3}" - [ -n "$ref" ] || - die "$what is empty, which is not a branch, tag or commit this tool will pass to \`git checkout\`.$out" - if [ "${#ref}" -gt 255 ]; then - die "$what is ${#ref} characters long; a branch, tag or commit this tool will pass to \`git checkout\` is at most 255 characters.$out" +# WHOSE REF IT IS IS AN ARGUMENT (#564): `$OPENREPOSHAPE_REF` by default, or +# the record `--install` wrote. `ref_fault` SAYS what is wrong and refuses +# nothing, so the record can be checked as it is read and refused only once +# the arguments are, after `--help` has had its chance; `checked_ref` is the +# refusing spelling. An EMPTY ref is a fault too, which no caller could reach +# before the record: an empty variable is no ref at all, while an empty file +# is one. +ref_fault() { + local ref="$1" what="$2" + if [ -z "$ref" ]; then + printf '%s' "$what is empty, which is not a branch, tag or commit this tool will pass to \`git checkout\`." + elif [ "${#ref}" -gt 255 ]; then + printf '%s' "$what is ${#ref} characters long; a branch, tag or commit this tool will pass to \`git checkout\` is at most 255 characters." + else + case "$ref" in + *[![:print:]]*) printf '%s' "$what carries a control character, which is not a branch, tag or commit this tool will pass to \`git checkout\`." ;; + [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) printf '%s' "$what is '$ref', which is not a branch, tag or commit this tool will pass to \`git checkout\`: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'." ;; + esac fi - case "$ref" in - *[![:print:]]*) die "$what carries a control character, which is not a branch, tag or commit this tool will pass to \`git checkout\`.$out" ;; - esac - case "$ref" in - [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) die "$what is '$ref', which is not a branch, tag or commit this tool will pass to \`git checkout\`: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'.$out" ;; - esac +} + +checked_ref() { + local fault + fault="$(ref_fault "$1" "${2:-\$OPENREPOSHAPE_REF}")" + [ -z "$fault" ] || die "$fault" +} + +# A PATH WITH ITS SYMLINKS FOLLOWED (#564, the review on #577). The record +# sits beside the FILE `--install` placed, so a link to the command from +# anywhere else (`~/bin/openRepoShape -> ~/.local/bin/openRepoShape`) must +# read the record there, not look for one beside the link. `readlink` with +# no `-f`, which macOS before 12.3 lacks, one link at a time and at most 40 +# of them, the kernel's own limit, so a loop of links ends. Where `readlink` +# fails or is missing the path is used as it is, which is where the record +# was looked for before. Git Bash makes copies for `ln -s` unless told +# otherwise, and a copy is no link. +resolved() { + local path="$1" link hops=0 + while [ -L "$path" ] && [ "$hops" -lt 40 ]; do + link="$(readlink -- "$path" 2>/dev/null)" && [ -n "$link" ] || break + case "$link" in + /*) path="$link" ;; + *) path="$(dirname -- "$path")/$link" ;; + esac + hops=$((hops + 1)) + done + printf '%s' "$path" } # THE RECORD `--install` LEFT BESIDE THIS FILE, READ ONCE, BEFORE ANY RUN # (#564). Only when no `$OPENREPOSHAPE_REF` was set, which is what lets that # variable win for a run, and only beside a FILE: a script arriving on stdin -# has no place of its own to have been installed at. CHECKED HERE, AT ONCE, -# rather than where the ref is used: it is a file on disk that every run +# has no place of its own to have been installed at. CHECKED HERE, as it is +# read, rather than where the ref is used: it is a file on disk that every run # reads, `--version` prints it, and a value nobody typed is not echoed until -# it has passed. `$(< file)` is bash's own read, so no `cat` is needed. +# it has passed. A record that fails is NOT refused here but kept in +# `$RECORD_FAULT`, and `refuse_bad_record` refuses with it once the arguments +# have been read: `--help` answers past a broken record, and every run that +# would use the ref refuses, `--version` first. `$(< file)` is bash's own +# read, so no `cat` is needed; a record that is not a readable FILE (a +# directory, say) is a fault in the same words as the rest, never a raw bash +# error. +RECORD_WAY_OUT="Run the installer again to record a release, or delete that file to follow main." read_recorded_ref() { - local record="$SELF$RECORD_SUFFIX" - [ "$REF_NAMED" -eq 0 ] && [ -f "$SELF" ] && [ -f "$record" ] || return 0 - REF="$(< "$record")" || - die "could not read $record, the ref \`--install\` recorded. Run the installer again, or delete that file to follow main." - checked_ref "$REF" "the ref recorded in $record" \ - "Run the installer again to record a release, or delete that file to follow main." + local record text fault + [ "$REF_NAMED" -eq 0 ] && [ -f "$SELF" ] || return 0 + record="$(resolved "$SELF")$RECORD_SUFFIX" + [ -e "$record" ] || return 0 + if [ ! -f "$record" ] || [ ! -r "$record" ]; then + RECORD_FAULT="$record is not a file this command can read, so it cannot read the ref \`--install\` recorded there. Remove it, then run the installer again to record a release, or leave it removed to follow main." + return 0 + fi + text="$(< "$record")" + fault="$(ref_fault "$text" "the ref recorded in $record")" + if [ -n "$fault" ]; then + RECORD_FAULT="$fault $RECORD_WAY_OUT" + return 0 + fi + REF="$text" REF_NAMED=1 + REF_FROM_RECORD=1 } read_recorded_ref +refuse_bad_record() { + [ -z "$RECORD_FAULT" ] || die "$RECORD_FAULT" +} + # `--doctor`: RUN THE STANDARD'S OWN SCRIPT, never a reimplementation here. # Beside a checkout -- this file sitting in one, which is how a developer runs # it -- that checkout answers, and nothing is fetched at all. Installed, or @@ -603,7 +683,7 @@ run_doctor() { while [ $# -gt 0 ]; do case "$1" in -h | --help) usage; exit 0 ;; - --version) say "openRepoShape ($REPO @ $REF)"; exit 0 ;; + --version) refuse_bad_record; say "openRepoShape ($REPO @ $REF)"; exit 0 ;; # TWO MODES IS A REFUSAL, NEVER LAST-WINS. `--doctor --install` # used to enter the install path and silently drop the diagnosis, and # the other order dropped the install (Copilot, PR #96). Each of these @@ -696,6 +776,10 @@ while [ $# -gt 0 ]; do esac done +# A BROKEN RECORD IS REFUSED HERE, after `--help` has had its chance and +# before any run that would use the ref it should have named (#564). +refuse_bad_record + if [ "$MODE" = "doctor" ]; then # ONE PATH AND THE DOCTOR'S TWO FLAGS, NOTHING ELSE. Every other # argument this command knows is about a run that CREATES something, and diff --git a/tests/test_install_sh.py b/tests/test_install_sh.py index 7ced9afc..34c7de5a 100644 --- a/tests/test_install_sh.py +++ b/tests/test_install_sh.py @@ -57,6 +57,11 @@ RELEASE = "v9.9.9" FILTER_REPO = "git-filter-repo==2.47.0" +#: The flags every curl in the installer and the launcher carries, as the +#: fakes log them once the shell has taken the quotes off (SonarCloud +#: shell:S6506: HTTPS only, redirects included, TLS 1.2 or newer). +HTTPS_ONLY = "--proto =https --proto-redir =https --tlsv1.2" + def latest(slug: str = SLUG) -> str: return f"https://github.com/{slug}/releases/latest" @@ -104,12 +109,16 @@ def launcher(directory, name): def curl(): url = next((arg for arg in ARGS if arg.startswith("https://")), "") + if url in NET.get("unreachable", []): + sys.stderr.write("curl: (6) Could not resolve host\n") + return 6 if "-fsSI" in ARGS: location = NET["redirects"].get(url) if location is None: return refuse("curl: (22) The requested URL returned error: 404") - sys.stdout.write("HTTP/2 302 \r\nlocation: %s\r\n" - "content-length: 0\r\n\r\n" % location) + sys.stdout.write("HTTP/2 302 \r\n%s: %s\r\n" + "content-length: 0\r\n\r\n" + % (NET.get("location_header", "location"), location)) return 0 served = NET["files"].get(url) if served is None: @@ -162,6 +171,7 @@ def setup_sh(): case "$1" in --install) mkdir -p "$OPENREPOSHAPE_BIN_DIR" && cp "$0" "$OPENREPOSHAPE_BIN_DIR/openRepoShape" ;; --version) printf 'openRepoShape (%s @ %s)\\n' "$OPENREPOSHAPE_REPO" "${OPENREPOSHAPE_REF:-main}" ;; +--preflight) printf 'old preflight at %s\\n' "${OPENREPOSHAPE_REF:-main}" ;; esac """ @@ -214,18 +224,21 @@ def serve(self, ref: str, slug: str = SLUG, self.config["files"][raw("setup.sh", ref, slug)] = str( self.fake("setup.sh", served)) - def run(self, *args: str, piped: bool = False, - env: dict | None = None) -> subprocess.CompletedProcess: - """`bash install.sh `, or `… | bash -s -- ` when piped.""" + def run(self, *args: str, piped: bool = False, env: dict | None = None, + script: str | None = None) -> subprocess.CompletedProcess: + """`bash install.sh `, or `… | bash -s -- ` when piped; + `script` pipes that text in place of the installer's own.""" (self.root / "net.json").write_text(json.dumps(self.config), encoding="utf-8") gh_config = self.tmp_path / "gh-config" gh_config.mkdir(exist_ok=True) + piped = piped or script is not None argv = [BASH, "-s", "--", *args] if piped \ else [BASH, str(INSTALLER), *args] return subprocess.run( argv, capture_output=True, text=True, check=False, - input=INSTALLER.read_text(encoding="utf-8") if piped else "", + input=(script or INSTALLER.read_text(encoding="utf-8")) + if piped else "", cwd=str(self.tmp_path), start_new_session=True, env=command_env(home=self.tmp_path, setup_sh=None, env={ "PATH": str(self.path), "GH_TOKEN": "bogus", @@ -283,22 +296,24 @@ def test_the_one_line_installs_the_newest_release_pinned_to_it(net): steps = [ f"release: {RELEASE}, the newest release of {SLUG} ({latest()})", f"fetch: {raw('openRepoShape')}", - f"install: OPENREPOSHAPE_REF={RELEASE} openRepoShape --install", + f"install: OPENREPOSHAPE_REF={RELEASE} OPENREPOSHAPE_RECORD_REF=1 " + "openRepoShape --install", f"openRepoShape: pinned to {RELEASE}, recorded in {target}.ref", "git-filter-repo: ", f"preflight: {target} --preflight", - f"done: openRepoShape {RELEASE} is installed at {target}, and this " - "machine is ready. Run the same line again to upgrade to the newest " - "release.", + f"done: openRepoShape {RELEASE} is installed at {target}, pinned to " + "the release tag, and this machine is ready. Run the same line again " + "to upgrade to the newest release.", ] lines = result.stdout.splitlines() at = [next(i for i, line in enumerate(lines) if line.startswith(step)) for step in steps] assert at == sorted(at), result.stdout calls = net.calls() - assert calls[0] == f"curl -fsSI {latest()}" - assert calls[1].startswith(f"curl -fsSL {raw('openRepoShape')} -o ") - assert f"curl -fsSL {raw('setup.sh')}" in calls + assert calls[0] == f"curl -fsSI {HTTPS_ONLY} {latest()}" + assert calls[1].startswith( + f"curl -fsSL {HTTPS_ONLY} {raw('openRepoShape')} -o ") + assert f"curl -fsSL {HTTPS_ONLY} {raw('setup.sh')}" in calls assert calls[-1] == preflight_call(), calls assert not any("releases/latest" in call for call in calls[1:]), calls @@ -375,15 +390,30 @@ def test_a_version_that_is_not_a_ref_is_refused_before_any_request( net.refused(result) assert said in result.stderr assert net.calls() == [] - assert "forged" not in result.stderr and "a" * 256 not in result.stderr + assert "forged" not in result.stderr + assert "a" * 256 not in result.stderr def test_a_failed_fetch_names_its_url_and_installs_nothing(net): + """An error from the server asks whether the release exists.""" del net.config["files"][raw("openRepoShape")] result = net.run() net.refused(result) - assert f"could not fetch {raw('openRepoShape')} (curl exit 22), so " \ - "nothing was installed" in result.stderr + assert f"could not fetch {raw('openRepoShape')}: the server answered " \ + "with an error (curl exit 22), so nothing was installed. Is " \ + f"{RELEASE} a release of {SLUG}?" in result.stderr + + +def test_a_fetch_with_no_answer_points_at_the_network_not_the_release(net): + """No answer at all (DNS, a proxy, TLS) is not a question about the + release, and the refusal does not send anyone looking there.""" + net.config["unreachable"] = [raw("openRepoShape")] + result = net.run() + net.refused(result) + assert f"could not fetch {raw('openRepoShape')}: curl got no answer " \ + "(curl exit 6), so nothing was installed. Check this machine's " \ + "network, DNS and proxy" in result.stderr + assert "a release of" not in result.stderr def test_a_page_that_is_not_the_command_is_refused(net): @@ -439,20 +469,29 @@ def no_pipx(net): def no_pipx_but_brew(net): no_pipx(net) net.fake("brew") + net.fake("apt-get") return " brew install git-filter-repo" +def no_pipx_but_apt(net): + no_pipx(net) + net.fake("apt-get") + return f" sudo apt-get install -y pipx && pipx install {FILTER_REPO}" + + @pytest.mark.parametrize("arrange", [ present_on_path, answering_git, pipx_onto_path, pipx_off_path, - pipx_failing, no_pipx, no_pipx_but_brew]) + pipx_failing, no_pipx, no_pipx_but_brew, no_pipx_but_apt]) def test_git_filter_repo_is_said_installed_or_printed(net, arrange): """The three branches, and the ways each can end. PRESENT, on `$PATH` or as `git filter-repo`: said, and pipx is never run. NOT PRESENT AND PIPX IS THERE: said first, then `pipx install` at CI's version, then what came of it. NEITHER: the one command for this - machine, printed and never run. None changes the exit status, and the - step sits between the install and the preflight. + machine, printed and never run: brew's where brew is, pipx from apt where + apt-get is (Debian 12 and Ubuntu 23.04 on refuse `pip install --user`), + and pip otherwise. Nothing runs `sudo` or `apt-get`. None changes the + exit status, and the step sits between the install and the preflight. """ expected = arrange(net) result = net.run() @@ -470,26 +509,35 @@ def test_git_filter_repo_is_said_installed_or_printed(net, arrange): net.calls().index(preflight_call()) else: assert ran_pipx == [], net.calls() - if arrange in (no_pipx, no_pipx_but_brew): + if arrange in (no_pipx, no_pipx_but_brew, no_pipx_but_apt): assert lines[lines.index(expected) - 1] == ( "git-filter-repo: not installed, and there is no pipx to install " "it with. Only adopt-project.py needs it; before adopting a " "repository, run:") assert lines.index(expected) < lines.index( f"preflight: {net.bin}/openRepoShape --preflight") + assert not any(call.split()[0] in ("apt-get", "brew", "sudo") + for call in net.calls()), net.calls() # --- the preflight, the exit status, and running it again -------------------- +@pytest.mark.parametrize("status, said", [ + pytest.param(1, "but the preflight did not pass (exit 1); it names what " + "is missing above.", id="not-ready"), + pytest.param(2, "but the preflight did not run to the end (exit 2); the " + "REFUSED line above says why.", id="did-not-run"), +]) def test_a_preflight_that_does_not_pass_exits_one_with_the_command_placed( - net): - net.config["preflight_exit"] = 1 + net, status, said): + """Exit 1 is the preflight's own "not ready", which names what is + missing; anything else stopped before it could say.""" + net.config["preflight_exit"] = status result = net.run() assert result.returncode == 1, result.stdout + result.stderr net.installed(result) assert f"done: openRepoShape {RELEASE} is installed at " \ - f"{net.bin}/openRepoShape, but the preflight did not pass (exit 1); " \ - "it names what is missing above." in result.stdout + f"{net.bin}/openRepoShape, {said}" in result.stdout def test_a_variable_in_the_shell_is_named_and_the_preflight_runs_the_pin(net): @@ -499,8 +547,9 @@ def test_a_variable_in_the_shell_is_named_and_the_preflight_runs_the_pin(net): result = net.run(env={"OPENREPOSHAPE_REF": "v0.0.1"}) assert result.returncode == 0, result.stdout + result.stderr net.installed(result) - assert f"install: $OPENREPOSHAPE_REF is set in this shell, and while it " \ - f"is, every run uses it instead of {RELEASE}. Unset it to run " \ + assert "install: $OPENREPOSHAPE_REF is set in this shell, by your " \ + "shell profile or by hand, and while it is, every run uses it " \ + f"instead of {RELEASE}. Remove it from your shell profile to run " \ f"{RELEASE}." in result.stdout assert net.calls()[-1] == preflight_call() @@ -528,7 +577,9 @@ def test_the_same_line_again_changes_nothing_and_then_upgrades(net): def test_a_release_whose_command_records_no_ref_exits_one(net): """A release before v1.0.1 installs a command that follows main. The - installer asks the installed command, says so, and exits 1.""" + installer asks the installed command, says so, and exits 1; and the + preflight it ends with is still handed the release's tag, so it is not + main's setup.sh that checks the machine.""" old = net.root / "old-openRepoShape" old.write_text(OLD_LAUNCHER, encoding="utf-8") net.serve(RELEASE, launcher=old) @@ -538,11 +589,86 @@ def test_a_release_whose_command_records_no_ref_exits_one(net): assert f"install: {net.bin}/openRepoShape does not stay on {RELEASE}: " \ f"it says 'openRepoShape ({SLUG} @ main)' to --version." \ in result.stdout - assert f"export OPENREPOSHAPE_REF={RELEASE}" in result.stdout + assert "A release older than v1.0.1 cannot record the tag it was " \ + "installed at: install v1.0.1 or newer, or export " \ + f"OPENREPOSHAPE_REF={RELEASE} in your shell profile to keep it " \ + "there." in result.stdout + assert f"old preflight at {RELEASE}" in result.stdout.splitlines() assert result.stdout.splitlines()[-1].endswith( f"but the command does not stay on {RELEASE} (see above).") +@pytest.mark.parametrize("header, location", [ + pytest.param("Location", tag_page(RELEASE), id="upper-case"), + pytest.param("location", f"/{SLUG}/releases/tag/{RELEASE}", + id="relative"), +]) +def test_the_redirect_is_read_in_any_case_and_relative(net, header, location): + """HTTP/1.1 may send `Location:`, and a location may be relative; both + name the release the same way.""" + net.config["location_header"] = header + net.config["redirects"][latest()] = location + result = net.run() + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"release: {RELEASE}, the newest release of {SLUG} ({latest()})" \ + in result.stdout.splitlines() + + +def test_openreposhape_bin_dir_is_where_the_command_and_its_record_go(net): + """The installer places, pins and preflights the command where the + launcher puts it, and leaves `~/.local/bin` alone.""" + default = net.bin + net.bin = net.tmp_path / "elsewhere" + result = net.run(env={"OPENREPOSHAPE_BIN_DIR": str(net.bin)}) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"preflight: {net.bin}/openRepoShape --preflight" \ + in result.stdout.splitlines() + assert not default.exists() + + +#: A curl that RUNS, in either script: tab-indented, alone or as the value +#: of an assignment. Usage text and refusals carry curl lines too, indented +#: with spaces, and are printed, never run. +CURL_RUN = re.compile(r"\t+(\w+=\"\$\()?curl -fsS") + + +@pytest.mark.parametrize("name", ["install.sh", "openRepoShape"]) +def test_every_curl_that_runs_is_https_only_redirects_included(name): + """SonarCloud shell:S6506, held here rather than found there: each curl + line refuses any protocol but HTTPS, for its redirects too, written out + on the line itself.""" + runs = [line for line in (REPO / name).read_text( + encoding="utf-8").splitlines() if CURL_RUN.match(line)] + assert runs, f"{name} runs no curl; this guard has nothing to hold" + for line in runs: + assert "--proto '=https' --proto-redir '=https' --tlsv1.2" in line, ( + f"{name}: a curl that is not HTTPS only:\n{line}") + + +#: Where a download can stop in the last line. Right after the word `main` +#: is the one the braces are for: unbraced, that line would run `main` with +#: none of the arguments, and install the newest release where one was named. +CUTS = [ + pytest.param(lambda text: text[:text.rindex("{ main")], id="before-it"), + pytest.param(lambda text: text[:text.rindex(" ${1+")], id="after-main"), + pytest.param(lambda text: text[:text.rindex(" }")], id="before-the-brace"), +] + + +@pytest.mark.parametrize("cut", CUTS) +def test_a_download_cut_off_in_the_last_line_runs_nothing(net, cut): + """Everything runs from `main`, called on the last line inside braces: + a script that stops anywhere in or before that line asks for nothing, + installs nothing and leaves nothing behind.""" + result = net.run("v9.9.8", + script=cut(INSTALLER.read_text(encoding="utf-8"))) + assert net.calls() == [] + assert result.stdout == "" + assert not net.bin.exists() + + # --- the repository it installs from ---------------------------------------- def test_openreposhape_repo_names_where_everything_comes_from(net): @@ -556,7 +682,8 @@ def test_openreposhape_repo_names_where_everything_comes_from(net): assert result.returncode == 0, result.stdout + result.stderr net.installed(result) assert all(SLUG not in call for call in net.calls()), net.calls() - assert f"curl -fsSL {raw('setup.sh', RELEASE, fork)}" in net.calls() + assert f"curl -fsSL {HTTPS_ONLY} {raw('setup.sh', RELEASE, fork)}" \ + in net.calls() @pytest.mark.parametrize("value", [ @@ -593,4 +720,5 @@ def test_it_installs_the_git_filter_repo_ci_runs(): ci = re.findall(r"git-filter-repo==[0-9.]+", ( REPO / ".github" / "workflows" / "tests.yml").read_text( encoding="utf-8")) - assert ci and set(ci) == {pinned}, (pinned, ci) + assert ci, "tests.yml installs no git-filter-repo" + assert set(ci) == {pinned}, (pinned, ci) diff --git a/tests/test_openreposhape_command.py b/tests/test_openreposhape_command.py index 59cb0c00..78a66ab0 100644 --- a/tests/test_openreposhape_command.py +++ b/tests/test_openreposhape_command.py @@ -98,7 +98,8 @@ def command_env(home: Path | None = None, env: dict | None = None, """ environ = dict(os.environ) for name in ("OPENREPOSHAPE_ORG", "OPENREPOSHAPE_REF", "OPENREPOSHAPE_REPO", - "OPENREPOSHAPE_BIN_DIR", "OPENREPOSHAPE_SETUP_SH"): + "OPENREPOSHAPE_BIN_DIR", "OPENREPOSHAPE_SETUP_SH", + "OPENREPOSHAPE_RECORD_REF"): environ.pop(name, None) if setup_sh is not None: environ["OPENREPOSHAPE_SETUP_SH"] = str(setup_sh) @@ -1018,14 +1019,17 @@ def test_install_from_stdin_fetches_itself_into_a_live_workdir(offline_github, OTHER = "other" -def install_at(tmp_path, ref=None, command=COMMAND): +def install_at(tmp_path, ref=None, command=COMMAND, opt_in=True): """`--install` from `command` into `/bin`, with - `$OPENREPOSHAPE_REF` set to `ref` when one is given; the result, and the - path of the command it places.""" + `$OPENREPOSHAPE_REF` set to `ref` when one is given, and asked to record + it (`$OPENREPOSHAPE_RECORD_REF=1`) unless `opt_in` is false; the result, + and the path of the command it places.""" bin_dir = tmp_path / "bin" env = {"OPENREPOSHAPE_BIN_DIR": str(bin_dir)} if ref is not None: env["OPENREPOSHAPE_REF"] = ref + if opt_in: + env["OPENREPOSHAPE_RECORD_REF"] = "1" return (run_cmd("--install", home=tmp_path, env=env, command=command), bin_dir / "openRepoShape") @@ -1147,12 +1151,78 @@ def test_a_record_that_is_not_a_ref_is_refused_by_name(tmp_path, text, shown): in result.stderr assert "Run the installer again to record a release, or delete that " \ "file to follow main." in result.stderr - assert "forged" not in result.stderr and "a" * 256 not in result.stderr + assert "forged" not in result.stderr + assert "a" * 256 not in result.stderr assert result.stdout == "" + assert run_cmd("--help", command=installed).returncode == 0, ( + "--help must answer past a broken record") assert version_of(installed, {"OPENREPOSHAPE_REF": OTHER}).endswith( f"@ {OTHER})") +def test_a_ref_set_without_the_opt_in_is_never_recorded(tmp_path): + """`$OPENREPOSHAPE_REF` alone is not a request for a pin (the review on + #577). A script that sets it around `--install` for its own reasons + (workBenches sets one that names no ref, so that any fetch fails loudly) + gets the install it always got: the same bytes, no record, `main` in + `--version`, and no check of a value it never meant to be read. A record + already there is removed, exactly as by an install with no ref.""" + install_at(tmp_path, PINNED) + result, installed = install_at( + tmp_path, "pinned-by-workBenches-no-fetch", opt_in=False) + assert result.returncode == 0, result.stderr + assert installed.read_bytes() == COMMAND.read_bytes() + assert not record_of(installed).exists() + assert f"no longer pinned to {PINNED}" in result.stdout + assert version_of(installed).endswith("@ main)") + + again, _ = install_at(tmp_path, "bad..ref", opt_in=False) + assert again.returncode == 0, again.stderr + assert "pinned" not in again.stdout + assert not record_of(installed).exists() + + +#: Links to the installed command from `/links`: each pair is a link's +#: name and what it points at, `None` meaning the command's absolute path, +#: and the run goes through the LAST one. +LINKS = [ + pytest.param([("ors", "../bin/openRepoShape")], id="relative"), + pytest.param([("ors", None)], id="absolute"), + pytest.param([("ors", "../bin/openRepoShape"), ("again", "ors")], + id="a-chain"), +] + + +@pytest.mark.parametrize("links", LINKS) +def test_a_link_to_the_installed_command_reads_its_record(tmp_path, links): + """The record sits beside the FILE `--install` placed, so a symlink to + the command from another directory follows the link, or a chain of them, + to read it.""" + _result, installed = install_at(tmp_path, PINNED) + directory = tmp_path / "links" + directory.mkdir() + for name, points_at in links: + (directory / name).symlink_to(points_at or installed) + through = directory / links[-1][0] + assert version_of(through) == \ + f"openRepoShape (opensoft/openRepoShape @ {PINNED})" + + +def test_a_record_that_is_a_directory_is_refused_in_the_usual_form(tmp_path): + """Not a raw bash error, by every run that reads it and by an install + that would write or remove it, which places nothing.""" + _result, installed = install_at(tmp_path, PINNED) + record = record_of(installed) + record.unlink() + record.mkdir() + for result in (run_cmd("--version", command=installed), + install_at(tmp_path, PINNED)[0]): + assert result.returncode == 2, result.stdout + result.stderr + assert result.stderr.startswith(f"\nREFUSED: {record} is not a file") + assert "Is a directory" not in result.stderr + assert installed.read_bytes() == COMMAND.read_bytes() + + #: WHICH REF EACH LATER RUN USES: recorded at install, then overridden by the #: variable for one run, then neither. The third column is what the run must #: be handed; `None` is `main`, unnamed, which nothing is told. From 322a305e8c6c3fdf7fa1f718ce9ff27e7c9ea708 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 11:21:55 +0000 Subject: [PATCH 04/25] README.md, docs/handbook.html, AGENTS.md and the v1.0.1 release notes, the fix round on #577: install.sh itself never runs sudo while the preflight it ends with offers what is missing behind a typed yes, the command is pinned to the release tag only when asked, step 3 folds the pin into one paragraph six lines shorter, AGENTS.md says the one-line install is the human's to run, and the notes gain the steps that attach install.sh to the release (#564) - README.md: step 3's paragraph now carries what "pinned" means and the corrected sudo sentence, and leaves the exit statuses to `install.sh --help` and the release notes; the worked example's lead-in is one line. The cap moves to 1444 (from 1450 in d0cdbf4), with the reason updated in test_readme_is_short_enough_to_be_read. - docs/handbook.html: the same paragraph, the same rule in "For AI assistants", the #564 footer clause updated, and the new README blob. - AGENTS.md: "The one-line install (`install.sh`) is theirs to run: relay it, never run it." under "Never answer an install prompt", paid for by tightening the `--allow-upstream-org` paragraph's middle clause; still 490 lines, every rule kept. - docs/release-notes/v1.0.1.md: the corrected sudo sentence, the opt-in (`OPENREPOSHAPE_RECORD_REF=1`) and what a script that exports `$OPENREPOSHAPE_REF` now gets, HTTPS only, Git Bash supported but its tests skip on the Windows job (a known limit, followed up in v1.1.0), and a "Before tagging (remove at tag time)" section: upload install.sh to v1.0.1, check the `releases/latest/download/install.sh` redirect, never attach it to v1.0.0, and publish a later patch on an older line with `--latest=false`. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 10 ++--- README.md | 24 +++++------- docs/handbook.html | 38 ++++++++---------- docs/release-notes/v1.0.1.md | 75 +++++++++++++++++++++++++----------- tests/test_repo_hygiene.py | 34 ++++++++-------- 5 files changed, 101 insertions(+), 80 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e579245c..993d2d21 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,13 +25,13 @@ types `yes` at its prompt; you never answer it for them. No terminal: section 2. **Never answer an install prompt.** The preflight asks `Type yes to continue:` for each thing it offers to run — an install, `gh auth login`, `gh auth setup-git`. The human types it; you never install software or log in for them. +The one-line install (`install.sh`) is theirs to run: relay it, never run it. **Never pass `--allow-upstream-org` on your own initiative.** It guards only -`--org opensoft` itself — scaffolding into opensoft's own namespace, the -upstream owner of openRepoShape. If setup.sh refuses because the organisation -you were given resolves to opensoft, STOP and relay what it said; the fix is -almost always an explicit `--org `. Pass `--allow-upstream-org` -only if the human asks for it in those words. +`--org opensoft`, the namespace of openRepoShape's upstream owner. If setup.sh +refuses because the organisation you were given resolves to opensoft, STOP and +relay what it said; the fix is almost always an explicit `--org `. +Pass `--allow-upstream-org` only if the human asks for it in those words. **Never bypass an organisation ruleset.** A ruleset that refuses a direct push to `main` is doing its job; the exit is in the tool's own failure message (for diff --git a/README.md b/README.md index 147b3cb5..70ded88c 100644 --- a/README.md +++ b/README.md @@ -231,19 +231,14 @@ three repositories are created and the push fails. curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash ``` -`install.sh` is attached to every release, so that line installs the newest: -it reads which one that is from github.com's `releases/latest` redirect (no -`gh`, no login), installs that release's command PINNED to it, installs -`git-filter-repo` with `pipx` when `pipx` is there and it is not, or prints -the one command that would, and ends with `openRepoShape --preflight`. It -never runs `sudo`. `| bash -s -- v1.0.1` installs a named release instead. -Exit 0 is installed and ready, 1 installed but the preflight or the pin needs -you, 2 nothing installed. Run the same line again to upgrade. - -**Pinned** means the command stays on its release while main moves on: -`--install` records the ref in `openRepoShape.ref` beside the command, and the -command fetches `setup.sh`, clones the standard and runs `--doctor` at it. -`$OPENREPOSHAPE_REF` overrides it for one run; `--version` prints the ref. +`install.sh` is attached to every release and installs the newest, with no `gh` +and no login. Its command is PINNED to the release tag, recorded in +`openRepoShape.ref` beside it: `setup.sh` and the standard are fetched at that +tag, not at main. It installs `git-filter-repo` with `pipx` where it can, or +prints the command, and ends with `openRepoShape --preflight`. `install.sh` +itself never runs `sudo`; the preflight it ends with offers what is missing, +each behind a typed `yes` at your terminal. `| bash -s -- v1.0.1` names a +release; the same line again upgrades. By hand, the same command at main, which records no ref and follows main: @@ -400,8 +395,7 @@ records as whose act the election was, and it defaults to the `gh` login. #### Install the command -Two ways in, the short one first. Install `openRepoShape` as a command, the -newest release and pinned to it, as step 3 above does: +Two ways in, the short one first. Install the command as step 3 above does: ```sh curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash diff --git a/docs/handbook.html b/docs/handbook.html index 377f934a..84d28811 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -488,8 +488,8 @@

The fastest path — macOS, Linux, WSL2

curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash
 openRepoShape <Project> --org <your-org>

That first line installs ONE command into ~/.local/bin: the - newest release, pinned to it, with git-filter-repo where pipx can - install it, and it ends with openRepoShape --preflight. The + newest release, pinned to its tag, with git-filter-repo where pipx + can install it, and it ends with openRepoShape --preflight. The estate verbs park and resume, for carrying in-flight work to another workstation, are opensoft/openRepoTools’ and have a one-liner of @@ -756,21 +756,15 @@

Install the openRepoShape command, on

Install the openRepoShape command, once, inside WSL

Fetch the flow, in the directory the new project should land in

curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash
-

install.sh is attached to every release, so - that line installs the newest: it reads which one that is from github.com’s - releases/latest redirect (no gh, no login), installs that - release’s command PINNED to it, installs git-filter-repo with - pipx when pipx is there and it is not, or prints the one command - that would, and ends with openRepoShape --preflight. It never runs - sudo. | bash -s -- v1.0.1 installs a named release instead. Exit - 0 is installed and ready, 1 installed but the preflight or the pin needs you, 2 nothing - installed. Run the same line again to upgrade.

-

Pinned means the command stays on its - release while main moves on: --install records the ref in - openRepoShape.ref beside the command, and the command fetches - setup.sh, clones the standard and runs --doctor at it. - $OPENREPOSHAPE_REF overrides it for one run; --version prints the - ref.

+

install.sh is attached to every release and + installs the newest, with no gh and no login. Its command is PINNED to the + release tag, recorded in openRepoShape.ref beside it: setup.sh + and the standard are fetched at that tag, not at main. It installs + git-filter-repo with pipx where it can, or prints the command, + and ends with openRepoShape --preflight. install.sh itself never + runs sudo; the preflight it ends with offers what is missing, each behind a + typed yes at your terminal. | bash -s -- v1.0.1 names a + release; the same line again upgrades.

By hand, the same command at main, which records no ref and follows main:

gh api repos/opensoft/openRepoShape/contents/openRepoShape \
     -H 'Accept: application/vnd.github.raw' | bash -s -- --install
@@ -1818,7 +1812,8 @@

For AI assistants

  • Never answer an install prompt. The preflight asks Type yes to continue: for each thing it offers to run — an install, gh auth login, gh auth setup-git. The human types it; you never - install software or log in for them.
  • + install software or log in for them. The one-line install (install.sh) is + theirs to run: relay it, never run it.
  • Never pass --allow-upstream-org on your own initiative. If setup.sh refuses because the organisation you were given resolves to opensoft, STOP and relay what it said; the fix is almost always an explicit --org <their-org>.
  • @@ -1906,10 +1901,11 @@

    For AI assistants

    merged-away verdict, whose row apply drops, added (#502); the line-ending block an adoption appends to a root .gitattributes the source keeps, and its verification as the source’s bytes plus that block, added (#471); the one-line - install, install.sh, which installs the newest release pinned to it, and the - command’s openRepoShape.ref record, added (#564) — a page + install, install.sh, which installs the newest release pinned to its tag, the + command’s openRepoShape.ref record, and the assistant’s rule that the + line is the human’s to run, added (#564) — a page cannot name the commit that contains it, and the blob below is the fact a test checks. - README.md blob 147b3cb5c1d91c96196d6cef09ba984cfa0f6576. Every command, flag, exit code + README.md blob 70ded88ce2ef4b0cd8faf4100d53fcd98d0da522. Every command, flag, exit code and file name on this page is quoted from that README and from AGENTS.md.

    diff --git a/docs/release-notes/v1.0.1.md b/docs/release-notes/v1.0.1.md index a5c5e3cb..b165c364 100644 --- a/docs/release-notes/v1.0.1.md +++ b/docs/release-notes/v1.0.1.md @@ -1,7 +1,7 @@ # openRepoShape v1.0.1 v1.0.1 is v1.0.0 plus one change: openRepoShape installs in one line, and the -command it installs stays on the release it was installed from (#564). +command it installs stays on the release tag it was installed from (#564). ## What 1.0.1 adds @@ -15,24 +15,26 @@ command it installs stays on the release it was installed from (#564). It reads which release is the newest from github.com's `releases/latest` redirect, with no `gh` and no login. It fetches that release's `openRepoShape` and installs it into `~/.local/bin`, or - `$OPENREPOSHAPE_BIN_DIR`, pinned to the release. It installs + `$OPENREPOSHAPE_BIN_DIR`, pinned to the release tag. It installs `git-filter-repo` 2.47.0 with `pipx` when `pipx` is there and `git-filter-repo` is not, and otherwise prints the one command that - installs it. It ends with `openRepoShape --preflight`. It never runs `sudo` - and installs nothing else. It exits 0 when the command is installed and the - preflight passed, 1 when the command is installed but the preflight did not - pass or the command does not stay on the release, and 2 when it installed - nothing. -- **An installed command that stays on its release.** `openRepoShape - --install` records the ref it runs at, `$OPENREPOSHAPE_REF`, in - `openRepoShape.ref` beside the command it places. The installed command then - fetches `setup.sh` at that ref, `setup.sh` clones the standard at it - (`--shape-ref`), and `--doctor` checks its clone out at it. In v1.0.0 an - installed command fetched main on every run unless `$OPENREPOSHAPE_REF` was - exported. `$OPENREPOSHAPE_REF` set for one run still overrides the record, - and `openRepoShape --version` prints the ref in force. An install with no - ref records nothing, removes any record already there, and follows main. The - installed command is byte for byte the release's own file. + installs it. It ends with `openRepoShape --preflight`. `install.sh` itself + never runs `sudo`; the preflight it ends with offers what is missing, each + behind a typed `yes` at your terminal. Every download is HTTPS only, + redirects included, at TLS 1.2 or newer. It exits 0 when the command is + installed and the preflight passed, 1 when the command is installed but the + preflight did not pass or the command does not stay on the release tag, and + 2 when it installed nothing. +- **An installed command that stays on its release tag.** With + `OPENREPOSHAPE_RECORD_REF=1`, which `install.sh` sets, `openRepoShape + --install` records `$OPENREPOSHAPE_REF` in `openRepoShape.ref` beside the + command it places. The installed command then fetches `setup.sh` at that + ref, `setup.sh` clones the standard at it (`--shape-ref`), and `--doctor` + checks its clone out at it, and so does a symlink to the command. In + v1.0.0 an installed command fetched main on every run unless + `$OPENREPOSHAPE_REF` was exported. `$OPENREPOSHAPE_REF` set for one run + still overrides the record, and `openRepoShape --version` prints the ref in + force. The installed command is byte for byte the release's own file. ## Install and upgrade @@ -46,20 +48,47 @@ command it installs stays on the release it was installed from (#564). - **By hand, at main:** the README's `gh api repos/opensoft/openRepoShape/contents/openRepoShape … | bash -s -- --install` line still installs main's command, which records no ref and follows main. -- **Windows:** `install.sh` is bash, for macOS, Linux and WSL2. Native - Windows still starts with `py setup-project.py`, as the README says. +- **Platforms:** macOS (the stock `/bin/bash` 3.2 is enough), Linux and WSL2. + Git Bash on Windows is supported too, but the tests of `install.sh` and of + the command skip on the Windows CI job, as the command's always have, so + no CI run exercises Git Bash; that is a known limit, followed up in v1.1.0. + Native Windows without Git Bash still starts with `py setup-project.py`, as + the README says. - **A project's pin** on the standard works as before. Follow [How to pin 1.0.0](v1.0.0.md#how-to-pin-100) with the v1.0.1 release commit. ## A script that runs `--install` -A script that runs `openRepoShape --install` with `$OPENREPOSHAPE_REF` -exported now pins the installed command to that value. If the value is not a -ref the command should run, unset it for the install. To keep the command on a -commit or a tag, set it to that commit or tag. +`openRepoShape --install` records a ref only when it is asked to, with +`OPENREPOSHAPE_RECORD_REF=1`. A script that exports `$OPENREPOSHAPE_REF` +around `--install` for its own reasons gets the install it got from v1.0.0: +nothing is recorded, a record already there is removed, and the command +follows main. To pin the command a script installs, set +`OPENREPOSHAPE_RECORD_REF=1` and set `$OPENREPOSHAPE_REF` to the commit or +tag the command should stay on. ## Known limits v1.0.1 changes nothing else, so its known limits are v1.0.0's: [Known limits in v1.0.0](v1.0.0.md#known-limits). + +## Before tagging (remove at tag time) + + + +The one-line install works only once `install.sh` is an asset of the release +GitHub marks as latest. Nothing in this repository attaches it: + +1. Publish v1.0.1 as the latest release, then attach the file from the + release commit: `gh release upload v1.0.1 install.sh -R opensoft/openRepoShape`. +2. Check the redirect: + `curl -fsSI https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh` + must answer with a `location:` naming the v1.0.1 asset, + `…/releases/download/v1.0.1/install.sh`. +3. Never attach `install.sh` to v1.0.0. Its tree has no `install.sh`, and its + `openRepoShape` cannot record a ref, so the installer would exit 1 over a + command that follows main. +4. Publish a later patch on an older line with `--latest=false` + (`gh release create … --latest=false`), so that `releases/latest`, and + with it the one-line install, keeps naming the newest line. diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index ff065cf7..3bd552a8 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -1184,25 +1184,27 @@ def test_readme_is_short_enough_to_be_read(): # shim's answer there changed from a clone of the upstream to a few # lines of its own, and a README that still promised the flag list # would send a reader to a command that no longer prints it. - # 2026-10-14: 1422 -> 1450 — the one-line install, v1.0.1 (#564). Brett + # 2026-10-14: 1422 -> 1444 — the one-line install, v1.0.1 (#564). Brett # Heap: "we need it to be more simple to install ... now and after, we # need a simple install process". Step 3 of the quick start now LEADS # with `curl -fsSL …/releases/latest/download/install.sh | bash`, and - # the worked example's "Install the command" with the same line: - # eleven lines for the two blocks and their lead-ins. Nine say what - # that line does that the old two did not, because each is a thing a - # reader would otherwise meet unannounced on their own machine: it - # reads the newest release from a redirect with no `gh`, it installs - # git-filter-repo with pipx or prints the command, it ends in the - # preflight, it never runs `sudo`, how to name a release, and its three - # exit statuses. Five say what PINNED means — the record beside the - # command, what reads it, and the variable that overrides it — because - # a command that quietly stops following main is a surprise to anyone - # who expected it to. Two re-introduce the existing lines as the - # by-hand way, at main, so neither is lost, and one is the Layout row - # for `install.sh`. - assert len(lines) <= 1450, ( - f"README.md is {len(lines)} lines; the cap is 1450") + # the worked example's "Install the command" with the same line: ten + # lines for the two blocks and the blank lines around them. Nine, step + # 3's paragraph and the blank after it, say what that line does that + # the old two did not, each a thing a reader would otherwise meet + # unannounced on their own machine: it needs no `gh` and no login, the + # command is PINNED to the release tag, where that is recorded and what + # it changes, it installs git-filter-repo with pipx or prints the + # command, it ends in the preflight, `install.sh` itself never runs + # `sudo` while each of the preflight's offers waits for a typed `yes`, + # how to name a release and how to upgrade. Two re-introduce the + # existing lines as the by-hand way, at main, so neither is lost, and + # one is the Layout row for `install.sh`. The review on #577 folded a + # separate "Pinned" paragraph into step 3's and left the exit statuses + # to `install.sh --help` and the release notes, which took the raise + # from 28 lines to 22. + assert len(lines) <= 1444, ( + f"README.md is {len(lines)} lines; the cap is 1444") #: #476: what each document that carries the first rule says an assistant does From 868c9bf886f58cc2901154e1c8c8ce729939c070 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 11:39:06 +0000 Subject: [PATCH 05/25] docs/release-notes/v1.0.1.md: the "Before tagging" steps are safe to follow in order (land, prove the squash tree or dispatch the three-OS run on it, tag it annotated, create the release with install.sh attached and --verify-tag, check the asset by redirect and by bytes), Git Bash is said to be meant to work but untested until #585, and only install.sh's own downloads are said to be HTTPS only (#564) The second fix round on #577 (RV577). The old checklist published before attaching install.sh, could tag main through `gh release create` with no tag, uploaded an unchecked working-tree file and never tested the release commit. Co-Authored-By: Claude Opus 5.5 --- docs/release-notes/v1.0.1.md | 82 +++++++++++++++++++++++++++--------- 1 file changed, 61 insertions(+), 21 deletions(-) diff --git a/docs/release-notes/v1.0.1.md b/docs/release-notes/v1.0.1.md index b165c364..9acb9734 100644 --- a/docs/release-notes/v1.0.1.md +++ b/docs/release-notes/v1.0.1.md @@ -20,8 +20,9 @@ command it installs stays on the release tag it was installed from (#564). `git-filter-repo` is not, and otherwise prints the one command that installs it. It ends with `openRepoShape --preflight`. `install.sh` itself never runs `sudo`; the preflight it ends with offers what is missing, each - behind a typed `yes` at your terminal. Every download is HTTPS only, - redirects included, at TLS 1.2 or newer. It exits 0 when the command is + behind a typed `yes` at your terminal. Every download `install.sh` makes + is HTTPS only, redirects included, at TLS 1.2 or newer. It exits 0 when the + command is installed and the preflight passed, 1 when the command is installed but the preflight did not pass or the command does not stay on the release tag, and 2 when it installed nothing. @@ -49,11 +50,10 @@ command it installs stays on the release tag it was installed from (#564). `gh api repos/opensoft/openRepoShape/contents/openRepoShape … | bash -s -- --install` line still installs main's command, which records no ref and follows main. - **Platforms:** macOS (the stock `/bin/bash` 3.2 is enough), Linux and WSL2. - Git Bash on Windows is supported too, but the tests of `install.sh` and of - the command skip on the Windows CI job, as the command's always have, so - no CI run exercises Git Bash; that is a known limit, followed up in v1.1.0. - Native Windows without Git Bash still starts with `py setup-project.py`, as - the README says. + Git Bash on Windows is meant to work, as #564 asks, but nothing has run it + yet: the launcher's and the installer's tests skip on the Windows job. + #585 (v1.1.0) runs them under Git Bash. Native Windows without Git Bash + still starts with `py setup-project.py`, as the README says. - **A project's pin** on the standard works as before. Follow [How to pin 1.0.0](v1.0.0.md#how-to-pin-100) with the v1.0.1 release commit. @@ -78,17 +78,57 @@ v1.0.1 changes nothing else, so its known limits are v1.0.0's: The one-line install works only once `install.sh` is an asset of the release -GitHub marks as latest. Nothing in this repository attaches it: - -1. Publish v1.0.1 as the latest release, then attach the file from the - release commit: `gh release upload v1.0.1 install.sh -R opensoft/openRepoShape`. -2. Check the redirect: - `curl -fsSI https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh` - must answer with a `location:` naming the v1.0.1 asset, - `…/releases/download/v1.0.1/install.sh`. -3. Never attach `install.sh` to v1.0.0. Its tree has no `install.sh`, and its - `openRepoShape` cannot record a ref, so the installer would exit 1 over a - command that follows main. -4. Publish a later patch on an older line with `--latest=false` - (`gh release create … --latest=false`), so that `releases/latest`, and - with it the one-line install, keeps naming the newest line. +GitHub marks as latest. Nothing in this repository attaches it, so the +release is made in this order, and each step is safe to stop after: + +1. **Land and prove the release commit.** Squash-land #577 into + `release/v1.0`; that squash commit is ``. Confirm that its + tree is the tree of the PR head that passed `tests`, `tests-macos` and + `tests-windows`: + + ```sh + git rev-parse ^{tree} ^{tree} + ``` + + The two are equal when `release/v1.0` had no other commits. If they + differ, run `gh workflow run tests.yml --ref release/v1.0 -R opensoft/openRepoShape` + and go on only when all three jobs pass on ``. A push to + `release/v1.0` runs no CI by itself: `tests.yml` runs on a push to main + only. +2. **Tag that commit, annotated, and push the tag:** + + ```sh + git tag -a v1.0.1 -m "openRepoShape v1.0.1" + git push origin refs/tags/v1.0.1 + ``` + +3. **Create the release with `install.sh` attached, in one command,** from a + checkout at `` and with this section cut from a copy of + these notes: + + ```sh + gh release create v1.0.1 install.sh --verify-tag --latest \ + --title "openRepoShape v1.0.1" --notes-file -R opensoft/openRepoShape + ``` + + `--verify-tag` refuses rather than tag whatever the default branch holds. + gh makes a draft, uploads `install.sh`, then publishes it, so the newest + release is never without the file. +4. **Check the published asset, by redirect and by bytes:** + + ```sh + curl -fsSI --proto '=https' --proto-redir '=https' \ + https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh + curl -fsSL --proto '=https' --proto-redir '=https' \ + https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh \ + | cmp - <(git show v1.0.1:install.sh) + ``` + + The first must answer with a `location:` naming the v1.0.1 asset, + `…/releases/download/v1.0.1/install.sh`; the second, run in bash or zsh, + must print nothing and exit 0. +5. **Never attach `install.sh` to v1.0.0.** Its tree has no `install.sh`, and + its `openRepoShape` cannot record a ref, so the installer would exit 1 over + a command that follows main. **Publish a later patch on an older line with + `--latest=false`**, so that `releases/latest`, and with it the one-line + install, keeps naming the newest line. From b2f0be333abaf7892ead4e4e1a02e7ce0a3a7daa Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 11:55:06 +0000 Subject: [PATCH 06/25] tests/readme-cap.d/564.txt: README.md's cap is 1449, raised by 22 for the one-line install in a ledger file of its own, as main raises it, rather than #577's inline 1422 -> 1444 on release/v1.0 (#564) README.md on main is 1427 lines at its cap (1422 + 5, #550); #564's lines take it to 1449. The reason is #577's, carried with its accounting: ten lines for the two install.sh blocks, nine for step 3's paragraph, two for the by-hand ways kept at main, one for the Layout row. Co-Authored-By: Claude Opus 5.5 --- tests/readme-cap.d/564.txt | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 tests/readme-cap.d/564.txt diff --git a/tests/readme-cap.d/564.txt b/tests/readme-cap.d/564.txt new file mode 100644 index 00000000..7a500425 --- /dev/null +++ b/tests/readme-cap.d/564.txt @@ -0,0 +1,21 @@ ++22 +2026-10-15 +Twenty-two lines for the one-line install, v1.0.1 (#564), carried into main +for v1.1.0 by Brett Heap's ruling 35. On release/v1.0 #577 raised the cap the +old way, 1422 -> 1444, in the test; here the same lines are this file. +Brett Heap: "we need it to be more simple to install ... now and after, we +need a simple install process". Step 3 of the quick start now LEADS with +`curl -fsSL …/releases/latest/download/install.sh | bash`, and the worked +example's "Install the command" with the same line: ten lines for the two +blocks and the blank lines around them (+10). Nine, step 3's paragraph and the +blank after it, say what that line does that the old two did not, each a thing +a reader would otherwise meet unannounced on their own machine: it needs no +`gh` and no login, the command is PINNED to the release tag, where that is +recorded and what it changes, it installs git-filter-repo with pipx or prints +the command, it ends in the preflight, `install.sh` itself never runs `sudo` +while each of the preflight's offers waits for a typed `yes`, how to name a +release and how to upgrade (+9). Two re-introduce the existing lines as the +by-hand way, at main, so neither is lost (+2), and one is the Layout row for +`install.sh` (+1). The review on #577 folded a separate "Pinned" paragraph +into step 3's and left the exit statuses to `install.sh --help` and the +release notes, which took the raise from 28 lines to 22. From 68b999f0cb3725ca2faf3184af0e6222e08ca709 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 11:55:48 +0000 Subject: [PATCH 07/25] =?UTF-8?q?docs/handbook.html:=20the=20consent=20rul?= =?UTF-8?q?es=20say=20every=20action=20the=20tool=20OFFERS=20is=20a=20type?= =?UTF-8?q?d=20yes=20and=20the=20one-line=20install=20asks=20nothing,=20pl?= =?UTF-8?q?acing=20the=20command=20and=20git-filter-repo=20through=20pipx?= =?UTF-8?q?=20without=20sudo,=20and=20the=20eight=20sections=20that=20read?= =?UTF-8?q?=20README.md=20=C2=A7=20Starting=20and=20=C2=A7=20Layout=20reco?= =?UTF-8?q?rd=20#564's=20README=20after=20a=20re-read=20(#564)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit What changed in README.md since the markers were set is #564 alone: `git diff f6eaab71 HEAD -- README.md` equals `git diff origin/main HEAD -- README.md` (the § Starting markers were set by #579, f6eaab71), and § Layout moved only by the install.sh row since #551 (5e59e3b3) set its marker. Re-read against the merged README, each section in full: - masthead: its fastest path is #577's install.sh line with an accurate summary (newest release, pinned, git-filter-repo where pipx can, ends in the preflight). No change. - start: carries #577's install step, the by-hand ways at main and "Each way installs". One statement was made false: "Every action that changes the machine or the account is a typed yes" stood a few lines below install.sh installing git-filter-repo with pipx unasked. It now scopes the rule to what the tool offers and says what the one-line install does, every fact quoted from README step 3. The machine: block paragraph, the preflight step, Run it and What setup.sh actually runs stay true as written. - partway (the WARNING), ci (private legs), legs (the --explain line), doublepin (the hand-edited pin), park (openRepoTools' line, which --install still prints when install.sh calls it): nothing #564 touched. No change. - layout: carries #577's install.sh row, in README's order; "standard library only" stays true (install.sh is bash, git-filter-repo a tool). No change. Markers: § Starting f0693d8a -> cc78d254 in masthead, start, partway, ci, legs, doublepin, park; § Layout c8be0af4 -> 94701efc in layout; both values as the hygiene check printed them. Co-Authored-By: Claude Opus 5.5 --- docs/handbook.html | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/docs/handbook.html b/docs/handbook.html index 37c4c26f..5c474cac 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -472,7 +472,7 @@ - + @@ -609,7 +609,7 @@

    The one thing to read before electing it

    - +

    Setup

    Start here

    @@ -961,8 +961,11 @@

    Joining a family

    Consent rules, on every platform

      -
    • Every action that changes the machine or the account is a typed yes, at - the same prompt: Type yes to continue:.
    • +
    • Every action the tool offers that changes the machine or the account is a typed + yes, at the same prompt: Type yes to continue:. The one-line + install asks nothing, because running it is your act: it places the command, installs + git-filter-repo with pipx where it can, and itself never runs + sudo.
    • --yes skips the scaffold question only. It does not answer an install offer, gh auth login or gh auth setup-git.
    • With no terminal attached there is no offer at all.
    • @@ -1034,7 +1037,7 @@

      What you have afterwards — the same on every platform

      - +

      Setup

      If it fails part-way

      @@ -1069,7 +1072,7 @@

      If it fails part-way

      - + @@ -1127,7 +1130,7 @@

      Reading private legs in CI

      - + @@ -1363,7 +1366,7 @@

      The InkRouter example

      - + @@ -1659,7 +1662,7 @@

      Is this repository compliant? openRepoShape --doctor

      - + @@ -1814,7 +1817,7 @@

      The degrade rule

      - +

      Reference

      Layout of this repository

      From c1e6b82f841ca81b1ef43c87e49ab61df10d8cd9 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 12:43:19 +0000 Subject: [PATCH 08/25] README.md: the NOTE says the preflight installs nothing without the typed yes, beside step 3's one-line install, which installs git-filter-repo with pipx where it can (#564) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Nothing is installed without that typed `yes`" became "The preflight installs nothing without that typed `yes`", and the rest of the sentence is kept. The longer clause re-wraps the NOTE's last eight lines into eight, so README.md stays at 1449, its ledger cap, and every line stays within 80 columns. `py setup-project.py --preflight` stays on one line, because test_the_windows_commands_name_files_that_exist reads `py ` across whitespace. Re-read against the new wording, the seven sections that read § Starting: start's consent rules ("Every action the tool offers ... is a typed yes", "The one-line install asks nothing ...", "no offer at all") and its four "The tool offers the rest" panels already put the typed yes on the preflight's offers; its `machine:` block line is about --install, and partway's "Nothing is created before the typed yes" quotes the WARNING, not the NOTE; masthead, ci, legs, doublepin and park render nothing from the NOTE. No page text changes. The markers move as the hygiene check printed: cc78d254 -> c9aebc08 in masthead, start, partway, ci, legs, doublepin, park. Co-Authored-By: Claude Opus 5.5 --- README.md | 16 ++++++++-------- docs/handbook.html | 14 +++++++------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 462a01f0..8d6539be 100644 --- a/README.md +++ b/README.md @@ -174,14 +174,14 @@ run the four steps below INSIDE the WSL shell. > opens with a preflight that names what is missing and, for `gh`, its login and > its credential helper, prints the command for THIS machine and asks once — > `Type yes to continue:`, the same words it asks everything else with — then -> runs it and carries on. Nothing is installed without that typed `yes`; `--yes` -> does not answer it, and with no terminal attached there is no offer at all. -> Homebrew is the one install never run for you: on a Mac without `brew` the -> preflight names and stops there. To check a machine and -> stop, creating nothing, run `openRepoShape --preflight` — `py setup-project.py -> --preflight` on Windows — which exits 0 when everything is present and 1 when -> it is not (`openRepoShape --preflight` exits 2 when `git` or a Python 3.9 or -> newer is missing). +> runs it and carries on. The preflight installs nothing without that typed +> `yes`; `--yes` does not answer it, and with no terminal attached there is no +> offer at all. Homebrew is the one install never run for you: on a Mac without +> `brew` the preflight names and stops there. To check a +> machine and stop, creating nothing, run `openRepoShape --preflight` — +> `py setup-project.py --preflight` on Windows — which exits 0 when everything +> is present and 1 when it is not (`openRepoShape --preflight` exits 2 when +> `git` or a Python 3.9 or newer is missing). **1. Install the GitHub CLI**, `gh`. The official instructions are at (and ): diff --git a/docs/handbook.html b/docs/handbook.html index 3f02d075..4a0ab624 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -472,7 +472,7 @@ - + @@ -609,7 +609,7 @@

      The one thing to read before electing it

      - +

      Setup

      Start here

      @@ -1037,7 +1037,7 @@

      What you have afterwards — the same on every platform

      - +

      Setup

      If it fails part-way

      @@ -1072,7 +1072,7 @@

      If it fails part-way

      - + @@ -1130,7 +1130,7 @@

      Reading private legs in CI

      - + @@ -1366,7 +1366,7 @@

      The InkRouter example

      - + @@ -1663,7 +1663,7 @@

      Is this repository compliant? openRepoShape --doctor

      - + From 5191f0eb9a74a8865550dd66cd5b7ec7ab0eae10 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 12:53:40 +0000 Subject: [PATCH 09/25] docs/handbook.html: the consent rules say install.sh asks nothing before its own two acts, and the preflight it ends with asks before each offer (#564) "The one-line install asks nothing" was not true as written: install.sh ends with `openRepoShape --preflight`, which asks `Type yes to continue:` before each offer, as step 3 above it and README.md:239-241 say. Only install.sh's own two acts, placing the command and installing git-filter-repo with pipx, go unasked. Found in #587's review. Handbook text only, in the same five lines: README.md is unchanged, so no marker moves, and the hygiene check asks for none. Co-Authored-By: Claude Opus 5.5 --- docs/handbook.html | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/handbook.html b/docs/handbook.html index 4a0ab624..0c3413df 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -962,10 +962,10 @@

      Joining a family

      Consent rules, on every platform

      • Every action the tool offers that changes the machine or the account is a typed - yes, at the same prompt: Type yes to continue:. The one-line - install asks nothing, because running it is your act: it places the command, installs - git-filter-repo with pipx where it can, and itself never runs - sudo.
      • + yes, at the same prompt: Type yes to continue:. install.sh + asks nothing before its own two acts, because running it is your act: it places the + command and installs git-filter-repo with pipx where it can, and itself + never runs sudo. The preflight it ends with asks before each offer, as above.
      • --yes skips the scaffold question only. It does not answer an install offer, gh auth login or gh auth setup-git.
      • With no terminal attached there is no offer at all.
      • From ba832315e20aa7ad2290ef1b76e2461894382c01 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:19:34 +0000 Subject: [PATCH 10/25] openRepoShape: --install refuses a target that is a symbolic link, such as the openRepoShape pipx places for the openreposhape package, places nothing, and names both ways out (#563) pipx places the package's commands in ~/.local/bin as links into the package's own environment, where `--install` places this command. pipx leaves a file it did not place alone, but `--install` copies with `cp`, which writes THROUGH a link: an install over pipx's `openRepoShape` would overwrite the package's command with the bash one and leave the link standing. So `--install` refuses such a target, a dangling link included, before anything is collected, placed or recorded, and names the two exits: keep the package's (`pipx upgrade openreposhape`) or remove the link (`pipx uninstall openreposhape`, or `rm`) and run it again. One loop in install_commands and one test after test_install_refuses_to_also_scaffold, kept apart from the rest of #563 so that it merges on its own. Approved with #563's design by Brett Heap at 2026-10-15T13:32:51Z ("approve all six for #563"). Co-Authored-By: Claude Opus 5.5 --- openRepoShape | 15 +++++++++++ tests/test_openreposhape_command.py | 40 +++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/openRepoShape b/openRepoShape index 2ab98fb7..60985343 100755 --- a/openRepoShape +++ b/openRepoShape @@ -391,6 +391,21 @@ install_commands() { { [ "$REF_NAMED" -eq 1 ] && [ "${OPENREPOSHAPE_RECORD_REF:-}" = 1 ]; }; then recording=1 fi + # A LINK IS NOT A PLACE TO COPY TO (#563). pipx places its + # `openRepoShape`, the openreposhape package's, as a symbolic link into + # the package's own environment, in this same `~/.local/bin`. `cp` + # writes THROUGH a link, so an install over one would overwrite the + # package's command with this file and leave the link where it was: + # two installers, one name, each breaking the other. Refused before + # anything is collected or placed, a dangling link included, and the + # refusal names both ways out. install.sh refuses the same target + # itself, for a release whose command predates this check. + for name in "${INSTALLABLES[@]}"; do + [ ! -L "$dir/$name" ] || + die "$dir/$name is a symbolic link, so --install will not copy this command through it: pipx places the openreposhape package's openRepoShape that way, and a copy written through the link would overwrite the package's command. Nothing was placed. Use one installer or the other: + keep the package's command, and update it with: pipx upgrade openreposhape + or remove the link (pipx uninstall openreposhape, or rm $dir/$name), then run this again." + done # THE RECORD SITS BESIDE THE FILE, symlinks followed, as it is read. record="$(resolved "$dir/openRepoShape")$RECORD_SUFFIX" # BOTH CHECKED BEFORE ANYTHING IS COLLECTED OR PLACED: a ref about to be diff --git a/tests/test_openreposhape_command.py b/tests/test_openreposhape_command.py index 5e495477..d52a7a8d 100644 --- a/tests/test_openreposhape_command.py +++ b/tests/test_openreposhape_command.py @@ -348,6 +348,46 @@ def test_install_refuses_to_also_scaffold(tmp_path): assert "--install takes no other arguments" in result.stderr +@pytest.mark.parametrize("dangling", [False, True], ids=["live", "dangling"]) +def test_install_refuses_a_target_that_is_a_symlink(tmp_path, dangling): + """pipx places the openreposhape package's `openRepoShape` as a LINK + into the package's own environment, in this same directory (#563). + + `cp` writes through a link, so an install over it would overwrite the + package's command and leave the link standing. It is refused before + anything is collected or placed, a dangling link too: the link stays as + it was, the file it names keeps its bytes, nothing is recorded, and the + refusal names both ways out. + """ + bin_dir = tmp_path / "bin" + bin_dir.mkdir() + packaged = tmp_path / "pipx" / "venvs" / "openreposhape" / "bin" / \ + "openRepoShape" + original = b"#!/usr/bin/env python3\n# the package's own command\n" + if not dangling: + packaged.parent.mkdir(parents=True) + packaged.write_bytes(original) + link = bin_dir / "openRepoShape" + link.symlink_to(packaged) + result = run_cmd("--install", home=tmp_path, + env={"OPENREPOSHAPE_BIN_DIR": str(bin_dir), + "OPENREPOSHAPE_REF": "v9.9.9", + "OPENREPOSHAPE_RECORD_REF": "1"}) + assert result.returncode == 2, result.stdout + result.stderr + said = " ".join(result.stderr.split()) + assert f"{link} is a symbolic link, so --install will not copy this " \ + "command through it" in said, said + assert "pipx upgrade openreposhape" in said + assert "pipx uninstall openreposhape" in said + assert link.is_symlink() and os.readlink(link) == str(packaged) + if not dangling: + assert packaged.read_bytes() == original + assert not packaged.with_name("openRepoShape.ref").exists() + assert sorted(path.name for path in bin_dir.iterdir()) == \ + ["openRepoShape"] + assert "placed" not in result.stdout + + # --- the real thing, offline ------------------------------------------------ def test_it_scaffolds_through_setup_sh(tmp_path): From 8cae45cb53c45934421c5c4262d187e7cf79d9e2 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:19:39 +0000 Subject: [PATCH 11/25] install.sh: the one-line installer refuses a target that is a symbolic link, such as the openRepoShape pipx places for the openreposhape package, before it asks the network anything (#563) The launcher's own `--install` now refuses a linked target, but install.sh installs a RELEASE's launcher, which may predate that check and copy through pipx's link into the package's environment. So install.sh checks the target itself, first, and exits 2 with nothing requested, fetched or placed, naming the same two exits: keep the package's (`pipx upgrade openreposhape`) or remove the link and run the line again. Its header's exit 2 lists the case. Kept apart from the launcher's commit and the rest of #563 so that it merges on its own. Approved with #563's design by Brett Heap at 2026-10-15T13:32:51Z ("approve all six for #563"). Co-Authored-By: Claude Opus 5.5 --- install.sh | 19 ++++++++++++++++--- tests/test_install_sh.py | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 3 deletions(-) diff --git a/install.sh b/install.sh index 453507bb..1c2c4e1f 100755 --- a/install.sh +++ b/install.sh @@ -38,7 +38,8 @@ # does not stay on the release tag (one older than v1.0.1 cannot record # it): the lines above say which; # 2 refused before anything was installed: a version that is not a ref, no -# release found, a fetch that failed, or a file that is not the command. +# release found, a fetch that failed, a file that is not the command, or +# a command already there as a symbolic link (pipx's, #563). # git-filter-repo never changes it: only adopting needs it, and # `adopt-project.py` refuses with its own install command where it is missing. # @@ -296,6 +297,20 @@ main() { command -v curl >/dev/null 2>&1 || die "curl is not installed, and this installer fetches with it. Install curl, or install by hand with: gh api repos/$REPO/contents/openRepoShape -H 'Accept: application/vnd.github.raw' | bash -s -- --install" + # A LINK IS NOT A PLACE TO INSTALL TO (#563), and this installer says so + # itself, before anything is asked of the network. pipx places the + # openreposhape package's `openRepoShape` as a symbolic link in this same + # directory, and the command's `--install` copies with `cp`, which writes + # through a link into the package's own environment. The command refuses + # such a target too, but only from the release that added the check: the + # release named here may be older, so its refusal cannot be relied on. + bin_dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" + target="$bin_dir/openRepoShape" + [ ! -L "$target" ] || + die "$target is a symbolic link, so nothing was installed: pipx places the openreposhape package's openRepoShape that way, and installing this one through the link would overwrite the package's command. Use one installer or the other: + keep the package's command, and update it with: pipx upgrade openreposhape + or remove the link (pipx uninstall openreposhape, or rm $target), then run this line again." + # 1. THE RELEASE. if [ -n "$version" ]; then checked_version "$version" "the version" @@ -339,8 +354,6 @@ main() { die "$launcher_url is not the openRepoShape command (its first line is not #!/usr/bin/env bash, or it has no INSTALLABLES line), so nothing was installed." say "fetch: $launcher_url" - bin_dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" - target="$bin_dir/openRepoShape" say "install: OPENREPOSHAPE_REF=$version OPENREPOSHAPE_RECORD_REF=1 openRepoShape --install" OPENREPOSHAPE_REF="$version" OPENREPOSHAPE_RECORD_REF=1 \ "${BASH:-bash}" "$WORKDIR/openRepoShape" --install || diff --git a/tests/test_install_sh.py b/tests/test_install_sh.py index 34c7de5a..f4fcd05b 100644 --- a/tests/test_install_sh.py +++ b/tests/test_install_sh.py @@ -628,6 +628,42 @@ def test_openreposhape_bin_dir_is_where_the_command_and_its_record_go(net): assert not default.exists() +@pytest.mark.parametrize("dangling", [False, True], ids=["live", "dangling"]) +def test_a_command_that_is_a_symlink_is_refused_before_any_request( + net, dangling): + """pipx places the openreposhape package's `openRepoShape` as a LINK + into the package's own environment, in this same directory (#563). + + The command's `--install` would copy through it, so install.sh refuses + the target ITSELF, before it asks the network anything: the release it + installs may carry a command older than that command's own check. The + link stays as it was, the file it names keeps its bytes, and nothing is + left in `$TMPDIR`. + """ + packaged = net.tmp_path / "pipx" / "venvs" / "openreposhape" / "bin" / \ + "openRepoShape" + original = b"#!/usr/bin/env python3\n# the package's own command\n" + if not dangling: + packaged.parent.mkdir(parents=True) + packaged.write_bytes(original) + net.bin.mkdir(parents=True) + link = net.bin / "openRepoShape" + link.symlink_to(packaged) + result = net.run() + assert result.returncode == 2, result.stdout + result.stderr + said = " ".join(result.stderr.split()) + assert f"{link} is a symbolic link, so nothing was installed" in said, said + assert "pipx upgrade openreposhape" in said + assert "pipx uninstall openreposhape" in said + assert net.calls() == [], net.calls() + assert link.is_symlink() and link.readlink() == packaged + if not dangling: + assert packaged.read_bytes() == original + assert sorted(path.name for path in net.bin.iterdir()) == \ + ["openRepoShape"] + assert list(net.temporary.iterdir()) == [] + + #: A curl that RUNS, in either script: tab-indented, alone or as the value #: of an assignment. Usage text and refusals carry curl lines too, indented #: with spaces, and are printed, never run. From 604fbda9bc2c7a04fe882345090232e47c3bcf59 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:24:41 +0000 Subject: [PATCH 12/25] pyproject.toml, src/openreposhape/: the openreposhape package gives openRepoShape, scaffold-project, adopt-project, update-shape and shape-doctor, each run from a full clone of the standard at the commit the package is stamped with, checked before every run (#563) Brett Heap's approval of #563's design, 2026-10-15T13:32:51Z: "approve all six for #563". This is its first pull request's package, as designed: - THIN. The wheel holds `openreposhape/` (standard library only, in SHIPPED) and the tree's `setup-project.py` byte for byte, beside the modules, so `openRepoShape --preflight` runs before git is installed. - THE STAMP. `src/openreposhape/_release.py` holds the version, tag, commit and digest; the committed one says 0.0.0 and unstamped, and pyproject.toml reads the version from it (`dynamic = ["version"]`). A stamp naming part of a release is refused. - THE CLONE. Every other run uses a full clone at the stamped 40-hex commit, made once per commit in the per-user cache (`$XDG_CACHE_HOME`, `~/Library/Caches`, `%LOCALAPPDATA%`) and kept. Before each run with no override it must be at the commit, carry no change, and digest to the stamp with its own `scripts/repo_shape.py` (`sorted-ls-tree-r-v1`). `$OPENREPOSHAPE_REF` overrides the commit for one run, with no digest checked; `$OPENREPOSHAPE_REPO` names a fork or a mirror. An unstamped build follows main and says so. Beside a checkout (`src/` of one), it uses that checkout as it stands, the launcher's `--doctor` rule. - THE COMMANDS. `openRepoShape` re-states the v1.0.1 launcher's modes, routing, one-mode rule, `--org` handling, usage, ref checks and `--doctor`'s run, and `--version` prints the version, tag, commit, digest and clone. `--install` refuses and names `pipx upgrade openreposhape`. A scaffold hands over to the clone's setup-project.py as setup.sh does (`--into` first, `OPENREPOSHAPE_SELF_BOOTSTRAP=1`), and a typed `--shape-ref` is that clone's ref, as it was setup.sh's. The other four run their file from the clone with every argument unchanged, in the directory typed in. - git-filter-repo==2.47.0 is the one dependency. Every child gets the package environment's scripts directory first on PATH whenever a git-filter-repo is there; no tool changes. `src/`, not the root, because the root holds `openRepoShape`, the launcher, and `openreposhape/` beside it is the same name on a case-insensitive disk (macOS's and Windows' default). The wheel still holds `openreposhape/`. Hatchling 1.27.0 builds it: its `force-include` places setup-project.py, and it is the newest hatchling that runs on Python 3.9. TESTS. tests/test_openreposhape_package.py imports the launcher's own cases from tests/test_openreposhape_command.py, unchanged, and an autouse fixture of its own points that module's `run_cmd` and `USAGE_LINES` at the Python command for each test it collects, so the same cases run against both, on all three OSes, Windows included, where the launcher's module skips. The four launcher cases that run its file (from stdin or by path) are re-stated for the Python command. Its own cases cover the stamp, pyproject, `--version`, the cache (once per commit, reused with the repository gone, nothing left by a failed clone), the refusals of a changed clone, a clone at another commit and a wrong digest, the override, main followed, the four tools' argument passthrough, the scaffold hand-over, `--install`'s refusal, a machine with no git, and the PATH every child gets, in process and through a real venv. tests/test_repo_hygiene.py holds every module of the package in SHIPPED, and the package's copy of openRepoTools' install line to the others. No existing test or skip condition changed. Co-Authored-By: Claude Opus 5.5 --- pyproject.toml | 82 +++ src/openreposhape/__init__.py | 16 + src/openreposhape/__main__.py | 9 + src/openreposhape/_release.py | 22 + src/openreposhape/cli.py | 547 ++++++++++++++++++ src/openreposhape/standard.py | 616 ++++++++++++++++++++ tests/test_openreposhape_package.py | 863 ++++++++++++++++++++++++++++ tests/test_repo_hygiene.py | 27 +- 8 files changed, 2180 insertions(+), 2 deletions(-) create mode 100644 pyproject.toml create mode 100644 src/openreposhape/__init__.py create mode 100644 src/openreposhape/__main__.py create mode 100644 src/openreposhape/_release.py create mode 100644 src/openreposhape/cli.py create mode 100644 src/openreposhape/standard.py create mode 100644 tests/test_openreposhape_package.py diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 00000000..e2b69c0c --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: Apache-2.0 +# +# THE openreposhape PACKAGE (#563): a SECOND WAY IN, NEVER A REQUIREMENT. +# Cloning this repository and running its tools with python3 and git alone +# keeps working, and every tool stays standard-library only +# (`tests/test_repo_hygiene.py` holds the package's own files to that too). +# Brett Heap approved the design on #563 at 2026-10-15T13:32:51Z ("approve +# all six for #563"). +# +# THIN: the wheel holds `src/openreposhape/` as `openreposhape/`, plus ONE +# file of this tree byte for byte, `setup-project.py`, so that `openRepoShape +# --preflight` runs before git is installed. Every other run uses a full +# clone of this repository at the commit the package was built from +# (`src/openreposhape/standard.py`), so a pin means what it always meant. +# +# `src/`, AND NOT THE ROOT, because the root already holds `openRepoShape`, +# the bash launcher: a directory `openreposhape/` beside it is the same name +# on a case-insensitive disk (macOS's and Windows' default), and a clone +# there could not hold both. + +[build-system] +# PINNED, as every tool this repository's CI installs is: a build that +# resolves the newest backend on the day it runs is a build whose bytes +# depend on that day. 1.27.0 is the newest hatchling that still runs on +# Python 3.9, the oldest Python this package supports (1.28.0 and later need +# 3.10). Hatchling, because `force-include` below puts `setup-project.py` +# into the package declaratively, and its wheels are reproducible from +# `SOURCE_DATE_EPOCH`. +requires = ["hatchling==1.27.0"] +build-backend = "hatchling.build" + +[project] +name = "openreposhape" +# THE TAG IS THE ONE SOURCE OF THE VERSION: the release build writes it into +# the stamp, `src/openreposhape/_release.py`, with the tag, the commit and +# the digest, and this reads it from there. The committed stamp says 0.0.0. +dynamic = ["version"] +description = "openRepoShape's commands, run from a clone of the standard at the release's own commit" +requires-python = ">=3.9" +license = "Apache-2.0" +license-files = ["LICENSE"] +authors = [{ name = "opensoft" }] +classifiers = [ + "Environment :: Console", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3 :: Only", + "Topic :: Software Development :: Version Control :: Git", +] +# THE ONE DEPENDENCY, and none of the tools': `adopt-project.py` extracts a +# leg's history with `git filter-repo`. The version CI runs and install.sh +# installs (`.github/workflows/tests.yml`, `install.sh`'s FILTER_REPO_PIN). +# pipx exposes only this package's commands, so each of them puts the +# package environment's scripts directory first on PATH for its children +# whenever a `git-filter-repo` is there (`standard.child_environment`). +dependencies = ["git-filter-repo==2.47.0"] + +[project.urls] +Homepage = "https://github.com/opensoft/openRepoShape" +Issues = "https://github.com/opensoft/openRepoShape/issues" + +# THE FIVE COMMANDS, each a native `.exe` on Windows. `openRepoShape` is the +# launcher's modes and flags; the other four run that tool from the clone, +# arguments unchanged. The names README, AGENTS.md and `docs/cli.md` already +# use, less `./` and `.py`. +[project.scripts] +openRepoShape = "openreposhape.cli:openreposhape" +scaffold-project = "openreposhape.cli:scaffold_project" +adopt-project = "openreposhape.cli:adopt_project" +update-shape = "openreposhape.cli:update_shape" +shape-doctor = "openreposhape.cli:shape_doctor" + +[tool.hatch.version] +path = "src/openreposhape/_release.py" + +[tool.hatch.build.targets.wheel] +packages = ["src/openreposhape"] + +# The release's own `setup-project.py`, beside the package's modules, which +# is where `standard.preflight_entry` looks for it. +[tool.hatch.build.targets.wheel.force-include] +"setup-project.py" = "openreposhape/setup-project.py" diff --git a/src/openreposhape/__init__.py b/src/openreposhape/__init__.py new file mode 100644 index 00000000..497fc755 --- /dev/null +++ b/src/openreposhape/__init__.py @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: Apache-2.0 +"""openreposhape: openRepoShape's commands, installed with pipx (#563). + +A SECOND WAY IN, NEVER A REQUIREMENT. Cloning this repository and running its +tools with `python3` and `git` keeps working exactly as it does without this +package, and every tool stays standard-library only. The package adds five +commands, `openRepoShape`, `scaffold-project`, `adopt-project`, +`update-shape` and `shape-doctor`, and one dependency, `git-filter-repo`, +which only `adopt-project.py` needs. + +IT IS THIN. It carries no tool, template or contract: `cli` runs them from a +full clone of this repository at the commit the package was built from +(`standard`), and `_release` is the stamp that names that commit. The one +file of the standard it does carry is `setup-project.py`, byte for byte, so +that `openRepoShape --preflight` runs on a machine with no git yet. +""" diff --git a/src/openreposhape/__main__.py b/src/openreposhape/__main__.py new file mode 100644 index 00000000..4db27e33 --- /dev/null +++ b/src/openreposhape/__main__.py @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: Apache-2.0 +"""`python -m openreposhape` is `openRepoShape` (#563): the same command, +for an interpreter whose scripts directory is not on PATH.""" + +import sys + +from .cli import openreposhape + +sys.exit(openreposhape()) diff --git a/src/openreposhape/_release.py b/src/openreposhape/_release.py new file mode 100644 index 00000000..b4b6198b --- /dev/null +++ b/src/openreposhape/_release.py @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: Apache-2.0 +"""The release this package was built from: its STAMP (#563). + +A release build GENERATES this file from the tag it builds, and nothing else +in the wheel differs from the tag's own tree. The four values are the +version (`v1.1.0` gives `1.1.0`), the tag, the tag's 40-hex commit, and the +`sorted-ls-tree-r-v1` digest of that commit's tree, the digest every pin this +standard writes carries (`scripts/repo_shape.py`, `tree_digest`). + +THE COMMITTED FILE IS UNSTAMPED: version 0.0.0 and no tag, commit or digest. +That is a build from a checkout rather than a release. It follows main, says +so in `openRepoShape --version`, and the release workflow refuses to publish +it. + +`pyproject.toml` reads VERSION from here (`dynamic = ["version"]`), so the +version is written down once. +""" + +VERSION = "0.0.0" +TAG = "" +COMMIT = "" +DIGEST = "" diff --git a/src/openreposhape/cli.py b/src/openreposhape/cli.py new file mode 100644 index 00000000..618f6f74 --- /dev/null +++ b/src/openreposhape/cli.py @@ -0,0 +1,547 @@ +# SPDX-License-Identifier: Apache-2.0 +"""The five commands the openreposhape package installs (#563). + +`openRepoShape` RE-STATES THE BASH LAUNCHER (v1.0.1's `openRepoShape`) IN +PYTHON: its four modes, ``, `--preflight`, `--doctor` and +`--help | --version`, its argument routing, one mode per run, `--org` from +the flag or `$OPENREPOSHAPE_ORG` or a prompt and never a default, its usage +text, its checks on a ref, and `--doctor`'s run of the standard's own +`shape-doctor.py`. What it does NOT re-state is fetching: the launcher +fetches `setup.sh` and clones the standard on every run, and this runs the +package's clone of it (`standard.prepare`). Nor `--install` and the ref it +records: pipx placed this command and the stamp names its commit, so +`--install` refuses and names `pipx upgrade openreposhape`. + +`scaffold-project`, `adopt-project`, `update-shape` and `shape-doctor` each +run that file from the clone, with every argument unchanged, in the +directory the command was typed in. They are the names README, AGENTS.md and +`docs/cli.md` already use, less `./` and `.py`, so `docs/cli.md` stays the +one reference for their flags. + +EVERY CHILD GETS `standard.child_environment()`: the package environment's +scripts directory first on PATH wherever the `git-filter-repo` it depends on +is there. + +PURE ASCII, like `setup-project.py`, because Windows PowerShell 5.1 renders a +console in the machine's ANSI code page: the usage text and every refusal +here arrive there intact. +""" + +from __future__ import annotations + +import os +import subprocess +import sys + +from . import standard +from .standard import Refusal + +#: THE USAGE, the launcher's own, less what pipx and the stamp replace: the +#: lines a person meets are the same lines, in the same columns, and the +#: `--install` line says where installing went. +USAGE = """\ +openRepoShape --install refused here: pipx upgrade openreposhape updates it +openRepoShape --preflight check this MACHINE and stop; creates nothing +openRepoShape --doctor [] [--json] [--placement-plan ] + diagnose a REPOSITORY against this standard +openRepoShape [--org ] [setup-project.py options] [-- ] +openRepoShape --help | --version + +Those lines are the whole command: check the machine you are on, diagnose a +repository, scaffold a project. Run with NO arguments it prints this and +exits 1, because a bare run has nothing to do. + + is the assembly-root name, ONE CamelCase token (e.g. Atlas); omit it +and setup asks, if a terminal is attached. `--org` comes from that flag, +else $OPENREPOSHAPE_ORG, else a prompt -- it is NEVER defaulted. Every other +argument reaches setup-project.py unchanged: --visibility, --elected-by, +--family, --into, --yes, --local-remote-dir, and anything after `--`, which +setup-project.py hands on to scaffold-project.py. `python3 setup-project.py +--help` lists them all, and so does `docs/cli.md` in the repository. The +project lands in the directory you ran this from, unless --into says where. + +This command came with the openreposhape package, which installed four more +beside it. Each runs that tool of the standard with every argument +unchanged, in the directory you typed it in; `docs/cli.md` is their +reference: + + scaffold-project adopt-project update-shape shape-doctor + +THE STANDARD THEY RUN is a full clone of openRepoShape at the commit this +package was built from, made once per commit in a cache in your home +directory and kept. Before every run the clone must be at that commit, carry +no change, and digest to the value the package carries (sorted-ls-tree-r-v1, +computed by the clone's own scripts/repo_shape.py), so a project scaffolded +from it pins exactly the release you installed. `--version` names the +release, its commit, its digest and the clone. A build that is not a release +follows main; installed from a checkout of the standard, it uses that +checkout as it stands. + + $OPENREPOSHAPE_REPO owner/name, URL or path to clone (default opensoft/openRepoShape) + $OPENREPOSHAPE_REF run at this branch, tag or commit instead, for one run; + no digest is checked then (`--shape-ref` on a + scaffold run does the same) + $OPENREPOSHAPE_ORG the organisation, when --org is not given + +`--install` places nothing here. pipx (or pip) placed this command, and +keeps it current: `pipx upgrade openreposhape`. The estate verbs are not +here either. `park` and `resume` -- commit, push and RECORD an estate's open +features, then rebuild them on the next workstation -- are +opensoft/openRepoTools', and its own installer places them: + + curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoTools/main/openRepoTools | bash -s -- --install + +`--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` +over (default `.`) and prints one row per check and ONE verdict: +`COMPLIANT`, `COMPLIANT, SHAPE BEHIND`, `DRIFTED`, `MISPLACED`, `INVALID`, +`NOT A SHAPE ROOT` or `CANNOT ANSWER`. Every finding names the command that +fixes it. Exit 0 compliant, 1 findings, 2 not a shape root, 3 when it cannot +answer (`CANNOT ANSWER`, no such , a `--placement-plan` it cannot +write); this command's own refusals exit 2. `--json` is the same report as +one JSON object. The doctor writes no file but the `--placement-plan ` +you name, at the path you give (a relative lands in the directory you +ran this from, even inside the repository), and moves nothing. + +It runs from the package's clone of the standard, which is KEPT for the NEXT +lines that name files in it; the report's last line names that clone (under +`--json`, on stderr, so stdout stays one JSON object). + +`--preflight` is the MACHINE check: setup-project.py's own, run from the copy +this package carries, so it answers on a machine with no git yet. +""" + +#: Why `--install` refuses, said where it does. +INSTALL_REFUSED = ( + "--install places nothing here: this openRepoShape is the " + "openreposhape package's, and the installer that placed it keeps it " + "current:\n" + " pipx upgrade openreposhape\n" + " The one-line install (install.sh) places the bash openRepoShape " + "instead. Use one or the other.") + +#: setup-project.py's value-taking flags, named so their VALUE is never +#: mistaken for the positional . `--shape-ref` is one of them, and +#: is also the ref of this run's clone (`Run.shape_ref`). +VALUE_FLAGS = ("--id", "--name", "--visibility", "--elected-by", "--family", + "--into", "--local-remote-dir", "--shape-ref") + + +def say(text: str = "") -> None: + print(text, flush=True) + + +def utf8_streams() -> None: + """`setup-project.py`'s guard: a child's output and a path in a refusal + travel through these streams, and the console's code page may not encode + them. A stream that cannot be reconfigured is left as it is.""" + for stream in (sys.stdout, sys.stderr): + try: + stream.reconfigure(errors="replace") + except (AttributeError, ValueError, OSError): + pass + + +def terminal_attached() -> bool: + """The launcher's `[ -t 0 ]`.""" + try: + return sys.stdin is not None and sys.stdin.isatty() + except (AttributeError, ValueError, OSError): + return False + + +def run_child(argv: list, env: dict, cwd: str | None = None) -> int: + """`argv`, its stdin, stdout and stderr this command's own, and its exit + status as a shell would report it. + + A CTRL-C REACHES THE CHILD TOO, which is in the same process group and + prints its own refusal, so the first one here waits for the child to + answer it, and only a second one kills it. A child killed by a signal + reports 128 + the signal, as bash does. + """ + sys.stdout.flush() + sys.stderr.flush() + try: + child = subprocess.Popen(argv, env=env, cwd=cwd) + except OSError as exc: + raise Refusal(f"could not start {argv[0]}: {exc}") from exc + interrupted = False + while True: + try: + status = child.wait() + break + except KeyboardInterrupt: + if interrupted: + child.kill() + interrupted = True + return 128 - status if status < 0 else status + + +def guarded(body, argv) -> int: + """`body(argv)`'s exit status, with a refusal printed the launcher's way + (`\\nREFUSED: ` on stderr) and its code returned.""" + utf8_streams() + args = sys.argv[1:] if argv is None else list(argv) + try: + return body(args) + except Refusal as refusal: + sys.stdout.flush() + print(f"\nREFUSED: {refusal}", file=sys.stderr, flush=True) + return refusal.code + except KeyboardInterrupt: + print("\nREFUSED: interrupted.", file=sys.stderr, flush=True) + return 130 + + +# --- openRepoShape ------------------------------------------------------------- + +class Run: + """What the arguments asked for: the launcher's variables, one each.""" + + def __init__(self) -> None: + self.mode = "run" + self.preflight = False + self.project = "" + self.org = "" + self.doctor_path = "" + self.doctor_path_given = False + self.doctor_args: list = [] + self.doctor_json = False + self.plan_file = "" + self.passthrough: list = [] + self.shape_ref = "" + + +def set_mode(run: Run, mode: str) -> None: + """ONE MODE PER RUN: two is a refusal, never last-wins (Copilot, PR + #96). A person who typed both meant one of them.""" + if run.mode not in ("run", mode): + raise Refusal( + f"--{run.mode} and --{mode} are two different runs; pass one.\n" + " --install places nothing here (pipx upgrade openreposhape " + "updates this command);\n" + " --doctor reads a repository and writes nothing into it but " + "a\n --placement-plan you name.") + run.mode = mode + + +def value_of(args: list, at: int, flag: str) -> str: + """The value after `flag`, which must be there and not be empty: the + launcher's `${2:? needs a value}`.""" + if at + 1 >= len(args) or args[at + 1] == "": + raise Refusal(f"{flag} needs a value") + return args[at + 1] + + +def take_placement_plan(run: Run, args: list, at: int) -> int: + """`--placement-plan ` or `--placement-plan=` after + `--doctor`; the index after what it took. The separate word is refused + when it begins with `-`, as argparse refuses it, and a second plan is a + refusal, never last-wins (#546).""" + before = run.plan_file + arg = args[at] + if arg.startswith("--placement-plan="): + run.plan_file = arg[len("--placement-plan="):] + at += 1 + else: + run.plan_file = args[at + 1] if at + 1 < len(args) else "" + at += 1 + if run.plan_file.startswith("-"): + run.plan_file = "" + elif at < len(args): + at += 1 + if not run.plan_file: + raise Refusal( + "--placement-plan needs the file to write the plan to: " + "--placement-plan , or --placement-plan= for a name " + "that begins with '-'.") + if before: + raise Refusal(f"two placement plans, '{before}' and " + f"'{run.plan_file}'. --doctor writes one: pass " + "--placement-plan once.") + run.doctor_args.append(f"--placement-plan={run.plan_file}") + return at + + +def parse(args: list) -> tuple: + """`(action, run)`: the launcher's argument loop, case for case. + `action` is "help" or "version" when one of those was met, which + answers at once, else "run".""" + run = Run() + at = 0 + while at < len(args): + arg = args[at] + if arg in ("-h", "--help"): + return "help", run + if arg == "--version": + return "version", run + if arg == "--install": + set_mode(run, "install") + at += 1 + elif arg == "--doctor": + # `--doctor []`: the optional path is eaten here, because + # it would otherwise be taken for a ; a word beginning + # with a dash is a flag, not the path. ONE path, however many + # `--doctor`s (#546). + set_mode(run, "doctor") + at += 1 + if (at < len(args) and not run.doctor_path_given + and not args[at].startswith("-")): + run.doctor_path, run.doctor_path_given = args[at], True + at += 1 + elif arg in ("--json", "--placement-plan") \ + or arg.startswith("--placement-plan="): + # THE DOCTOR'S TWO FLAGS ONLY AFTER `--doctor` (#500); anywhere + # else a lone flag for setup-project.py, which refuses both. + if run.mode != "doctor": + run.passthrough.append(arg) + at += 1 + elif arg == "--json": + run.doctor_args.append("--json") + run.doctor_json = True + at += 1 + else: + at = take_placement_plan(run, args, at) + elif arg == "--preflight": + # NAMED for the flag it sets: `--preflight` creates nothing, so + # the organisation and the are not asked for. + run.preflight = True + run.passthrough.append(arg) + at += 1 + elif arg in ("--org", "--project"): + setattr(run, arg[2:], value_of(args, at, arg)) + at += 2 + elif arg in VALUE_FLAGS: + value = value_of(args, at, arg) + run.passthrough += [arg, value] + if arg == "--shape-ref": + run.shape_ref = value + at += 2 + elif arg == "--": + run.passthrough += ["--", *args[at + 1:]] + break + elif arg.startswith("-"): + run.passthrough.append(arg) + at += 1 + elif run.mode == "doctor" and not run.doctor_path_given: + # After `--doctor` and its flags, the one path it has not had. + run.doctor_path, run.doctor_path_given = arg, True + at += 1 + elif run.project: + raise Refusal(f"two project names, '{run.project}' and '{arg}'. " + "One positional , then flags.") + else: + run.project = arg + at += 1 + return "run", run + + +def version() -> int: + """WHICH BYTES A RUN WOULD USE: the version, the tag, the commit, the + digest and the clone, with no network and no clone made. The ref in + force is in the first line's `( @ )`, as the launcher puts + it.""" + stamp = standard.read_stamp() + beside = standard.beside_checkout() + if beside is not None: + say(f"openRepoShape {stamp.version} (the checkout this package sits " + f"in: {beside})") + say(f" version {stamp.version}, unstamped: a build from a checkout " + "of the standard, not a release") + say(f" checkout {beside}: this package sits in it, so every run " + "uses it as it stands and clones nothing") + return 0 + repo = standard.repository() + standard.clone_url(repo) + wanted = standard.target(stamp) + cache = standard.cache_root() + say(f"openRepoShape {stamp.version} ({repo} @ {wanted.ref})") + if stamp.stamped: + say(f" version {stamp.version}") + say(f" tag {stamp.tag}") + say(f" commit {stamp.commit}") + say(f" digest {stamp.digest} ({standard.DIGEST_DEFINITION})") + else: + say(f" version {stamp.version}, unstamped: a build from a checkout, " + "not a release, so it follows main") + say(" tag none") + say(f" commit none: each run asks {repo} which commit main is at") + say(" digest none: no stamp to check a clone against") + if wanted.overridden: + say(f" ref {wanted.ref}, from {wanted.source}, for this run " + "instead of the commit above; no digest is checked") + if wanted.commit: + clone = cache / wanted.commit + made = "" if clone.is_dir() else \ + " (not cloned yet: the first run that needs it clones it)" + say(f" checkout {clone}{made}") + else: + say(f" checkout {cache}{os.sep}, one clone per commit, " + "kept") + return 0 + + +def say_the_standard(std: standard.Standard) -> None: + """One line, before a scaffold hands over, naming the standard it uses.""" + if std.beside: + say(f"openRepoShape: the standard is {std.describe()}.") + else: + say(f"openRepoShape: the standard is {std.describe()}, cloned at " + f"{std.root}.") + + +def doctor(run: Run) -> int: + """`--doctor`: RUN THE STANDARD'S OWN `shape-doctor.py`, never a + reimplementation, IN THE PERSON'S OWN DIRECTORY, so a relative `--root` + and a relative `--placement-plan` resolve where they typed (#500). + + THE CLONE IS KEPT AND SAID SO LAST, once the doctor has run: its NEXT + lines name files in it. On stderr under `--json`, so stdout stays one + object. Beside a checkout nothing was cloned, so nothing is said. + """ + std = standard.prepare() + script = std.root / "shape-doctor.py" + if not script.is_file(): + raise Refusal( + f"{std.root} has no shape-doctor.py, so there is nothing to run. " + f"That checkout is {std.describe()}.\n" + " Set $OPENREPOSHAPE_REF to a ref that carries it, or run this " + "from a checkout of the standard.") + status = run_child([sys.executable, str(script), "--root", + run.doctor_path or ".", *run.doctor_args], + standard.child_environment()) + if not std.beside: + print(f"the standard this run used is kept at {std.root}, for any " + "NEXT command above that names it; it is safe to delete once " + "you are done with them, and the next run that needs it clones " + "it again.", file=sys.stderr if run.doctor_json else sys.stdout, + flush=True) + return status + + +def scaffold(run: Run) -> int: + """`` and `--preflight`: setup-project.py, handed what the + launcher hands setup.sh. + + `--preflight` runs the copy this package carries, so it needs no git and + no clone; it examines the machine and stops. A scaffold runs the + clone's own, which pins that clone's commit, with `--into` this + directory FIRST (a later `--into` of the person's wins) and + `$OPENREPOSHAPE_SELF_BOOTSTRAP=1`: setup.sh's hand-over from a clone it + made, where there is no fork origin behind the person to read an + organisation from. + """ + if run.preflight: + entry = standard.preflight_entry() + return run_child([sys.executable, str(entry), *forwarded(run)], + standard.child_environment()) + std = standard.prepare(typed_ref=run.shape_ref) + entry = std.root / "setup-project.py" + if not entry.is_file(): + raise Refusal( + f"{std.root} has no setup-project.py, so there is nothing to run. " + f"That checkout is {std.describe()}.\n" + " Set $OPENREPOSHAPE_REF to a ref that carries it.") + say_the_standard(std) + env = standard.child_environment() + env["OPENREPOSHAPE_SELF_BOOTSTRAP"] = "1" + return run_child([sys.executable, str(entry), "--into", os.getcwd(), + *forwarded(run)], env) + + +def forwarded(run: Run) -> list: + """`--org`, `--project` and every other argument, as the launcher's + `ARGS`. An organisation typed alongside `--preflight` still travels: the + preflight ignores it, and a flag this command ate would be a flag the + person has to type twice to find out about.""" + args = [] + if run.org: + args += ["--org", run.org] + if run.project: + args += ["--project", run.project] + return args + run.passthrough + + +def main_openreposhape(args: list) -> int: + action, run = parse(args) + if action == "help": + sys.stdout.write(USAGE) + return 0 + if action == "version": + return version() + if run.mode == "doctor": + # ONE PATH AND THE DOCTOR'S TWO FLAGS, NOTHING ELSE: every other + # argument is about a run that CREATES something. + if run.project or run.org or run.passthrough: + raise Refusal( + "--doctor takes a path, --json and --placement-plan , " + "and nothing else; it reads a repository and writes nothing " + "into it but that .\n" + " openRepoShape --doctor [] [--json] " + "[--placement-plan ]\n" + " To scaffold, drop --doctor. To check this machine, run: " + "openRepoShape --preflight") + return doctor(run) + if run.mode == "install": + raise Refusal(INSTALL_REFUSED) + # A BARE RUN HAS NOTHING TO DO: the usage, and exit 1. Any argument at + # all keeps the scaffold path and the refusal it earned. + if not (run.preflight or run.project or run.org or run.passthrough): + sys.stdout.write(USAGE) + return 1 + if not run.preflight: + # NEVER a default organisation: a wrong one creates three + # repositories in somebody else's namespace. + run.org = run.org or os.environ.get("OPENREPOSHAPE_ORG", "") + if not run.org and terminal_attached(): + try: + run.org = input("organisation to scaffold into: ").strip() + except EOFError: + run.org = "" + if not run.org: + raise Refusal("no organisation to scaffold into. Re-run with " + "--org , or set $OPENREPOSHAPE_ORG.") + if not run.project and not terminal_attached(): + sys.stderr.write(USAGE) + raise Refusal("no given, and no terminal to ask on.") + return scaffold(run) + + +def openreposhape(argv=None) -> int: + """The `openRepoShape` console script.""" + return guarded(main_openreposhape, argv) + + +# --- the four tools, passed through --------------------------------------------- + +def run_tool(filename: str, args: list) -> int: + """`/ `, every argument unchanged, in the + directory the command was typed in, and its exit status.""" + std = standard.prepare() + script = std.root / filename + if not script.is_file(): + raise Refusal( + f"{std.root} has no {filename}, so there is nothing to run. That " + f"checkout is {std.describe()}.\n" + " Set $OPENREPOSHAPE_REF to a ref that carries it.") + return run_child([sys.executable, str(script), *args], + standard.child_environment()) + + +def scaffold_project(argv=None) -> int: + """The `scaffold-project` console script: `scaffold-project.py`.""" + return guarded(lambda args: run_tool("scaffold-project.py", args), argv) + + +def adopt_project(argv=None) -> int: + """The `adopt-project` console script: `adopt-project.py`.""" + return guarded(lambda args: run_tool("adopt-project.py", args), argv) + + +def update_shape(argv=None) -> int: + """The `update-shape` console script: `update-shape.py`.""" + return guarded(lambda args: run_tool("update-shape.py", args), argv) + + +def shape_doctor(argv=None) -> int: + """The `shape-doctor` console script: `shape-doctor.py`.""" + return guarded(lambda args: run_tool("shape-doctor.py", args), argv) diff --git a/src/openreposhape/standard.py b/src/openreposhape/standard.py new file mode 100644 index 00000000..9fde4f3f --- /dev/null +++ b/src/openreposhape/standard.py @@ -0,0 +1,616 @@ +# SPDX-License-Identifier: Apache-2.0 +"""The standard a packaged command runs: a clone of openRepoShape (#563). + +A WHEEL HAS NO CHECKOUT, AND THE TOOLS NEED ONE. `scaffold-project.py` pins +the commit it runs from (`git rev-parse HEAD` in its own checkout), +`update-shape.py` and the doctor's currency row read the standard's history, +and every tool finds its templates beside itself. So the package carries no +tool: every command but `--preflight` runs the tools from a FULL clone of +this repository at the commit the package was built from, the commit its +stamp (`_release`) names. + +ONE CLONE PER COMMIT, IN A PER-USER CACHE, KEPT. The first run that needs the +standard clones it into `/` and every later run at that commit +uses it as it is. It is kept because the doctor's NEXT lines name files in it +(#500), and it is a full clone because the doctor's currency row compares +commits (`--depth 1` holds one). + +CHECKED BEFORE EVERY RUN, WHEN NOTHING OVERRIDES THE STAMP. The clone must be +at the stamped commit, carry no change, and digest to the stamped value. The +digest is computed by the clone's OWN `scripts/repo_shape.py` +(`tree_digest`, `sorted-ls-tree-r-v1`), never by a copy in this package, so +it is the digest the scaffold then pins. + +THE REF IS THE STAMPED 40-HEX COMMIT, NEVER THE TAG, because a tag can be +moved. `$OPENREPOSHAPE_REF`, set for one run, names another branch, tag or +commit, as it does for the bash launcher, and then no digest is checked: +the stamp's digest is the stamped commit's. `$OPENREPOSHAPE_REPO` names a +fork or a mirror to clone from. A build with no stamp follows main. + +BESIDE A CHECKOUT, THAT CHECKOUT ANSWERS. Run from the `src/` of a checkout +of this standard (an editable install, or the test suite), the package uses +the checkout it sits in, as it stands, and clones nothing: the launcher's +`--doctor` rule for a copy of itself that sits in a checkout. + +STANDARD LIBRARY ONLY, like every tool here: `tests/test_repo_hygiene.py` +holds this file to it. +""" + +from __future__ import annotations + +import os +import re +import shutil +import stat +import subprocess +import sys +import sysconfig +import tempfile +import time +from pathlib import Path + +from . import _release + +#: Where the standard comes from when `$OPENREPOSHAPE_REPO` is unset or empty. +DEFAULT_REPO = "opensoft/openRepoShape" + +#: What the stamped digest means: `scripts/repo_shape.py`'s +#: `TREE_DIGEST_DEFINITION`, the definition every pin this standard writes +#: records. A clone whose own module says otherwise is not compared. +DIGEST_DEFINITION = "sorted-ls-tree-r-v1" + +#: The one dependency, and the only program whose presence moves PATH. +FILTER_REPO = "git-filter-repo" + +#: A stamped commit and digest are lowercase hex of exactly these lengths. +COMMIT_RE = re.compile(r"[0-9a-f]{40}") +ANY_CASE_COMMIT_RE = re.compile(r"[0-9a-fA-F]{40}") +DIGEST_RE = re.compile(r"[0-9a-f]{64}") + +#: The launcher's ref checks (`ref_fault`), in its order: a control +#: character, then the shape. +CONTROL_RE = re.compile(r"[\x00-\x1f\x7f-\x9f]") +REF_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._/-]*") + +#: A Windows path with a drive letter, which is a path to clone and never an +#: owner/name on github.com. +DRIVE_RE = re.compile(r"[A-Za-z]:[\\/]") + +#: What makes a directory a checkout of this standard for the BESIDE rule: +#: the three tools a run reaches for and the naming policy `setup-project.py` +#: itself asks for (`is_shape_checkout`). +CHECKOUT_MARKERS = ("setup-project.py", "scaffold-project.py", + "shape-doctor.py", "contracts/repository-naming.yaml") + +#: The way out of every refusal about a cached clone. The cache is this +#: command's, and a clone it made again is the same clone. +CLONE_AFRESH = "Delete it and run this again: the next run clones it afresh." + +#: Run by the CLONE'S OWN interpreter path with `-I`: its `scripts/` first on +#: `sys.path`, and its `repo_shape` answers, never a copy carried here. +DIGEST_PROGRAM = """\ +import sys +from pathlib import Path +sys.path.insert(0, sys.argv[1]) +import repo_shape +print(repo_shape.TREE_DIGEST_DEFINITION) +print(repo_shape.tree_digest(Path(sys.argv[2]), sys.argv[3])) +""" + + +class Refusal(Exception): + """A refusal and its exit code, printed as `REFUSED: ` by the + command that raised it, as the launcher's `die` prints one.""" + + def __init__(self, message: str, code: int = 2) -> None: + super().__init__(message) + self.code = code + + +def note(text: str) -> None: + """A line about what this command is doing, on stderr, so that a + `--json` report on stdout stays one object.""" + print(text, file=sys.stderr, flush=True) + + +# --- the stamp --------------------------------------------------------------- + +class Stamp: + """`_release`, read and checked: the version, the tag, the commit and the + digest this package was built from, or a version and three empty strings + for a build with no stamp.""" + + def __init__(self, version: str, tag: str, commit: str, + digest: str) -> None: + self.version = version + self.tag = tag + self.commit = commit + self.digest = digest + + @property + def stamped(self) -> bool: + return bool(self.commit) + + +def read_stamp(module=_release) -> Stamp: + """The stamp, ALL FOUR VALUES OR NONE OF THEM. + + A stamp that names part of a release is a broken build: a commit with no + digest would run unchecked, and a digest with no commit names nothing. + It is refused, never read as the half it holds. + """ + values = [getattr(module, name, "") for name in + ("VERSION", "TAG", "COMMIT", "DIGEST")] + if not all(isinstance(value, str) for value in values): + values = ["", "?", "?", "?"] + version, tag, commit, digest = values + if not (tag or commit or digest): + return Stamp(version or "0.0.0", "", "", "") + if not (version and version != "0.0.0" and tag + and COMMIT_RE.fullmatch(commit) and DIGEST_RE.fullmatch(digest)): + raise Refusal( + "this package's stamp (openreposhape/_release.py) names part of " + "a release and not all of it: a release carries its version, its " + "tag, a 40-hex commit and a 64-hex digest, together. This build " + "is broken. Install a release: pipx install --force openreposhape") + return Stamp(version, tag, commit, digest) + + +# --- where the standard comes from --------------------------------------------- + +def repository() -> str: + """`$OPENREPOSHAPE_REPO`, or opensoft/openRepoShape where it is unset or + empty: the shell's `${VAR:-default}`, as setup.sh and the launcher read + it.""" + return os.environ.get("OPENREPOSHAPE_REPO") or DEFAULT_REPO + + +def clone_url(repo: str) -> str: + """What `git clone` is handed for `repo`: an owner/name becomes its + github.com URL, and a URL, an scp-style address or a path on disk goes + through untouched (the launcher's `shape_clone_url`, and a Windows + drive path besides). + + REFUSED, NOT ECHOED, when it starts with `-` or carries a control + character: the launcher's guard, in its words. The `--` before it on + every git command line is defence in depth, not a substitute. + """ + after_at = repo.split("@", 1)[1] if "@" in repo else "" + if ("://" in repo or ":" in after_at or repo.startswith(("/", ".")) + or os.path.isabs(repo) or DRIVE_RE.match(repo)): + url = repo + else: + url = f"https://github.com/{repo}.git" + if url.startswith("-") or CONTROL_RE.search(url): + raise Refusal( + "OPENREPOSHAPE_REPO is not a value this tool will put on a `git " + "clone` command line: it starts with '-' or carries a control " + "character. Set it to a URL or a path.") + return url + + +def ref_fault(ref: str, what: str) -> str: + """What is wrong with `ref`, or "" when nothing is: the launcher's + `ref_fault`, in its order and its words. LENGTH first, so an over-long + value is refused without being echoed; a CONTROL CHARACTER next, because + a newline in it could forge a line of output; and only then the SHAPE, + which is safe to echo back because it has passed both.""" + if not ref: + return (f"{what} is empty, which is not a branch, tag or commit this " + "tool will pass to `git checkout`.") + if len(ref) > 255: + return (f"{what} is {len(ref)} characters long; a branch, tag or " + "commit this tool will pass to `git checkout` is at most 255 " + "characters.") + if CONTROL_RE.search(ref): + return (f"{what} carries a control character, which is not a " + "branch, tag or commit this tool will pass to `git " + "checkout`.") + if not REF_RE.fullmatch(ref) or ".." in ref or ref.endswith(".lock"): + return (f"{what} is '{ref}', which is not a branch, tag or commit " + "this tool will pass to `git checkout`: a ref starts with a " + "letter or digit, uses only letters, digits, '.', '_', '/', " + "'-', has no '..' and does not end in '.lock'.") + return "" + + +def checked_ref(ref: str, what: str = "$OPENREPOSHAPE_REF") -> str: + """`ref`, or a refusal saying what is wrong with it.""" + fault = ref_fault(ref, what) + if fault: + raise Refusal(fault) + return ref + + +class Target: + """The commit a run asks for: `ref` as a person would read it, the + 40-hex `commit` where that is already known, the `digest` to check the + clone against (the stamp's, and only with no override), and `source`, + which of the four ways it was named.""" + + def __init__(self, ref: str, commit: str, digest: str, + source: str) -> None: + self.ref = ref + self.commit = commit + self.digest = digest + self.source = source + + @property + def overridden(self) -> bool: + return self.source in ("--shape-ref", "$OPENREPOSHAPE_REF") + + +def target(stamp: Stamp, typed_ref: str = "") -> Target: + """WHICH COMMIT, first match wins: a `--shape-ref` typed on a scaffold + run (the launcher hands setup.sh its own ref FIRST, so a typed one is the + last and wins), `$OPENREPOSHAPE_REF` set for this run, the stamped + commit, and with none of those main. Each named ref is checked here, + before anything reaches git.""" + for ref, source in ((typed_ref, "--shape-ref"), + (os.environ.get("OPENREPOSHAPE_REF", ""), + "$OPENREPOSHAPE_REF")): + if ref: + checked_ref(ref, source) + commit = ref.lower() if ANY_CASE_COMMIT_RE.fullmatch(ref) else "" + return Target(ref, commit, "", source) + if stamp.stamped: + return Target(stamp.commit, stamp.commit, stamp.digest, "stamp") + return Target("main", "", "", "main") + + +# --- the cache ----------------------------------------------------------------- + +def cache_root() -> Path: + """Where the clones live, one directory per commit: the platform's own + per-user cache. `%LOCALAPPDATA%` on Windows, `~/Library/Caches` on macOS, + and `$XDG_CACHE_HOME` (else `~/.cache`) everywhere else; a relative + `$XDG_CACHE_HOME` is ignored, as the XDG specification says.""" + try: + if os.name == "nt": + base = os.environ.get("LOCALAPPDATA") or str( + Path.home() / "AppData" / "Local") + return Path(base) / "openreposhape" / "checkouts" + if sys.platform == "darwin": + return (Path.home() / "Library" / "Caches" / "openreposhape" + / "checkouts") + base = os.environ.get("XDG_CACHE_HOME", "") + if not os.path.isabs(base): + base = str(Path.home() / ".cache") + return Path(base) / "openreposhape" / "checkouts" + except RuntimeError as exc: + raise Refusal(f"could not find your home directory, where this " + f"command keeps its clone of the standard: {exc}. Set " + f"HOME.") from exc + + +def remove_tree(path: Path) -> None: + """`shutil.rmtree` that also removes a clone on Windows, where git's + object files are read-only and an unlink of one is refused until the bit + is cleared. `onerror`, not `onexc`, which is 3.12 and newer.""" + def clear_readonly(func, target_path, _exc): + os.chmod(target_path, stat.S_IWRITE) + func(target_path) + + shutil.rmtree(str(path), onerror=clear_readonly) + + +# --- the environment every child gets ------------------------------------------ + +def filter_repo_in(directory: str) -> bool: + """Whether `directory` holds a `git-filter-repo` a command line would + start: the bare name, or on Windows any name `PATHEXT` adds an extension + to, which is how pip writes it (`git-filter-repo.exe`).""" + names = [FILTER_REPO] + if os.name == "nt": + extensions = os.environ.get("PATHEXT", ".COM;.EXE;.BAT;.CMD") + names += [FILTER_REPO + ext for ext in extensions.split(os.pathsep) + if ext] + for name in names: + candidate = os.path.join(directory, name) + if os.path.isfile(candidate) and os.access(candidate, os.X_OK): + return True + return False + + +def child_environment(environ=None) -> dict: + """This process's environment for a child, with the package's own + scripts directory FIRST ON PATH WHENEVER A `git-filter-repo` IS THERE. + + pipx exposes the package's own commands and not its dependencies', so the + `git-filter-repo` the package depends on sits in the package + environment's scripts directory, which is on nobody's PATH. adopt finds + it with `shutil.which` and runs `git filter-repo`, which git finds on + PATH; the doctor's machine row looks for it the same way. Putting that + directory first for the children this command starts is the whole of the + fix, and no tool changes for it: where the directory holds none, PATH is + left exactly as it was, and adopt still refuses with + `filter-repo-missing` where the tool is absent. + """ + env = dict(os.environ if environ is None else environ) + scripts = sysconfig.get_path("scripts") + if scripts and filter_repo_in(scripts): + current = env.get("PATH", "") + env["PATH"] = scripts + (os.pathsep + current if current else "") + return env + + +def git(args: list, env: dict) -> subprocess.CompletedProcess: + """`git `, its output captured and decoded as UTF-8.""" + return subprocess.run(["git", *args], env=env, capture_output=True, + encoding="utf-8", errors="replace", check=False) + + +def last_line(text: str) -> str: + lines = [line for line in (text or "").splitlines() if line.strip()] + return lines[-1].strip() if lines else "(nothing on stderr)" + + +# --- the checkout a run uses --------------------------------------------------- + +class Standard: + """The standard a run uses: its `root`, the `commit` it is at, which + `repo` it came from and which `target` named it; or, `beside`, the + checkout this package sits in, used as it stands.""" + + def __init__(self, root: Path, commit: str = "", repo: str = "", + wanted: Target | None = None, beside: bool = False) -> None: + self.root = root + self.commit = commit + self.repo = repo + self.target = wanted + self.beside = beside + + def describe(self) -> str: + """Which standard this is, in a refusal or a progress line.""" + if self.beside: + return f"the checkout this package sits in, {self.root}" + named = self.target.ref if self.target else self.commit + if named == self.commit: + return f"{self.repo} at {self.commit}" + return f"{self.repo} at {named} ({self.commit})" + + +def beside_checkout() -> Path | None: + """The checkout of the standard this package sits in, or None. + + `src/openreposhape/` in a checkout: the package's parent is `src`, and + the directory above that holds the files a run needs. An installed + package sits in a `site-packages`, which holds none of them. + """ + here = Path(__file__).resolve().parent + root = here.parent.parent + if here.parent.name == "src" and all( + (root / marker).is_file() for marker in CHECKOUT_MARKERS): + return root + return None + + +def preflight_entry() -> Path: + """The `setup-project.py` `--preflight` runs: the copy the wheel carries + beside this file, byte for byte the release's own, or in a checkout the + one at its root. Either way no git is needed to reach it.""" + packaged = Path(__file__).resolve().parent / "setup-project.py" + if packaged.is_file(): + return packaged + beside = beside_checkout() + if beside is not None: + return beside / "setup-project.py" + raise Refusal("this package carries no setup-project.py, so there is no " + "preflight to run. The install is incomplete: pipx install " + "--force openreposhape") + + +def remote_commit(url: str, repo: str, ref: str, env: dict) -> str: + """The commit `ref` names on the remote, read with `git ls-remote`, or + "" when no ref has exactly that name (an abbreviated commit, say, which + only a clone can resolve). + + A TAG FIRST, then a branch, then the name as given (`HEAD`, or a full + `refs/...`): `git checkout ` in a fresh clone resolves it in that + order, and the two must not disagree. An annotated tag is read through + to its commit (`^{}`). + """ + proc = git(["ls-remote", "--", url, ref], env) + if proc.returncode != 0: + raise Refusal( + f"could not ask {repo} which commit {ref} is (`git ls-remote` " + f"exited {proc.returncode}: {last_line(proc.stderr)}). This " + "command runs the standard from a clone of it: make the network " + "work, or set $OPENREPOSHAPE_REPO to a mirror you can reach.") + found = {} + for line in proc.stdout.splitlines(): + sha, _, name = line.partition("\t") + found[name.strip()] = sha.strip().lower() + for name in (f"refs/tags/{ref}^{{}}", f"refs/tags/{ref}", + f"refs/heads/{ref}", ref): + if COMMIT_RE.fullmatch(found.get(name, "")): + return found[name] + return "" + + +def move_into_place(incoming: Path, final: Path) -> None: + """`os.rename`, and on Windows a few more tries a moment apart. A + scanner that opens a file git has just written holds it for a moment, + and Windows refuses to rename a directory while it does; an existing + `final` is never retried, because another run placed it first.""" + tries = 10 if os.name == "nt" else 1 + for attempt in range(tries): + try: + os.rename(str(incoming), str(final)) + return + except PermissionError: + if attempt == tries - 1 or final.exists(): + raise + time.sleep(0.2) + + +def clone_into(cache: Path, url: str, repo: str, wanted: Target, + commit: str, env: dict) -> Path: + """A full clone of `url` at `commit` (or at `wanted.ref`, resolved by the + clone where `commit` is not known), placed at `/`. + + CLONED BESIDE ITS PLACE AND MOVED IN WHOLE. A clone that failed half way, + or a run interrupted, leaves nothing at `/` for the next + run to mistake for a clone; and of two runs cloning at once, the one that + moves second uses the first one's and discards its own. + """ + try: + cache.mkdir(parents=True, exist_ok=True) + incoming = Path(tempfile.mkdtemp(prefix=".incoming-", dir=str(cache))) + except OSError as exc: + raise Refusal(f"could not make a directory under {cache}, where " + f"this command keeps its clones of the standard: " + f"{exc}") from exc + placed = False + try: + named = f"commit {commit}" if commit else wanted.ref + note(f"openRepoShape: cloning {repo} into {cache}, once for {named}; " + "every later run at that commit uses this clone.") + cloned = git(["clone", "--quiet", "--no-checkout", "--", url, + str(incoming)], env) + if cloned.returncode != 0: + raise Refusal( + f"could not clone {repo} ({url}): `git clone` exited " + f"{cloned.returncode}: {last_line(cloned.stderr)}. This " + "command runs the standard from a clone of it: make the " + "clone work, or set $OPENREPOSHAPE_REPO to a mirror.") + # ` --`, never `-- `: before the `--` git reads it as a + # commit, after it as a pathspec. A commit already known is checked + # out detached; a name only the clone can resolve is checked out as + # git resolves it. + checkout = (["checkout", "--quiet", "--detach", commit] if commit + else ["checkout", "--quiet", wanted.ref, "--"]) + checked_out = git(["-C", str(incoming), *checkout], env) + if checked_out.returncode != 0: + raise Refusal( + f"cloned {repo} but could not check out " + f"'{commit or wanted.ref}': {last_line(checked_out.stderr)}. " + "Set $OPENREPOSHAPE_REF to a ref that repository carries, or " + "$OPENREPOSHAPE_REPO to one that carries this one.") + if not commit: + head = git(["-C", str(incoming), "rev-parse", "HEAD"], env) + commit = head.stdout.strip().lower() + if head.returncode != 0 or not COMMIT_RE.fullmatch(commit): + raise Refusal(f"cloned {repo} at '{wanted.ref}', but git " + f"could not say which commit that is: " + f"{last_line(head.stderr)}") + final = cache / commit + try: + move_into_place(incoming, final) + placed = True + except OSError as exc: + if not final.is_dir(): + raise Refusal(f"could not move the new clone to {final}: " + f"{exc}") from exc + return final + finally: + if not placed and incoming.exists(): + remove_tree(incoming) + + +def tree_digest(root: Path, commit: str, env: dict) -> tuple: + """`(definition, digest)` of `commit`'s tree, as the clone's own + `scripts/repo_shape.py` computes them. `-I`, so nothing in this process's + environment or user site can stand in for that module, and `-B`, so the + clone gains no bytecode.""" + module = root / "scripts" / "repo_shape.py" + if not module.is_file(): + raise Refusal( + f"{root} has no scripts/repo_shape.py, so its tree digest cannot " + "be computed the way every pin this standard writes computes " + f"it, and nothing was run from it. {CLONE_AFRESH}") + proc = subprocess.run( + [sys.executable, "-I", "-B", "-c", DIGEST_PROGRAM, + str(root / "scripts"), str(root), commit], + env=env, capture_output=True, encoding="utf-8", errors="replace", + check=False) + said = proc.stdout.split() + if proc.returncode != 0 or len(said) != 2: + raise Refusal( + f"could not compute the tree digest of {root} with its own " + f"scripts/repo_shape.py (exit {proc.returncode}): " + f"{last_line(proc.stderr)}") + return said[0], said[1] + + +def verify(standard: Standard, digest: str, env: dict) -> None: + """REFUSE TO RUN FROM A CLONE THAT IS NOT THE COMMIT AS IT WAS MADE: at + another commit, carrying a change (an untracked file included), or, where + a digest is given, digesting to anything else. Nothing is repaired or + removed here, because the doctor's NEXT lines may have sent somebody into + this directory, and what they changed is theirs to look at first.""" + root, commit = standard.root, standard.commit + head = git(["-C", str(root), "rev-parse", "--verify", "HEAD"], env) + at = head.stdout.strip().lower() + if head.returncode != 0 or at != commit: + raise Refusal( + f"{root} should be this command's clone of the standard at " + f"{commit}, and it is at {at or 'no commit git can read'}, so " + f"nothing was run from it. {CLONE_AFRESH}") + status = git(["-C", str(root), "status", "--porcelain", + "--untracked-files=normal"], env) + if status.returncode != 0: + raise Refusal(f"could not ask git whether {root} has changes: " + f"{last_line(status.stderr)}. {CLONE_AFRESH}") + changed = [line[3:] for line in status.stdout.splitlines() if line.strip()] + if changed: + shown = ", ".join(changed[:3]) + if len(changed) > 3: + shown += f", and {len(changed) - 3} more" + raise Refusal( + f"{root}, this command's clone of the standard at {commit}, has " + f"changes ({shown}), so nothing was run from it: a run uses the " + f"commit exactly as it was made. {CLONE_AFRESH}") + if not digest: + return + definition, computed = tree_digest(root, commit, env) + if definition != DIGEST_DEFINITION: + raise Refusal( + f"{root}'s own scripts/repo_shape.py computes the " + f"{definition} digest, and this package was stamped with a " + f"{DIGEST_DEFINITION} one, so the two cannot be compared and " + f"nothing was run from it. {CLONE_AFRESH}") + if computed != digest: + raise Refusal( + f"{root} is at {commit}, and its tree digests to {computed} " + f"({DIGEST_DEFINITION}), not to {digest}, the digest this " + "package was stamped with, so nothing was run from it: that " + f"clone is not the release this package names. {CLONE_AFRESH} " + f"If the new clone says the same, {standard.repo} does not hold " + "that release.") + + +def prepare(typed_ref: str = "") -> Standard: + """The standard this run uses, cloned if it has to be and checked. + + In order: BESIDE a checkout, that checkout, untouched. Otherwise the ref + is checked before anything reaches git; git must be there, because the + standard is a clone; the commit is the stamp's, the 40-hex ref's, or what + the remote says the ref is; a clone of it already in the cache is used, + and one that is not is made; and the clone is verified, against the + stamped digest where nothing overrode the stamp. + """ + beside = beside_checkout() + if beside is not None: + return Standard(beside, beside=True) + stamp = read_stamp() + wanted = target(stamp, typed_ref) + repo = repository() + url = clone_url(repo) + env = child_environment() + if shutil.which("git", path=env.get("PATH")) is None: + raise Refusal( + "git is not installed, and this command runs the standard from " + "a clone of it, so there is nothing to clone one with. Install " + "it: https://git-scm.com/downloads") + commit = wanted.commit or remote_commit( + url, repo, "HEAD" if wanted.source == "main" else wanted.ref, env) + cache = cache_root() + if commit and (cache / commit).is_dir(): + root = cache / commit + else: + root = clone_into(cache, url, repo, wanted, commit, env) + commit = root.name + standard = Standard(root, commit, repo, wanted) + verify(standard, wanted.digest, env) + return standard diff --git a/tests/test_openreposhape_package.py b/tests/test_openreposhape_package.py new file mode 100644 index 00000000..977993d3 --- /dev/null +++ b/tests/test_openreposhape_package.py @@ -0,0 +1,863 @@ +# SPDX-License-Identifier: Apache-2.0 +"""The openreposhape package (#563): its five commands, its stamp, its clone +of the standard, and the PATH its children get. + +THE LAUNCHER'S CASES FIRST, UNCHANGED. `tests/test_openreposhape_command.py` +states the bash `openRepoShape`'s rules; the cases that run it through its +`run_cmd` are imported below, so pytest collects them here as well. A test +function reads `run_cmd` and `USAGE_LINES` from its OWN module when it runs, +so `the_python_command`, autouse here and only here, points both at the +package's `openRepoShape` for each test this module collects: one case, one +rule, both commands, and not a line of the launcher's module changed for +it. This module carries no Windows skip, so those cases run on the Windows +job too, where the launcher's own module skips. The four launcher cases that +run its FILE (from stdin, or by its path) are re-stated below for this +command. + +`PythonCommand`, called, runs the package from this checkout's `src/`, +where it sits BESIDE a checkout of the standard and uses this one, as the +launcher uses the `setup.sh` beside it; `detached`, it runs an installed +copy of the package (its modules and `setup-project.py`, laid out as the +wheel lays them out) from a directory that is no checkout. + +THEN THE PACKAGE'S OWN. A SYNTHETIC STANDARD stands in for this repository: +a bare repository whose tools are stubs that print what they were handed, +at three refs (the `PINNED` tag, the `OTHER` branch and `main`), with this +checkout's own `scripts/repo_shape.py`, because the package computes a +clone's digest with the clone's own module. A test stamps its installed copy +with that standard's commit and digest, or leaves it unstamped. + +OFFLINE, LIKE THE REST OF THIS SUITE. Every clone is of a bare repository on +disk, and every cache is under the test's own directory: `$XDG_CACHE_HOME`, +`%LOCALAPPDATA%` and, for macOS's `~/Library/Caches`, `$HOME`. Nothing here +builds the wheel or installs it: that needs a build backend and +git-filter-repo from PyPI, so the `package` job in +`.github/workflows/tests.yml` does it, outside pytest. +""" + +from __future__ import annotations + +import ast +import importlib +import json +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +import test_openreposhape_command as launcher_cases +from conftest import REPO, git +from repo_shape import tree_digest +# THE LAUNCHER'S CASES, collected again in this module, where +# `the_python_command` runs them against the package's command. +from test_openreposhape_command import ( + test_a_bare_run_prints_the_usage_and_exits_one, + test_a_family_value_is_never_taken_as_the_project, + test_any_scaffold_argument_keeps_the_old_behaviour, + test_help_prints_every_usage_line, + test_help_still_exits_zero, + test_it_scaffolds_through_setup_sh, + test_no_org_and_no_terminal_refuses_before_setup_sh_runs, + test_no_project_and_no_terminal_refuses_with_the_usage, + test_preflight_forwards_an_org_it_was_given, + test_preflight_passes_through_without_an_org, + test_scaffold_flags_after_a_double_dash_still_reach_the_scaffold, + test_the_org_can_come_from_the_environment, + test_two_modes_is_a_refusal_not_last_wins, +) +from test_openreposhape_command import USAGE_LINES as LAUNCHER_USAGE +from test_openreposhape_command import command_env + +#: The launcher's cases this module runs again, named once more: they are +#: imported for pytest to collect, and nothing here calls them. +LAUNCHER_CASES = ( + test_a_bare_run_prints_the_usage_and_exits_one, + test_a_family_value_is_never_taken_as_the_project, + test_any_scaffold_argument_keeps_the_old_behaviour, + test_help_prints_every_usage_line, + test_help_still_exits_zero, + test_it_scaffolds_through_setup_sh, + test_no_org_and_no_terminal_refuses_before_setup_sh_runs, + test_no_project_and_no_terminal_refuses_with_the_usage, + test_preflight_forwards_an_org_it_was_given, + test_preflight_passes_through_without_an_org, + test_scaffold_flags_after_a_double_dash_still_reach_the_scaffold, + test_the_org_can_come_from_the_environment, + test_two_modes_is_a_refusal_not_last_wins, +) + +SOURCE = REPO / "src" +PACKAGE = SOURCE / "openreposhape" + +#: The package's first usage line, where the launcher's says what +#: `--install` places; the other four are the launcher's own. +USAGE_FIRST = ("openRepoShape --install refused here: pipx " + "upgrade openreposhape updates it") + +#: What a console script does, for the entry named: the package's directory +#: first on `sys.path`, then the function. `-B`, so no bytecode lands in +#: this checkout's `src/`. +BOOT = ("import sys; sys.path.insert(0, sys.argv.pop(1)); " + "import openreposhape.cli as cli; sys.exit(cli.{entry}())") + +#: The synthetic standard's refs, as `test_openreposhape_command.py` names +#: its own. +PINNED = "v9.9.9" +OTHER = "other" + +#: The tools a stub stands in for, each named as the package runs it. +TOOLS = ("setup-project.py", "scaffold-project.py", "adopt-project.py", + "update-shape.py", "shape-doctor.py") + +#: Every stub: one line, `STUB {json}`, naming the file, the ref it was +#: committed at, its arguments, where it ran, what it finds of +#: git-filter-repo, the first PATH entry and the hand-over variable; then +#: `$STUB_EXIT`, so a test sees an exit status travel back. +STUB = '''\ +import json, os, shutil, sys +print("STUB " + json.dumps({ + "tool": os.path.basename(__file__), "at": AT, "argv": sys.argv[1:], + "cwd": os.getcwd(), "filter_repo": shutil.which("git-filter-repo"), + "path0": os.environ.get("PATH", "").split(os.pathsep)[0], + "self_bootstrap": os.environ.get("OPENREPOSHAPE_SELF_BOOTSTRAP")}), + flush=True) +sys.exit(int(os.environ.get("STUB_EXIT", "0"))) +''' + + +# --- the package, run --------------------------------------------------------- + +def installed_copy(directory: Path, stamp: dict | None = None) -> Path: + """The package as the wheel lays it out, under `/site`: its + modules, `setup-project.py` beside them, and `stamp` (`version`, `tag`, + `commit`, `digest`) written into `_release.py` where one is given. The + directory above `site` is no checkout, so the copy is not BESIDE one.""" + package = directory / "site" / "openreposhape" + package.mkdir(parents=True) + for module in sorted(PACKAGE.glob("*.py")): + shutil.copyfile(module, package / module.name) + shutil.copyfile(REPO / "setup-project.py", package / "setup-project.py") + if stamp is not None: + (package / "_release.py").write_bytes("".join( + f'{name.upper()} = "{stamp[name]}"\n' + for name in ("version", "tag", "commit", "digest")).encode()) + return package.parent + + +def package_env(tmp_path: Path, home: Path | None = None, + env: dict | None = None, detached: bool = False) -> dict: + """`command_env`, with every per-user cache the package could reach + under `tmp_path`. `HOME` too for a DETACHED run, where it is where + macOS keeps the cache; a run beside the checkout keeps the person's.""" + caches = {"XDG_CACHE_HOME": str(tmp_path / "cache" / "xdg"), + "LOCALAPPDATA": str(tmp_path / "cache" / "local")} + if detached and home is None: + home = tmp_path / "home" + return command_env(home=home, setup_sh=None, env={**caches, **(env or {})}) + + +def cache_dir(tmp_path: Path) -> Path: + """Where `package_env` puts a detached run's clones, by the platform's + own rule, said here a second time so the package's is checked.""" + if os.name == "nt": + return tmp_path / "cache" / "local" / "openreposhape" / "checkouts" + if sys.platform == "darwin": + return (tmp_path / "home" / "Library" / "Caches" / "openreposhape" + / "checkouts") + return tmp_path / "cache" / "xdg" / "openreposhape" / "checkouts" + + +def run_package(site: Path, entry: str, args, env: dict, + cwd: Path | None = None, + python: Path | str = sys.executable + ) -> subprocess.CompletedProcess: + """The console script `entry` of the package at `site`, run as a person + runs one: stdin a pipe, and a session of its own (no terminal).""" + return subprocess.run( + [str(python), "-B", "-c", BOOT.format(entry=entry), str(site), + *args], capture_output=True, encoding="utf-8", errors="replace", + check=False, input="", cwd=str(cwd) if cwd else None, env=env, + start_new_session=True) + + +class PythonCommand: + """The package's `openRepoShape`, run as the launcher's `run_cmd` runs + the bash one: called with the same keywords, of which `setup_sh` and + `command` mean nothing here, because this command fetches no `setup.sh` + and has no file of its own to run. + + Called, `$OPENREPOSHAPE_REPO` names a repository that does not exist, + unless the test names one: beside this checkout nothing is cloned, and a + run that failed to see the checkout would fail on disk rather than ask + github.com where main is.""" + + usage_lines = (USAGE_FIRST, *LAUNCHER_USAGE[1:]) + + def __init__(self, tmp_path: Path) -> None: + self.tmp_path = tmp_path + + def __call__(self, *args: str, home: Path | None = None, + env: dict | None = None, setup_sh: Path | None = None, + command: Path | None = None) -> subprocess.CompletedProcess: + nowhere = {"OPENREPOSHAPE_REPO": str(self.tmp_path / "nowhere.git")} + return run_package(SOURCE, "openreposhape", args, + package_env(self.tmp_path, home, + {**nowhere, **(env or {})})) + + def detached(self, *args: str, home: Path | None = None, + env: dict | None = None, + cwd: Path | None = None) -> subprocess.CompletedProcess: + site = self.tmp_path / "package" / "site" + if not site.is_dir(): + installed_copy(self.tmp_path / "package") + return run_package(site, "openreposhape", args, + package_env(self.tmp_path, home, env, + detached=True), + cwd=cwd or self.tmp_path) + + +@pytest.fixture +def cmd(tmp_path) -> PythonCommand: + return PythonCommand(tmp_path) + + +@pytest.fixture(autouse=True) +def the_python_command(monkeypatch, cmd) -> None: + """For each test THIS module collects, the launcher's module runs the + package's command: its `run_cmd` is `cmd`, and the usage lines it + expects are this command's. Put back after every test, so the + launcher's own module, collected on its own, runs the bash command.""" + monkeypatch.setattr(launcher_cases, "run_cmd", cmd) + monkeypatch.setattr(launcher_cases, "USAGE_LINES", cmd.usage_lines) + + +def stub_report(stdout: str) -> dict: + """What the one stub that ran said about itself.""" + reports = [json.loads(line[len("STUB "):]) for line in stdout.splitlines() + if line.startswith("STUB ")] + assert len(reports) == 1, stdout + return reports[0] + + +def said(result: subprocess.CompletedProcess) -> str: + """stderr with its line breaks folded, for a refusal's sentences.""" + return " ".join(result.stderr.split()) + + +# --- the synthetic standard ----------------------------------------------------- + +class Standard: + """A bare repository on disk, its commit at each ref, and the + `sorted-ls-tree-r-v1` digest of each commit's tree.""" + + def __init__(self, bare: Path, commits: dict, digests: dict) -> None: + self.bare = bare + self.commits = commits + self.digests = digests + + def stamp(self, ref: str = PINNED, **changed) -> dict: + """The stamp a release at `ref` would carry, with `changed` over it.""" + return {"version": "9.9.9", "tag": PINNED, + "commit": self.commits[ref], "digest": self.digests[ref], + **changed} + + +def make_standard(directory: Path) -> Standard: + """`PINNED` (a tag), then `OTHER` (a branch off it), then `main`, each a + commit of every stub saying which ref it is.""" + seed = directory / "seed" + seed.mkdir(parents=True) + git("init", "-q", "-b", "main", cwd=seed) + (seed / "scripts").mkdir() + shutil.copyfile(REPO / "scripts" / "repo_shape.py", + seed / "scripts" / "repo_shape.py") + commits, digests = {}, {} + + def commit(name: str) -> None: + for tool in TOOLS: + (seed / tool).write_text(f"AT = {name!r}\n" + STUB, + encoding="utf-8") + git("add", "-A", cwd=seed) + git("commit", "-q", "-m", name, cwd=seed) + commits[name] = git("rev-parse", "HEAD", cwd=seed).stdout.strip() + digests[name] = tree_digest(seed, commits[name]) + + commit(PINNED) + git("tag", PINNED, cwd=seed) + git("switch", "-q", "-c", OTHER, cwd=seed) + commit(OTHER) + git("switch", "-q", "main", cwd=seed) + commit("main") + bare = directory / "standard.git" + git("clone", "-q", "--bare", str(seed), str(bare), cwd=directory) + return Standard(bare, commits, digests) + + +@pytest.fixture(scope="module") +def shape_standard(tmp_path_factory) -> Standard: + """One synthetic standard for the module: every test only clones it.""" + return make_standard(tmp_path_factory.mktemp("standard")) + + +def stamped_run(tmp_path: Path, standard: Standard, args, *, + stamp: dict | None = None, entry: str = "openreposhape", + env: dict | None = None, cwd: Path | None = None, + site: Path | None = None) -> subprocess.CompletedProcess: + """`entry` of a copy stamped at `PINNED` (or with `stamp`), cloning from + the synthetic standard, in `cwd` (the test's directory by default).""" + if site is None: + site = installed_copy(tmp_path / "package", + standard.stamp() if stamp is None else stamp) + return run_package(site, entry, args, package_env( + tmp_path, env={"OPENREPOSHAPE_REPO": str(standard.bare), + **(env or {})}, detached=True), cwd=cwd or tmp_path) + + +# --- the stamp and the build ------------------------------------------------------ + +def stamp_values(source: bytes) -> dict: + """The names `_release.py` assigns, and their values.""" + return {node.targets[0].id: node.value.value + for node in ast.parse(source).body + if isinstance(node, ast.Assign)} + + +def test_the_committed_stamp_is_unstamped(): + """0.0.0, and no tag, commit or digest: a build from a checkout, which + the release workflow refuses to publish. A release build writes the four + values; nothing commits them.""" + assert stamp_values((PACKAGE / "_release.py").read_bytes()) == { + "VERSION": "0.0.0", "TAG": "", "COMMIT": "", "DIGEST": ""} + + +def test_pyproject_takes_its_version_from_the_stamp_and_names_five_commands(): + """The version is written once, in the stamp; the one dependency is the + git-filter-repo CI runs and install.sh installs; the wheel is the + package plus `setup-project.py` beside its modules; and each of the five + commands names a function the package has.""" + tomllib = pytest.importorskip("tomllib") + data = tomllib.loads((REPO / "pyproject.toml").read_text(encoding="utf-8")) + project = data["project"] + assert project["name"] == "openreposhape" + assert project["dynamic"] == ["version"] and "version" not in project + assert project["requires-python"] == ">=3.9" + assert project["dependencies"] == ["git-filter-repo==2.47.0"] + assert 'FILTER_REPO_PIN="git-filter-repo==2.47.0"' in \ + (REPO / "install.sh").read_text(encoding="utf-8") + assert data["build-system"] == {"requires": ["hatchling==1.27.0"], + "build-backend": "hatchling.build"} + hatch = data["tool"]["hatch"] + assert hatch["version"] == {"path": "src/openreposhape/_release.py"} + # hatchling's own default pattern for a version file, read against the + # stamp: the version it finds is the stamp's. + found = re.search(r"(?i)^(__version__|VERSION) *= *(['\"])v?" + r"(?P.+?)\2", + (PACKAGE / "_release.py").read_text(encoding="utf-8"), + re.M) + assert found and found.group("version") == "0.0.0" + wheel = hatch["build"]["targets"]["wheel"] + assert wheel["packages"] == ["src/openreposhape"] + assert wheel["force-include"] == { + "setup-project.py": "openreposhape/setup-project.py"} + scripts = project["scripts"] + assert sorted(scripts) == ["adopt-project", "openRepoShape", + "scaffold-project", "shape-doctor", + "update-shape"] + sys.path.insert(0, str(SOURCE)) + wrote_bytecode, sys.dont_write_bytecode = sys.dont_write_bytecode, True + try: + for name, target in scripts.items(): + module, _, function = target.partition(":") + assert module == "openreposhape.cli", name + assert callable(getattr(importlib.import_module(module), + function)), name + finally: + sys.dont_write_bytecode = wrote_bytecode + sys.path.remove(str(SOURCE)) + + +def test_a_stamp_that_names_part_of_a_release_is_refused(tmp_path): + """A commit with no digest would run unchecked: the whole stamp or + none of it.""" + site = installed_copy(tmp_path, {"version": "9.9.9", "tag": PINNED, + "commit": "a" * 40, "digest": ""}) + result = run_package(site, "openreposhape", ["--version"], + package_env(tmp_path, detached=True)) + assert result.returncode == 2, result.stdout + result.stderr + assert "names part of a release and not all of it" in said(result) + assert result.stdout == "" + + +# --- --version ---------------------------------------------------------------------- + +def test_version_names_the_release_and_clones_nothing(tmp_path, + shape_standard): + """The version, the tag, the commit, the digest and the clone, from the + stamp alone: no network and no clone, before the first run or after.""" + stamp = shape_standard.stamp() + result = stamped_run(tmp_path, shape_standard, ["--version"]) + assert result.returncode == 0, result.stdout + result.stderr + clone = cache_dir(tmp_path) / stamp["commit"] + assert result.stdout.splitlines() == [ + f"openRepoShape 9.9.9 ({shape_standard.bare} @ {stamp['commit']})", + " version 9.9.9", + f" tag {PINNED}", + f" commit {stamp['commit']}", + f" digest {stamp['digest']} (sorted-ls-tree-r-v1)", + f" checkout {clone} (not cloned yet: the first run that needs it " + "clones it)"] + assert not cache_dir(tmp_path).exists() + + +def test_version_says_an_unstamped_build_follows_main(cmd): + result = cmd.detached("--version") + assert result.returncode == 0, result.stderr + lines = result.stdout.splitlines() + assert lines[0] == "openRepoShape 0.0.0 (opensoft/openRepoShape @ main)" + assert " version 0.0.0, unstamped: a build from a checkout, not a " \ + "release, so it follows main" in lines + + +def test_version_beside_a_checkout_names_the_checkout(cmd): + result = cmd("--version", env={"OPENREPOSHAPE_REF": OTHER}) + assert result.returncode == 0, result.stderr + assert f" checkout {REPO}: this package sits in it, so every run uses " \ + "it as it stands and clones nothing" in result.stdout.splitlines() + + +# --- the launcher's cases that run its FILE, for this command ----------------------- + +def test_the_install_line_keeps_the_launchers_description_column(cmd): + """The launcher's column rule (#546) for this command's usage: the + `--install` line describes itself in the column `--preflight`'s does, + and the doctor's description sits under its form, in that column.""" + lines = cmd("--help").stdout.splitlines() + column = LAUNCHER_USAGE[1].index("check this MACHINE") + assert lines[0] == USAGE_FIRST + assert USAGE_FIRST.index("refused here") == column + doctor = lines.index(LAUNCHER_USAGE[2]) + assert lines[doctor + 1] == \ + " " * column + "diagnose a REPOSITORY against this standard" + + +def test_a_bare_run_asks_nothing_of_git_gh_or_curl(cmd, tmp_path): + """The usage, exit 1, and NO CALL: a `git`, a `gh` and a `curl` that + log every call stand first on PATH (a POSIX script and a `.cmd` each), + and the log must not exist afterwards.""" + log = tmp_path / "calls.log" + fakes = tmp_path / "fakes" + fakes.mkdir() + for name in ("git", "gh", "curl"): + posix = fakes / name + posix.write_bytes(f"#!/bin/sh\necho {name} \"$*\" >> '{log}'\n" + "exit 1\n".encode("utf-8")) + posix.chmod(0o755) + (fakes / f"{name}.cmd").write_bytes( + f'@echo {name} %* >> "{log}"\r\n@exit /b 1\r\n'.encode("utf-8")) + result = cmd(env={"PATH": f"{fakes}{os.pathsep}{os.environ['PATH']}"}) + assert result.returncode == 1, result.stdout + result.stderr + assert USAGE_FIRST in result.stdout.splitlines() + assert not log.exists(), log.read_text(encoding="utf-8") + + +@pytest.mark.parametrize("ref", ["-x", "--upload-pack=touch /tmp/pwned", + "a" * 256, "refs/heads/..", "x.lock", + "v1\nforged"]) +def test_a_ref_is_checked_before_anything_is_cloned(cmd, tmp_path, ref): + """`$OPENREPOSHAPE_REF` reaches `git checkout`, so it is checked first, + in the launcher's three steps and its words. The repository named here + does not exist, so a ref that got past the check would fail on the + clone instead; the over-long and the two-line value are not echoed.""" + result = cmd.detached("--doctor", ".", env={ + "OPENREPOSHAPE_REPO": str(tmp_path / "never-cloned.git"), + "OPENREPOSHAPE_REF": ref}) + assert result.returncode == 2, result.stdout + result.stderr + assert "$OPENREPOSHAPE_REF" in result.stderr + assert "git checkout" in result.stderr + assert "clone" not in said(result) + assert "a" * 256 not in result.stderr and "forged" not in result.stderr + + +def test_a_standard_with_no_doctor_is_refused_by_name(cmd, tmp_path): + """A ref of the standard older than `shape-doctor.py` is refused by + name, and the way out is a ref that carries it.""" + seed = tmp_path / "seed" + seed.mkdir() + git("init", "-q", "-b", "main", cwd=seed) + (seed / "README.md").write_bytes(b"# an older standard\n") + git("add", "-A", cwd=seed) + git("commit", "-q", "-m", "before the doctor", cwd=seed) + bare = tmp_path / "old.git" + git("clone", "-q", "--bare", str(seed), str(bare), cwd=tmp_path) + result = cmd.detached("--doctor", ".", + env={"OPENREPOSHAPE_REPO": str(bare)}) + assert result.returncode == 2, result.stdout + result.stderr + assert "has no shape-doctor.py, so there is nothing to run" \ + in said(result) + assert "Set $OPENREPOSHAPE_REF to a ref that carries it" in said(result) + + +def test_version_names_the_ref_set_for_the_run(cmd): + result = cmd.detached("--version", env={"OPENREPOSHAPE_REF": "v1.2.3"}) + assert result.returncode == 0, result.stderr + assert result.stdout.splitlines()[0] == \ + "openRepoShape 0.0.0 (opensoft/openRepoShape @ v1.2.3)" + + +# --- the clone: once per commit, and checked before every run ------------------------ + +def test_a_stamped_run_clones_its_commit_once_and_runs_from_it( + tmp_path, shape_standard): + """The first run clones the stamped commit into the cache and runs the + clone's own tool; the second uses that clone as it is, and asks the + repository nothing: it answers with the repository gone.""" + commit = shape_standard.stamp()["commit"] + clone = cache_dir(tmp_path) / commit + first = stamped_run(tmp_path, shape_standard, ["--doctor", "."]) + assert first.returncode == 0, first.stdout + first.stderr + assert stub_report(first.stdout)["at"] == PINNED + assert f"openRepoShape: cloning {shape_standard.bare} into " \ + f"{cache_dir(tmp_path)}, once for commit {commit}" in said(first) + second = run_package( + tmp_path / "package" / "site", "openreposhape", ["--doctor", "."], + package_env(tmp_path, detached=True, + env={"OPENREPOSHAPE_REPO": str(tmp_path / "gone.git")}), + cwd=tmp_path) + assert second.returncode == 0, second.stdout + second.stderr + assert stub_report(second.stdout)["at"] == PINNED + assert "cloning" not in second.stderr + assert sorted(cache_dir(tmp_path).iterdir()) == [clone], ( + "a clone half made, or a second one, was left in the cache") + assert git("rev-parse", "HEAD", cwd=clone).stdout.strip() == commit + + +def test_a_clone_whose_digest_is_not_the_stamped_one_is_refused( + tmp_path, shape_standard): + """The digest is the clone's own `repo_shape.tree_digest`, compared with + the stamp's; a clone that digests to anything else runs nothing.""" + wrong = "0" * 64 + result = stamped_run(tmp_path, shape_standard, ["--doctor", "."], + stamp=shape_standard.stamp(digest=wrong)) + assert result.returncode == 2, result.stdout + result.stderr + assert f"its tree digests to {shape_standard.digests[PINNED]} " \ + f"(sorted-ls-tree-r-v1), not to {wrong}, the digest this package " \ + "was stamped with, so nothing was run from it" in said(result) + assert "STUB" not in result.stdout + + +@pytest.mark.parametrize("change", ["modified", "untracked"]) +def test_a_clone_with_changes_is_refused_and_left_as_it_is( + tmp_path, shape_standard, change): + """A run uses the commit exactly as it was made. A clone somebody + changed, an untracked file included, runs nothing and is not repaired: + what changed is theirs to look at first.""" + assert stamped_run(tmp_path, shape_standard, + ["--doctor", "."]).returncode == 0 + clone = cache_dir(tmp_path) / shape_standard.stamp()["commit"] + name = "shape-doctor.py" if change == "modified" else "extra.txt" + with open(clone / name, "ab") as handle: + handle.write(b"# a change\n") + result = stamped_run(tmp_path, shape_standard, ["--doctor", "."], + site=tmp_path / "package" / "site") + assert result.returncode == 2, result.stdout + result.stderr + assert f"has changes ({name}), so nothing was run from it" in said(result) + assert "Delete it and run this again" in said(result) + assert "STUB" not in result.stdout + assert (clone / name).read_bytes().endswith(b"# a change\n") + + +def test_a_clone_at_another_commit_is_refused(tmp_path, shape_standard): + assert stamped_run(tmp_path, shape_standard, + ["--doctor", "."]).returncode == 0 + clone = cache_dir(tmp_path) / shape_standard.stamp()["commit"] + git("checkout", "-q", "--detach", shape_standard.commits["main"], + cwd=clone) + result = stamped_run(tmp_path, shape_standard, ["--doctor", "."], + site=tmp_path / "package" / "site") + assert result.returncode == 2, result.stdout + result.stderr + assert f"and it is at {shape_standard.commits['main']}, so nothing was " \ + "run from it" in said(result) + assert "STUB" not in result.stdout + + +def test_a_commit_the_repository_lacks_leaves_nothing_in_the_cache( + tmp_path, shape_standard): + """The clone is made beside its place and moved in whole, so a clone + that could not check out the commit leaves nothing to mistake for one.""" + result = stamped_run(tmp_path, shape_standard, ["--doctor", "."], + stamp=shape_standard.stamp(commit="b" * 40)) + assert result.returncode == 2, result.stdout + result.stderr + assert f"could not check out '{'b' * 40}'" in said(result) + assert list(cache_dir(tmp_path).iterdir()) == [] + + +# --- which commit ------------------------------------------------------------------- + +@pytest.mark.parametrize("ref", [OTHER, "main", PINNED]) +def test_the_variable_overrides_the_stamp_for_one_run(tmp_path, + shape_standard, ref): + """`$OPENREPOSHAPE_REF` names another branch, tag or commit for the run + it is set for, and no digest is checked then: the stamped digest is the + stamped commit's. (The stamp's digest here is wrong, which a stamped run + would refuse.)""" + stamp = shape_standard.stamp(digest="0" * 64) + result = stamped_run(tmp_path, shape_standard, ["--doctor", "."], + stamp=stamp, env={"OPENREPOSHAPE_REF": ref}) + assert result.returncode == 0, result.stdout + result.stderr + assert stub_report(result.stdout)["at"] == ref + version = stamped_run(tmp_path, shape_standard, ["--version"], + site=tmp_path / "package" / "site", + env={"OPENREPOSHAPE_REF": ref}) + assert f"({shape_standard.bare} @ {ref})" in version.stdout + assert f" ref {ref}, from $OPENREPOSHAPE_REF, for this run " \ + "instead of the commit above; no digest is checked" in version.stdout + + +def test_an_unstamped_build_follows_main(tmp_path): + """Each run asks the repository where main is: a commit landing there + is the next run's standard, in a clone of its own.""" + standard = make_standard(tmp_path / "moving") + site = installed_copy(tmp_path / "package") + env = package_env(tmp_path, detached=True, + env={"OPENREPOSHAPE_REPO": str(standard.bare)}) + first = run_package(site, "openreposhape", ["--doctor", "."], env, + cwd=tmp_path) + assert first.returncode == 0, first.stdout + first.stderr + assert stub_report(first.stdout)["at"] == "main" + seed = tmp_path / "moving" / "seed" + (seed / "shape-doctor.py").write_text("AT = 'main, later'\n" + STUB, + encoding="utf-8") + git("commit", "-q", "-am", "later", cwd=seed) + git("push", "-q", str(standard.bare), "main", cwd=seed) + later = git("rev-parse", "HEAD", cwd=seed).stdout.strip() + second = run_package(site, "openreposhape", ["--doctor", "."], env, + cwd=tmp_path) + assert second.returncode == 0, second.stdout + second.stderr + assert stub_report(second.stdout)["at"] == "main, later" + assert sorted(path.name for path in cache_dir(tmp_path).iterdir()) == \ + sorted([standard.commits["main"], later]) + + +# --- the five commands ------------------------------------------------------------ + +@pytest.mark.parametrize("entry, tool", [ + ("scaffold_project", "scaffold-project.py"), + ("adopt_project", "adopt-project.py"), + ("update_shape", "update-shape.py"), + ("shape_doctor", "shape-doctor.py"), +]) +def test_each_tool_command_runs_its_file_with_every_argument_unchanged( + tmp_path, shape_standard, entry, tool): + """The clone's own file, every argument as typed (an empty one, a `--` + and a flag after it included), in the directory the command was typed + in, and the file's exit status back.""" + args = ["plan", "--source", "a path/with a space", "--", "-x", "", + "--json"] + typed_in = tmp_path / "typed here" + typed_in.mkdir() + result = stamped_run(tmp_path, shape_standard, args, entry=entry, + env={"STUB_EXIT": "7"}, cwd=typed_in) + assert result.returncode == 7, result.stdout + result.stderr + report = stub_report(result.stdout) + assert (report["tool"], report["at"]) == (tool, PINNED) + assert report["argv"] == args + assert Path(report["cwd"]).resolve() == typed_in.resolve() + + +def test_a_scaffold_hands_over_to_the_clones_setup_project(tmp_path, + shape_standard): + """setup.sh's hand-over from a clone it made: `--into` where the + command was typed, FIRST, so a later one of the person's wins; then + `--org` and `--project`; then every other argument; and + `$OPENREPOSHAPE_SELF_BOOTSTRAP=1`, because no fork origin stands behind + the person.""" + typed_in = tmp_path / "here" + typed_in.mkdir() + result = stamped_run(tmp_path, shape_standard, + ["Atlas", "--org", "TestOrg", "--visibility", + "private", "--", "--reference", "x.md"], + cwd=typed_in) + assert result.returncode == 0, result.stdout + result.stderr + report = stub_report(result.stdout) + assert (report["tool"], report["at"]) == ("setup-project.py", PINNED) + assert report["argv"][0] == "--into" + assert Path(report["argv"][1]).resolve() == typed_in.resolve() + assert report["argv"][2:] == ["--org", "TestOrg", "--project", "Atlas", + "--visibility", "private", "--", + "--reference", "x.md"] + assert report["self_bootstrap"] == "1" + commit = shape_standard.stamp()["commit"] + assert f"openRepoShape: the standard is {shape_standard.bare} at " \ + f"{commit}, cloned at {cache_dir(tmp_path) / commit}." \ + in result.stdout.splitlines() + + +def test_a_typed_shape_ref_is_the_ref_of_the_scaffolds_clone( + tmp_path, shape_standard): + """`--shape-ref` reached setup.sh after the launcher's own ref, so it + named the ref of the scaffold's clone. Here it does the same, over the + stamp, with no digest checked, and still travels on to setup-project.py, + which checks it as it checks every flag.""" + result = stamped_run(tmp_path, shape_standard, + ["Atlas", "--org", "TestOrg", "--shape-ref", OTHER], + stamp=shape_standard.stamp(digest="0" * 64)) + assert result.returncode == 0, result.stdout + result.stderr + report = stub_report(result.stdout) + assert report["at"] == OTHER + assert report["argv"][-2:] == ["--shape-ref", OTHER] + + +def test_the_doctor_says_last_which_clone_it_kept(tmp_path, shape_standard): + """Its NEXT lines name files in the clone, so the clone is kept and the + report's last line says where; under `--json` that line goes to stderr + and stdout stays the doctor's. The doctor's own flags arrive as the + launcher hands them on.""" + clone = cache_dir(tmp_path) / shape_standard.stamp()["commit"] + kept = (f"the standard this run used is kept at {clone}, for any NEXT " + "command above that names it") + plain = stamped_run(tmp_path, shape_standard, ["--doctor", "../x"]) + assert plain.returncode == 0, plain.stdout + plain.stderr + assert plain.stdout.splitlines()[-1].startswith(kept), plain.stdout + as_json = stamped_run(tmp_path, shape_standard, + ["--doctor", "--json", "../x", + "--placement-plan", "plan.yaml"], + site=tmp_path / "package" / "site") + assert as_json.returncode == 0, as_json.stdout + as_json.stderr + assert stub_report(as_json.stdout)["argv"] == [ + "--root", "../x", "--json", "--placement-plan=plan.yaml"] + assert "kept at" not in as_json.stdout + assert kept in as_json.stderr + + +def test_install_is_refused_and_names_pipx_upgrade(cmd, tmp_path): + """pipx (or pip) placed this command and keeps it current, so + `--install` places nothing, from a checkout or installed, and says how + to update it.""" + for run in (cmd, cmd.detached): + result = run("--install", home=tmp_path, + env={"OPENREPOSHAPE_BIN_DIR": str(tmp_path / "bin")}) + assert result.returncode == 2, result.stdout + result.stderr + assert "--install places nothing here" in said(result) + assert "pipx upgrade openreposhape" in said(result) + assert not (tmp_path / "bin").exists() + assert not (tmp_path / ".local").exists() + + +# --- with no git ---------------------------------------------------------------------- + +def no_git_path(tmp_path: Path) -> dict: + """A PATH with nothing on it, and so no git.""" + empty = tmp_path / "nothing-on-path" + empty.mkdir() + return {"PATH": str(empty)} + + +def test_preflight_runs_before_git_is_installed(tmp_path): + """`--preflight` runs the `setup-project.py` the package carries, so a + machine with no git still gets the preflight, which says git is + missing, rather than this command's refusal for want of a clone.""" + site = installed_copy(tmp_path) + result = run_package(site, "openreposhape", + ["--preflight", "--local-remote-dir", + str(tmp_path / "remotes")], + package_env(tmp_path, detached=True, + env=no_git_path(tmp_path))) + assert result.returncode == 1, result.stdout + result.stderr + assert "(1) preflight" in result.stdout + assert "git is not installed" in result.stderr + assert "REFUSED" not in result.stderr + assert not cache_dir(tmp_path).exists() + + +def test_with_no_git_a_run_that_needs_the_standard_is_refused(tmp_path): + site = installed_copy(tmp_path) + result = run_package(site, "openreposhape", ["--doctor", "."], + package_env(tmp_path, detached=True, + env=no_git_path(tmp_path)), + cwd=tmp_path) + assert result.returncode == 2, result.stdout + result.stderr + assert "git is not installed, and this command runs the standard from " \ + "a clone of it" in said(result) + + +# --- git-filter-repo, for every child ------------------------------------------------- + +def package_module(name: str): + """`openreposhape.` from this checkout's `src/`, imported in this + process with no bytecode written.""" + sys.path.insert(0, str(SOURCE)) + wrote_bytecode, sys.dont_write_bytecode = sys.dont_write_bytecode, True + try: + return importlib.import_module(f"openreposhape.{name}") + finally: + sys.dont_write_bytecode = wrote_bytecode + sys.path.remove(str(SOURCE)) + + +def fake_filter_repo(directory: Path) -> Path: + """A `git-filter-repo` in `directory` a command line would start, as pip + writes one: a script on POSIX, and on Windows a name `PATHEXT` knows.""" + if os.name == "nt": + fake = directory / "git-filter-repo.cmd" + fake.write_bytes(b"@exit /b 0\r\n") + else: + fake = directory / "git-filter-repo" + fake.write_bytes(b"#!/bin/sh\nexit 0\n") + fake.chmod(0o755) + return fake + + +def test_children_get_the_scripts_directory_first_when_it_holds_filter_repo( + tmp_path, monkeypatch): + """First on PATH when a `git-filter-repo` is there, and PATH untouched + when none is.""" + standard = package_module("standard") + scripts = tmp_path / "scripts" + scripts.mkdir() + monkeypatch.setattr(standard.sysconfig, "get_path", + lambda name, *rest: str(scripts)) + before = {"PATH": os.pathsep.join(["/one", "/two"]), "OTHER": "kept"} + assert standard.child_environment(before) == before + fake_filter_repo(scripts) + assert standard.child_environment(before) == { + "PATH": os.pathsep.join([str(scripts), "/one", "/two"]), + "OTHER": "kept"} + + +def test_a_child_finds_the_git_filter_repo_of_the_packages_environment( + tmp_path, shape_standard): + """THE PIPX CASE, END TO END: the package runs in an environment of its + own whose scripts directory holds `git-filter-repo` and is on nobody's + PATH. The tool it starts finds that one, first on its PATH; and before + there is one there, it finds none, and PATH is as it was.""" + venv = tmp_path / "venv" + subprocess.run([sys.executable, "-m", "venv", "--without-pip", str(venv)], + check=True, capture_output=True) + python = venv / ("Scripts/python.exe" if os.name == "nt" else "bin/python") + scripts = python.parent + standard = package_module("standard") + elsewhere = [entry for entry in os.environ["PATH"].split(os.pathsep) + if entry and not standard.filter_repo_in(entry)] + site = installed_copy(tmp_path / "package", shape_standard.stamp()) + env = package_env(tmp_path, detached=True, env={ + "OPENREPOSHAPE_REPO": str(shape_standard.bare), + "PATH": os.pathsep.join(elsewhere)}) + + def adopt(): + result = run_package(site, "adopt_project", ["plan"], env, + cwd=tmp_path, python=python) + assert result.returncode == 0, result.stdout + result.stderr + return stub_report(result.stdout) + + before = adopt() + assert before["filter_repo"] is None + assert before["path0"] == elsewhere[0] + fake = fake_filter_repo(scripts) + after = adopt() + assert os.path.normcase(after["filter_repo"]) == os.path.normcase( + str(fake)) + assert os.path.normcase(after["path0"]) == os.path.normcase(str(scripts)) diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index cbdf9d89..d1e35997 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -43,7 +43,20 @@ REPO / "templates" / "family-root" / "scripts" / "validate-family.py", REPO / "templates" / "family-root" / "scripts" / "bootstrap.py", REPO / "templates" / "family-root" / "scripts" / "siblings.py", + REPO / "src" / "openreposhape" / "__init__.py", + REPO / "src" / "openreposhape" / "__main__.py", + REPO / "src" / "openreposhape" / "_release.py", + REPO / "src" / "openreposhape" / "cli.py", + REPO / "src" / "openreposhape" / "standard.py", ] +#: THE openreposhape PACKAGE joins them on 2026-10-15 (#563), every module of +#: it, and `test_the_package_is_every_module_shipped` below holds the list to +#: the directory. pipx installs it with one dependency, git-filter-repo, which +#: it never imports: it only puts that program on its children's PATH. A +#: package that imported anything else would make the second way in need +#: more than the first, and the first is python3 and git alone. Its modules +#: import each other relatively, which the walk above does not count. +#: #: `bootstrap` joins them on 2026-09-09 (#76): the holder's `siblings.py` #: imports the credential resolution and the member rows from the #: `bootstrap.py` beside it rather than defining either a second time, and the @@ -108,6 +121,14 @@ def test_every_shipped_script_compiles(): compile(path.read_text(encoding="utf-8"), str(path), "exec") +def test_the_package_is_every_module_shipped(): + """Every module of `src/openreposhape/` is in `SHIPPED`, so a module + added to the package is held to the standard library like the rest.""" + package = REPO / "src" / "openreposhape" + assert sorted(package.glob("*.py")) == sorted( + path for path in SHIPPED if path.parent == package) + + @pytest.mark.parametrize("template", ["assembly-root", "family-root"]) def test_the_materializers_account_for_every_template_file(template): """A file added to a root template but named in no copy list would @@ -1473,8 +1494,10 @@ def test_setup_project_py_is_pure_ascii(name): #: Every file that carries it. The shim is where a person meets it, README.md #: is where they look it up, and `docs/handbook.html` is the README's designed -#: reading - a page a reader copies from as readily as either. -OPENREPOTOOLS_INSTALL_IN = ["openRepoShape", "README.md", "docs/handbook.html"] +#: reading - a page a reader copies from as readily as either. The package's +#: `openRepoShape` prints it in its usage, as the shim does (#563). +OPENREPOTOOLS_INSTALL_IN = ["openRepoShape", "README.md", "docs/handbook.html", + "src/openreposhape/cli.py"] @pytest.mark.parametrize("name", OPENREPOTOOLS_INSTALL_IN) From 4759eea6431bbb1b2133be9aac1bf283184c97ac Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:24:41 +0000 Subject: [PATCH 13/25] .github/workflows/tests.yml: a package job builds the openreposhape wheel twice on Python 3.9, checks the two are one file holding this commit's bytes, installs it with pipx and runs the five commands, outside pytest (#563) The pytest suite stays network-free, as the header says, and now says which job is the exception: building the wheel needs the backend pyproject.toml pins and installing it needs git-filter-repo, both from PyPI. So a fifth job on ubuntu, with no pytest in it: - builds the wheel twice at the commit's own time (SOURCE_DATE_EPOCH) and `cmp`s the two; - reads every file in the wheel but its metadata against `git show HEAD:`, byte for byte, setup-project.py included; - installs it with pipx into the job's own directories, on Python 3.9, the oldest the package supports; - runs `--version`, `--help`, `--install` (refused, naming pipx upgrade), `--preflight`, and `--doctor --json` over an empty repository against this commit served from a bare repository on disk, and requires the machine row to find the git-filter-repo pipx installed beside the package, with none on PATH; then the four tools' `--help`. actionlint 1.7.12 (the pinned one) passes over the file, with shellcheck. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/tests.yml | 131 ++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index aeb0de45..a20503d9 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -4,6 +4,10 @@ name: tests # clones them over the filesystem. It never calls `gh`, never touches the # network, and must never create a real repository. # +# The `package` job at the end is the one job that reaches PyPI, and it runs +# no pytest: building the openreposhape wheel and installing it with pipx +# need the build backend and git-filter-repo from there (#563). +# # `push` to `main` is deliberate and LINUX-ONLY: a squash lands a tree no # pull-request run ever tested, and the ubuntu job alone is priced to catch # that (macOS bills at 10x, Windows at 2x, and a squash-interaction defect is @@ -298,3 +302,130 @@ jobs: - name: tests run: python3 -m pytest tests -q + + # THE openreposhape PACKAGE, BUILT AND INSTALLED AS A PERSON INSTALLS IT + # (#563). A JOB OF ITS OWN, AND NO PYTEST IN IT: the build backend + # `pyproject.toml` pins (hatchling) and git-filter-repo, the package's one + # dependency, both come from PyPI, and the suite in the three jobs above + # never touches the network. What can be checked offline is checked there, + # on all three runners (`tests/test_openreposhape_package.py`); this job is + # what only a real build and a real pipx can say. + # + # UBUNTU, ON THE OLDEST PYTHON THE PACKAGE SUPPORTS, 3.9. The wheel is pure + # Python, one file for every system; the release workflow (#563's next + # pull request) installs it on all three before it uploads anything. + package: + runs-on: ubuntu-latest + steps: + # The same two commits the other jobs pin. + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: '3.9' + + # TWICE, AT THE COMMIT'S OWN TIME, AND THE TWO MUST BE ONE FILE: a + # release uploads bytes it can build again. `--no-deps`, because the + # wheel is this package alone; git-filter-repo arrives with the install. + - name: build the wheel twice, and compare the two + run: | + set -euo pipefail + SOURCE_DATE_EPOCH="$(git log -1 --format=%ct HEAD)" + export SOURCE_DATE_EPOCH + for out in a b; do + python -m pip wheel --disable-pip-version-check --no-deps \ + --wheel-dir "$RUNNER_TEMP/dist-$out" . + done + cmp "$RUNNER_TEMP"/dist-a/*.whl "$RUNNER_TEMP"/dist-b/*.whl + sha256sum "$RUNNER_TEMP"/dist-a/*.whl + + # EVERY FILE IN THE WHEEL BUT ITS METADATA IS THIS COMMIT'S OWN, BYTE + # FOR BYTE, read from git: the package's modules from `src/`, and + # `setup-project.py`, which the wheel carries so that `--preflight` + # runs before git is installed. + - name: the wheel holds this commit's files, byte for byte + run: | + python - "$RUNNER_TEMP"/dist-a/*.whl <<'EOF' + import subprocess, sys, zipfile + with zipfile.ZipFile(sys.argv[1]) as wheel: + for name in wheel.namelist(): + if ".dist-info/" in name: + continue + source = ("setup-project.py" + if name == "openreposhape/setup-project.py" + else "src/" + name) + committed = subprocess.run( + ["git", "show", "HEAD:" + source], + capture_output=True, check=True).stdout + if wheel.read(name) != committed: + sys.exit(f"{name} in the wheel is not {source} at HEAD") + print(f"{name} = HEAD:{source}") + EOF + + # pipx AS A PERSON RUNS IT, into directories of this job's own, so the + # runner's own pipx and its apps are left alone, and with the 3.9 above. + - name: install the wheel with pipx + run: | + set -euo pipefail + { + echo "PIPX_HOME=$RUNNER_TEMP/pipx/home" + echo "PIPX_BIN_DIR=$RUNNER_TEMP/pipx/bin" + } >> "$GITHUB_ENV" + export PIPX_HOME="$RUNNER_TEMP/pipx/home" + export PIPX_BIN_DIR="$RUNNER_TEMP/pipx/bin" + pipx --version + pipx install --python "$(command -v python)" \ + "$RUNNER_TEMP"/dist-a/*.whl + "$PIPX_HOME/venvs/openreposhape/bin/python" --version + ls -l "$PIPX_BIN_DIR" + + # THE FIVE COMMANDS PIPX PLACED, AGAINST THIS COMMIT. A bare repository + # on disk holds it as `main`, which an unstamped build follows, so + # nothing is cloned from github.com. No git-filter-repo is on PATH, and + # the doctor's machine row must still find the one pipx installed + # beside the package: the PATH every command gives its children. + - name: run the commands pipx placed + env: + OPENREPOSHAPE_REPO: ${{ runner.temp }}/standard.git + run: | + set -euo pipefail + git init -q --bare --initial-branch=main "$OPENREPOSHAPE_REPO" + git push -q "$OPENREPOSHAPE_REPO" HEAD:refs/heads/main + export PATH="$PIPX_BIN_DIR:$PATH" + if command -v git-filter-repo; then + echo "::error::git-filter-repo is on PATH already, so this job cannot show the commands finding their own" + exit 1 + fi + out="$RUNNER_TEMP/out" + mkdir -p "$out" + openRepoShape --version | tee "$out/version" + grep -q '^openRepoShape 0.0.0 (.* @ main)$' "$out/version" + openRepoShape --help >/dev/null + status=0 + openRepoShape --install 2>"$out/install" || status=$? + cat "$out/install" + test "$status" -eq 2 + grep -q 'pipx upgrade openreposhape' "$out/install" + openRepoShape --preflight --local-remote-dir "$RUNNER_TEMP/no-remotes" + git init -q "$RUNNER_TEMP/empty" + status=0 + openRepoShape --doctor "$RUNNER_TEMP/empty" --json \ + >"$out/doctor.json" || status=$? + test "$status" -eq 2 + python - "$out/doctor.json" \ + "$PIPX_HOME/venvs/openreposhape/bin/git-filter-repo" <<'EOF' + import json, sys + with open(sys.argv[1], encoding="utf-8") as handle: + report = json.load(handle) + print("verdict:", report["verdict"]) + [machine] = [row for row in report["rows"] if row["id"] == "machine"] + found = machine["detail"]["git-filter-repo"] + print("machine row: git-filter-repo at", found) + if found != sys.argv[2]: + sys.exit(f"the doctor found git-filter-repo at {found!r}, " + f"not at {sys.argv[2]!r}") + EOF + for tool in scaffold-project adopt-project update-shape shape-doctor; do + "$tool" --help >/dev/null + done From ab8bfe3460dc47a1ad1752dd8589985a2ff52dd2 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:35:27 +0000 Subject: [PATCH 14/25] tests/test_openreposhape_command.py: the symlink-target test asserts the link and where it points in two assertions, not one (#563) SonarCloud python:S9073 on #590: one `and` hid which half failed. Only the test ba832315 added changes. Co-Authored-By: Claude Opus 5.5 --- tests/test_openreposhape_command.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/test_openreposhape_command.py b/tests/test_openreposhape_command.py index d52a7a8d..69eac55e 100644 --- a/tests/test_openreposhape_command.py +++ b/tests/test_openreposhape_command.py @@ -379,7 +379,8 @@ def test_install_refuses_a_target_that_is_a_symlink(tmp_path, dangling): "command through it" in said, said assert "pipx upgrade openreposhape" in said assert "pipx uninstall openreposhape" in said - assert link.is_symlink() and os.readlink(link) == str(packaged) + assert link.is_symlink() + assert os.readlink(link) == str(packaged) if not dangling: assert packaged.read_bytes() == original assert not packaged.with_name("openRepoShape.ref").exists() From bfd927a94fbb688e8f6653be6a179a2670e7baf8 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:35:27 +0000 Subject: [PATCH 15/25] install.sh, tests/test_install_sh.py: the symlink-target check uses `[[`, and its test asserts the link and where it points in two assertions (#563) SonarCloud on #590: shelldre:S7688 on the new test in install.sh (bash 3.2 has `[[`), and python:S9073 on the new test's one `and`. Only the lines 8cae45cb added change. Co-Authored-By: Claude Opus 5.5 --- install.sh | 2 +- tests/test_install_sh.py | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/install.sh b/install.sh index 1c2c4e1f..1360ce4f 100755 --- a/install.sh +++ b/install.sh @@ -306,7 +306,7 @@ main() { # release named here may be older, so its refusal cannot be relied on. bin_dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" target="$bin_dir/openRepoShape" - [ ! -L "$target" ] || + [[ ! -L "$target" ]] || die "$target is a symbolic link, so nothing was installed: pipx places the openreposhape package's openRepoShape that way, and installing this one through the link would overwrite the package's command. Use one installer or the other: keep the package's command, and update it with: pipx upgrade openreposhape or remove the link (pipx uninstall openreposhape, or rm $target), then run this line again." diff --git a/tests/test_install_sh.py b/tests/test_install_sh.py index f4fcd05b..e63baafd 100644 --- a/tests/test_install_sh.py +++ b/tests/test_install_sh.py @@ -656,7 +656,8 @@ def test_a_command_that_is_a_symlink_is_refused_before_any_request( assert "pipx upgrade openreposhape" in said assert "pipx uninstall openreposhape" in said assert net.calls() == [], net.calls() - assert link.is_symlink() and link.readlink() == packaged + assert link.is_symlink() + assert link.readlink() == packaged if not dangling: assert packaged.read_bytes() == original assert sorted(path.name for path in net.bin.iterdir()) == \ From cb910c41c603c2efc01adb66ccc8d5471240f9b7 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:35:27 +0000 Subject: [PATCH 16/25] src/openreposhape/, tests/test_openreposhape_package.py: the argument loop is one small function per kind of argument, the repeated literals are named once, and the tests change sys.path only through monkeypatch (#563) SonarCloud's findings on #590, none of them in behaviour: - python:S3776: `parse` (cognitive complexity 31) takes each argument through `take` and five helpers, and `main_openreposhape` (22) through `doctor_alone` and `settle_the_names`. The routing is case for case what it was; every refusal keeps its words. - python:S3516: `version` always returned 0; `print_version` returns nothing and the caller exits 0. - python:S1192: `--placement-plan`, `setup-project.py`, `$OPENREPOSHAPE_REF` and `--shape-ref` are constants. - python:S8997 and S9073 in the tests: `package_module` uses `monkeypatch.syspath_prepend` and `monkeypatch.setattr`, and three composite assertions are split. The gate's one failing condition, pythonsecurity:S8705 in `run_child`, is not changed: handing the person's arguments to the standard's own script, unchanged and with no shell, is what the four commands are. Co-Authored-By: Claude Opus 5.5 --- src/openreposhape/cli.py | 232 ++++++++++++++++------------ src/openreposhape/standard.py | 20 ++- tests/test_openreposhape_package.py | 49 +++--- 3 files changed, 170 insertions(+), 131 deletions(-) diff --git a/src/openreposhape/cli.py b/src/openreposhape/cli.py index 618f6f74..b0c958b0 100644 --- a/src/openreposhape/cli.py +++ b/src/openreposhape/cli.py @@ -232,6 +232,11 @@ def value_of(args: list, at: int, flag: str) -> str: return args[at + 1] +#: The doctor's plan flag, in its two spellings. +PLAN_FLAG = "--placement-plan" +PLAN_EQUALS = PLAN_FLAG + "=" + + def take_placement_plan(run: Run, args: list, at: int) -> int: """`--placement-plan ` or `--placement-plan=` after `--doctor`; the index after what it took. The separate word is refused @@ -239,8 +244,8 @@ def take_placement_plan(run: Run, args: list, at: int) -> int: refusal, never last-wins (#546).""" before = run.plan_file arg = args[at] - if arg.startswith("--placement-plan="): - run.plan_file = arg[len("--placement-plan="):] + if arg.startswith(PLAN_EQUALS): + run.plan_file = arg[len(PLAN_EQUALS):] at += 1 else: run.plan_file = args[at + 1] if at + 1 < len(args) else "" @@ -251,17 +256,98 @@ def take_placement_plan(run: Run, args: list, at: int) -> int: at += 1 if not run.plan_file: raise Refusal( - "--placement-plan needs the file to write the plan to: " - "--placement-plan , or --placement-plan= for a name " - "that begins with '-'.") + f"{PLAN_FLAG} needs the file to write the plan to: {PLAN_FLAG} " + f", or {PLAN_EQUALS} for a name that begins with " + "'-'.") if before: raise Refusal(f"two placement plans, '{before}' and " f"'{run.plan_file}'. --doctor writes one: pass " - "--placement-plan once.") - run.doctor_args.append(f"--placement-plan={run.plan_file}") + f"{PLAN_FLAG} once.") + run.doctor_args.append(PLAN_EQUALS + run.plan_file) return at +def take_doctor(run: Run, args: list, at: int) -> int: + """`--doctor []`: the optional path is eaten here, because it would + otherwise be taken for a ; a word beginning with a dash is a + flag, not the path. ONE path, however many `--doctor`s (#546).""" + set_mode(run, "doctor") + at += 1 + if (at < len(args) and not run.doctor_path_given + and not args[at].startswith("-")): + run.doctor_path, run.doctor_path_given = args[at], True + at += 1 + return at + + +def take_doctor_flag(run: Run, args: list, at: int) -> int: + """THE DOCTOR'S TWO FLAGS, ONLY AFTER `--doctor` (#500); anywhere else a + lone flag for setup-project.py, which refuses both.""" + arg = args[at] + if run.mode != "doctor": + run.passthrough.append(arg) + return at + 1 + if arg == "--json": + run.doctor_args.append(arg) + run.doctor_json = True + return at + 1 + return take_placement_plan(run, args, at) + + +def take_value(run: Run, args: list, at: int) -> int: + """A flag and its value: `--org` and `--project` are this command's, and + every other one in `VALUE_FLAGS` travels on, `--shape-ref` noted too.""" + arg = args[at] + value = value_of(args, at, arg) + if arg in ("--org", "--project"): + setattr(run, arg[2:], value) + else: + run.passthrough += [arg, value] + if arg == "--shape-ref": + run.shape_ref = value + return at + 2 + + +def take_word(run: Run, arg: str) -> None: + """A word that is no flag: after `--doctor` and its flags the one path it + has not had, and otherwise the one .""" + if run.mode == "doctor" and not run.doctor_path_given: + run.doctor_path, run.doctor_path_given = arg, True + elif run.project: + raise Refusal(f"two project names, '{run.project}' and '{arg}'. " + "One positional , then flags.") + else: + run.project = arg + + +def take(run: Run, args: list, at: int) -> int: + """The argument at `at`, taken; the index of the next one.""" + arg = args[at] + if arg == "--install": + set_mode(run, "install") + return at + 1 + if arg == "--doctor": + return take_doctor(run, args, at) + if arg in ("--json", PLAN_FLAG) or arg.startswith(PLAN_EQUALS): + return take_doctor_flag(run, args, at) + if arg == "--preflight": + # NAMED for the flag it sets: `--preflight` creates nothing, so the + # organisation and the are not asked for. + run.preflight = True + run.passthrough.append(arg) + return at + 1 + if arg in ("--org", "--project") or arg in VALUE_FLAGS: + return take_value(run, args, at) + if arg == "--": + run.passthrough += ["--", *args[at + 1:]] + return len(args) + if arg.startswith("-"): + run.passthrough.append(arg) + return at + 1 + take_word(run, arg) + return at + 1 + + def parse(args: list) -> tuple: """`(action, run)`: the launcher's argument loop, case for case. `action` is "help" or "version" when one of those was met, which @@ -269,73 +355,15 @@ def parse(args: list) -> tuple: run = Run() at = 0 while at < len(args): - arg = args[at] - if arg in ("-h", "--help"): + if args[at] in ("-h", "--help"): return "help", run - if arg == "--version": + if args[at] == "--version": return "version", run - if arg == "--install": - set_mode(run, "install") - at += 1 - elif arg == "--doctor": - # `--doctor []`: the optional path is eaten here, because - # it would otherwise be taken for a ; a word beginning - # with a dash is a flag, not the path. ONE path, however many - # `--doctor`s (#546). - set_mode(run, "doctor") - at += 1 - if (at < len(args) and not run.doctor_path_given - and not args[at].startswith("-")): - run.doctor_path, run.doctor_path_given = args[at], True - at += 1 - elif arg in ("--json", "--placement-plan") \ - or arg.startswith("--placement-plan="): - # THE DOCTOR'S TWO FLAGS ONLY AFTER `--doctor` (#500); anywhere - # else a lone flag for setup-project.py, which refuses both. - if run.mode != "doctor": - run.passthrough.append(arg) - at += 1 - elif arg == "--json": - run.doctor_args.append("--json") - run.doctor_json = True - at += 1 - else: - at = take_placement_plan(run, args, at) - elif arg == "--preflight": - # NAMED for the flag it sets: `--preflight` creates nothing, so - # the organisation and the are not asked for. - run.preflight = True - run.passthrough.append(arg) - at += 1 - elif arg in ("--org", "--project"): - setattr(run, arg[2:], value_of(args, at, arg)) - at += 2 - elif arg in VALUE_FLAGS: - value = value_of(args, at, arg) - run.passthrough += [arg, value] - if arg == "--shape-ref": - run.shape_ref = value - at += 2 - elif arg == "--": - run.passthrough += ["--", *args[at + 1:]] - break - elif arg.startswith("-"): - run.passthrough.append(arg) - at += 1 - elif run.mode == "doctor" and not run.doctor_path_given: - # After `--doctor` and its flags, the one path it has not had. - run.doctor_path, run.doctor_path_given = arg, True - at += 1 - elif run.project: - raise Refusal(f"two project names, '{run.project}' and '{arg}'. " - "One positional , then flags.") - else: - run.project = arg - at += 1 + at = take(run, args, at) return "run", run -def version() -> int: +def print_version() -> None: """WHICH BYTES A RUN WOULD USE: the version, the tag, the commit, the digest and the clone, with no network and no clone made. The ref in force is in the first line's `( @ )`, as the launcher puts @@ -349,7 +377,7 @@ def version() -> int: "of the standard, not a release") say(f" checkout {beside}: this package sits in it, so every run " "uses it as it stands and clones nothing") - return 0 + return repo = standard.repository() standard.clone_url(repo) wanted = standard.target(stamp) @@ -377,7 +405,6 @@ def version() -> int: else: say(f" checkout {cache}{os.sep}, one clone per commit, " "kept") - return 0 def say_the_standard(std: standard.Standard) -> None: @@ -461,25 +488,49 @@ def forwarded(run: Run) -> list: return args + run.passthrough +def doctor_alone(run: Run) -> None: + """ONE PATH AND THE DOCTOR'S TWO FLAGS, NOTHING ELSE: every other + argument is about a run that CREATES something.""" + if run.project or run.org or run.passthrough: + raise Refusal( + f"--doctor takes a path, --json and {PLAN_FLAG} , and " + "nothing else; it reads a repository and writes nothing into it " + "but that .\n" + f" openRepoShape --doctor [] [--json] [{PLAN_FLAG} " + "]\n" + " To scaffold, drop --doctor. To check this machine, run: " + "openRepoShape --preflight") + + +def settle_the_names(run: Run) -> None: + """NEVER a default organisation: a wrong one creates three repositories + in somebody else's namespace. The flag, else `$OPENREPOSHAPE_ORG`, else + the terminal; and a setup-project.py can ask for only where a + terminal is attached.""" + run.org = run.org or os.environ.get("OPENREPOSHAPE_ORG", "") + if not run.org and terminal_attached(): + try: + run.org = input("organisation to scaffold into: ").strip() + except EOFError: + run.org = "" + if not run.org: + raise Refusal("no organisation to scaffold into. Re-run with " + "--org , or set $OPENREPOSHAPE_ORG.") + if not run.project and not terminal_attached(): + sys.stderr.write(USAGE) + raise Refusal("no given, and no terminal to ask on.") + + def main_openreposhape(args: list) -> int: action, run = parse(args) if action == "help": sys.stdout.write(USAGE) return 0 if action == "version": - return version() + print_version() + return 0 if run.mode == "doctor": - # ONE PATH AND THE DOCTOR'S TWO FLAGS, NOTHING ELSE: every other - # argument is about a run that CREATES something. - if run.project or run.org or run.passthrough: - raise Refusal( - "--doctor takes a path, --json and --placement-plan , " - "and nothing else; it reads a repository and writes nothing " - "into it but that .\n" - " openRepoShape --doctor [] [--json] " - "[--placement-plan ]\n" - " To scaffold, drop --doctor. To check this machine, run: " - "openRepoShape --preflight") + doctor_alone(run) return doctor(run) if run.mode == "install": raise Refusal(INSTALL_REFUSED) @@ -489,20 +540,7 @@ def main_openreposhape(args: list) -> int: sys.stdout.write(USAGE) return 1 if not run.preflight: - # NEVER a default organisation: a wrong one creates three - # repositories in somebody else's namespace. - run.org = run.org or os.environ.get("OPENREPOSHAPE_ORG", "") - if not run.org and terminal_attached(): - try: - run.org = input("organisation to scaffold into: ").strip() - except EOFError: - run.org = "" - if not run.org: - raise Refusal("no organisation to scaffold into. Re-run with " - "--org , or set $OPENREPOSHAPE_ORG.") - if not run.project and not terminal_attached(): - sys.stderr.write(USAGE) - raise Refusal("no given, and no terminal to ask on.") + settle_the_names(run) return scaffold(run) diff --git a/src/openreposhape/standard.py b/src/openreposhape/standard.py index 9fde4f3f..57a471fb 100644 --- a/src/openreposhape/standard.py +++ b/src/openreposhape/standard.py @@ -76,10 +76,16 @@ #: owner/name on github.com. DRIVE_RE = re.compile(r"[A-Za-z]:[\\/]") +#: The entry point the preflight and a scaffold run, and the two names a +#: ref can come by besides the stamp. +SETUP_ENTRY = "setup-project.py" +REF_VARIABLE = "$OPENREPOSHAPE_REF" +REF_FLAG = "--shape-ref" + #: What makes a directory a checkout of this standard for the BESIDE rule: #: the three tools a run reaches for and the naming policy `setup-project.py` #: itself asks for (`is_shape_checkout`). -CHECKOUT_MARKERS = ("setup-project.py", "scaffold-project.py", +CHECKOUT_MARKERS = (SETUP_ENTRY, "scaffold-project.py", "shape-doctor.py", "contracts/repository-naming.yaml") #: The way out of every refusal about a cached clone. The cache is this @@ -214,7 +220,7 @@ def ref_fault(ref: str, what: str) -> str: return "" -def checked_ref(ref: str, what: str = "$OPENREPOSHAPE_REF") -> str: +def checked_ref(ref: str, what: str = REF_VARIABLE) -> str: """`ref`, or a refusal saying what is wrong with it.""" fault = ref_fault(ref, what) if fault: @@ -237,7 +243,7 @@ def __init__(self, ref: str, commit: str, digest: str, @property def overridden(self) -> bool: - return self.source in ("--shape-ref", "$OPENREPOSHAPE_REF") + return self.source in (REF_FLAG, REF_VARIABLE) def target(stamp: Stamp, typed_ref: str = "") -> Target: @@ -246,9 +252,9 @@ def target(stamp: Stamp, typed_ref: str = "") -> Target: last and wins), `$OPENREPOSHAPE_REF` set for this run, the stamped commit, and with none of those main. Each named ref is checked here, before anything reaches git.""" - for ref, source in ((typed_ref, "--shape-ref"), + for ref, source in ((typed_ref, REF_FLAG), (os.environ.get("OPENREPOSHAPE_REF", ""), - "$OPENREPOSHAPE_REF")): + REF_VARIABLE)): if ref: checked_ref(ref, source) commit = ref.lower() if ANY_CASE_COMMIT_RE.fullmatch(ref) else "" @@ -389,12 +395,12 @@ def preflight_entry() -> Path: """The `setup-project.py` `--preflight` runs: the copy the wheel carries beside this file, byte for byte the release's own, or in a checkout the one at its root. Either way no git is needed to reach it.""" - packaged = Path(__file__).resolve().parent / "setup-project.py" + packaged = Path(__file__).resolve().parent / SETUP_ENTRY if packaged.is_file(): return packaged beside = beside_checkout() if beside is not None: - return beside / "setup-project.py" + return beside / SETUP_ENTRY raise Refusal("this package carries no setup-project.py, so there is no " "preflight to run. The install is incomplete: pipx install " "--force openreposhape") diff --git a/tests/test_openreposhape_package.py b/tests/test_openreposhape_package.py index 977993d3..4369a7b4 100644 --- a/tests/test_openreposhape_package.py +++ b/tests/test_openreposhape_package.py @@ -334,7 +334,8 @@ def test_the_committed_stamp_is_unstamped(): "VERSION": "0.0.0", "TAG": "", "COMMIT": "", "DIGEST": ""} -def test_pyproject_takes_its_version_from_the_stamp_and_names_five_commands(): +def test_pyproject_takes_its_version_from_the_stamp_and_names_five_commands( + monkeypatch): """The version is written once, in the stamp; the one dependency is the git-filter-repo CI runs and install.sh installs; the wheel is the package plus `setup-project.py` beside its modules; and each of the five @@ -343,7 +344,8 @@ def test_pyproject_takes_its_version_from_the_stamp_and_names_five_commands(): data = tomllib.loads((REPO / "pyproject.toml").read_text(encoding="utf-8")) project = data["project"] assert project["name"] == "openreposhape" - assert project["dynamic"] == ["version"] and "version" not in project + assert project["dynamic"] == ["version"] + assert "version" not in project assert project["requires-python"] == ">=3.9" assert project["dependencies"] == ["git-filter-repo==2.47.0"] assert 'FILTER_REPO_PIN="git-filter-repo==2.47.0"' in \ @@ -358,7 +360,8 @@ def test_pyproject_takes_its_version_from_the_stamp_and_names_five_commands(): r"(?P.+?)\2", (PACKAGE / "_release.py").read_text(encoding="utf-8"), re.M) - assert found and found.group("version") == "0.0.0" + assert found is not None + assert found.group("version") == "0.0.0" wheel = hatch["build"]["targets"]["wheel"] assert wheel["packages"] == ["src/openreposhape"] assert wheel["force-include"] == { @@ -367,17 +370,11 @@ def test_pyproject_takes_its_version_from_the_stamp_and_names_five_commands(): assert sorted(scripts) == ["adopt-project", "openRepoShape", "scaffold-project", "shape-doctor", "update-shape"] - sys.path.insert(0, str(SOURCE)) - wrote_bytecode, sys.dont_write_bytecode = sys.dont_write_bytecode, True - try: - for name, target in scripts.items(): - module, _, function = target.partition(":") - assert module == "openreposhape.cli", name - assert callable(getattr(importlib.import_module(module), - function)), name - finally: - sys.dont_write_bytecode = wrote_bytecode - sys.path.remove(str(SOURCE)) + cli = package_module("cli", monkeypatch) + for name, target in scripts.items(): + module, _, function = target.partition(":") + assert module == "openreposhape.cli", name + assert callable(getattr(cli, function)), name def test_a_stamp_that_names_part_of_a_release_is_refused(tmp_path): @@ -479,7 +476,8 @@ def test_a_ref_is_checked_before_anything_is_cloned(cmd, tmp_path, ref): assert "$OPENREPOSHAPE_REF" in result.stderr assert "git checkout" in result.stderr assert "clone" not in said(result) - assert "a" * 256 not in result.stderr and "forged" not in result.stderr + assert "a" * 256 not in result.stderr + assert "forged" not in result.stderr def test_a_standard_with_no_doctor_is_refused_by_name(cmd, tmp_path): @@ -786,16 +784,13 @@ def test_with_no_git_a_run_that_needs_the_standard_is_refused(tmp_path): # --- git-filter-repo, for every child ------------------------------------------------- -def package_module(name: str): +def package_module(name: str, monkeypatch): """`openreposhape.` from this checkout's `src/`, imported in this - process with no bytecode written.""" - sys.path.insert(0, str(SOURCE)) - wrote_bytecode, sys.dont_write_bytecode = sys.dont_write_bytecode, True - try: - return importlib.import_module(f"openreposhape.{name}") - finally: - sys.dont_write_bytecode = wrote_bytecode - sys.path.remove(str(SOURCE)) + process with no bytecode written; `sys.path` and the bytecode flag are + put back when the test ends.""" + monkeypatch.syspath_prepend(str(SOURCE)) + monkeypatch.setattr(sys, "dont_write_bytecode", True) + return importlib.import_module(f"openreposhape.{name}") def fake_filter_repo(directory: Path) -> Path: @@ -815,7 +810,7 @@ def test_children_get_the_scripts_directory_first_when_it_holds_filter_repo( tmp_path, monkeypatch): """First on PATH when a `git-filter-repo` is there, and PATH untouched when none is.""" - standard = package_module("standard") + standard = package_module("standard", monkeypatch) scripts = tmp_path / "scripts" scripts.mkdir() monkeypatch.setattr(standard.sysconfig, "get_path", @@ -829,7 +824,7 @@ def test_children_get_the_scripts_directory_first_when_it_holds_filter_repo( def test_a_child_finds_the_git_filter_repo_of_the_packages_environment( - tmp_path, shape_standard): + tmp_path, shape_standard, monkeypatch): """THE PIPX CASE, END TO END: the package runs in an environment of its own whose scripts directory holds `git-filter-repo` and is on nobody's PATH. The tool it starts finds that one, first on its PATH; and before @@ -839,7 +834,7 @@ def test_a_child_finds_the_git_filter_repo_of_the_packages_environment( check=True, capture_output=True) python = venv / ("Scripts/python.exe" if os.name == "nt" else "bin/python") scripts = python.parent - standard = package_module("standard") + standard = package_module("standard", monkeypatch) elsewhere = [entry for entry in os.environ["PATH"].split(os.pathsep) if entry and not standard.filter_repo_in(entry)] site = installed_copy(tmp_path / "package", shape_standard.stamp()) From 9b0a29e7d35a2c0b2fb7c278d1b039a58c5d3254 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:35:27 +0000 Subject: [PATCH 17/25] .github/workflows/tests.yml: the package job runs pipx 1.11.1 on the Python 3.9 it installs with, not the runner's pipx with --python (#563) The job's first run on #590 failed in `pipx install --python <3.9>`: the runner's pipx 1.16.7 runs on the image's Python 3.12 and shares one pip with every environment it makes, and that pip cannot run on 3.9 (`TypeError: dataclass() got an unexpected keyword argument 'slots'`). So the job installs pipx into the 3.9 itself, at 1.11.1, the newest release that installs on 3.9, and its shared pip is then one 3.9 runs. Rehearsed locally on Python 3.9.25: the wheel installs, and the doctor's machine row finds the git-filter-repo pipx installed beside the package. The first run also showed what the plan left to check: the ubuntu image ships pipx (1.16.7), actions/setup-python still offers 3.9 (3.9.25), and two builds of the wheel at the commit's time are one file whose every module is this commit's. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/tests.yml | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index a20503d9..47d9c46d 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -363,8 +363,13 @@ jobs: print(f"{name} = HEAD:{source}") EOF - # pipx AS A PERSON RUNS IT, into directories of this job's own, so the - # runner's own pipx and its apps are left alone, and with the 3.9 above. + # pipx AS A PERSON RUNS IT, into directories of this job's own, and ON + # THE 3.9 ABOVE. Not the runner's own pipx with `--python`: that pipx + # runs on the image's Python and shares ONE pip with every environment + # it makes, a pip that no longer runs on 3.9 (`dataclass(slots=True)`, + # this job's first run, 2026-10-15). So pipx itself runs on 3.9, at + # 1.11.1, the newest release that installs there, and its shared pip + # is then one that 3.9 runs. - name: install the wheel with pipx run: | set -euo pipefail @@ -374,9 +379,9 @@ jobs: } >> "$GITHUB_ENV" export PIPX_HOME="$RUNNER_TEMP/pipx/home" export PIPX_BIN_DIR="$RUNNER_TEMP/pipx/bin" - pipx --version - pipx install --python "$(command -v python)" \ - "$RUNNER_TEMP"/dist-a/*.whl + python -m pip install --disable-pip-version-check pipx==1.11.1 + python -m pipx --version + python -m pipx install "$RUNNER_TEMP"/dist-a/*.whl "$PIPX_HOME/venvs/openreposhape/bin/python" --version ls -l "$PIPX_BIN_DIR" From d53080ae704496e5e2a69bc71dd42a7cbdf7d350 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 14:38:50 +0000 Subject: [PATCH 18/25] .github/workflows/tests.yml: the package job installs pipx from wheels only (#563) SonarCloud githubactions:S8541 on #590: a `pip install` without `--only-binary :all:` may run a package's setup script. pipx 1.11.1 and everything it needs on Python 3.9 come as wheels (checked on 3.9.25: argcomplete, click, packaging, platformdirs, tomli, userpath), so the job says so. The wheel this repository builds is still built from its source, which is what the job is for. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/tests.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 47d9c46d..95f897d9 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -369,7 +369,8 @@ jobs: # it makes, a pip that no longer runs on 3.9 (`dataclass(slots=True)`, # this job's first run, 2026-10-15). So pipx itself runs on 3.9, at # 1.11.1, the newest release that installs there, and its shared pip - # is then one that 3.9 runs. + # is then one that 3.9 runs. Wheels only (`--only-binary :all:`), so + # installing pipx runs no package's setup script. - name: install the wheel with pipx run: | set -euo pipefail @@ -379,7 +380,8 @@ jobs: } >> "$GITHUB_ENV" export PIPX_HOME="$RUNNER_TEMP/pipx/home" export PIPX_BIN_DIR="$RUNNER_TEMP/pipx/bin" - python -m pip install --disable-pip-version-check pipx==1.11.1 + python -m pip install --disable-pip-version-check \ + --only-binary :all: pipx==1.11.1 python -m pipx --version python -m pipx install "$RUNNER_TEMP"/dist-a/*.whl "$PIPX_HOME/venvs/openreposhape/bin/python" --version From 22ade8e3c3f1db2be152159ed6d9dffebc7d2f8e Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 15:52:32 +0000 Subject: [PATCH 19/25] openRepoShape: the comment over `--install`'s record says the target is no link, so `resolved` hands it back unchanged (#563) RV590's nit: "symlinks followed" described nothing once the refusal above it made a linked target impossible. The call stays, because it names the record as `read_recorded_ref` does. Comment only. Co-Authored-By: Claude Opus 5.5 --- openRepoShape | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/openRepoShape b/openRepoShape index 60985343..4c95873c 100755 --- a/openRepoShape +++ b/openRepoShape @@ -406,7 +406,9 @@ install_commands() { keep the package's command, and update it with: pipx upgrade openreposhape or remove the link (pipx uninstall openreposhape, or rm $dir/$name), then run this again." done - # THE RECORD SITS BESIDE THE FILE, symlinks followed, as it is read. + # THE RECORD SITS BESIDE THE FILE. The target is no link (refused just + # above), so `resolved` hands it back unchanged; the call stays because + # it is how `read_recorded_ref` names the record, and the two must agree. record="$(resolved "$dir/openRepoShape")$RECORD_SUFFIX" # BOTH CHECKED BEFORE ANYTHING IS COLLECTED OR PLACED: a ref about to be # written down and read back by every later run, and a record path this From 66bb2c7e2a66e754fa6f13310b4b447656a7fe28 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 16:00:23 +0000 Subject: [PATCH 20/25] src/openreposhape/, pyproject.toml, tests/test_openreposhape_package.py: an annotated tag named in $OPENREPOSHAPE_REF runs its commit, a killed run's half-made clone is swept after an hour, and the clone's fullness, the tag-before-branch order and 128 + the signal are tested (#563) RV590's fix round: - A BUG THE NEW TEST FOUND. `remote_commit` looked for a tag's peeled line (`refs/tags/^{}`), but `git ls-remote -- ` prints that line only when a pattern names it too, so an annotated tag resolved to its tag object; the clone then checked out the tag's commit and `verify` refused the run ("it is at "). `remote_commit` now asks for `` and `^{}`; a ref is letters, digits and `._/-`, so neither pattern holds a wildcard. Shown with protocol v0 and v2. - SHOULD 1: `test_the_clone_is_full` clones through `file://`, where git honours `--depth`, and checks `--is-shallow-repository` is false and every commit of the synthetic standard is in the clone. - SHOULD 2: the bare-run test runs DETACHED, with the logging `git` first on PATH and the repository nowhere. - SHOULD 3: `PythonCommand.detached` sets `$OPENREPOSHAPE_REPO` nowhere as `__call__` does, and `__call__` runs in the test's directory; the two tests that read the default repository's name set it empty, which is unset (`${VAR:-default}`). - SHOULD 4: the usage says "run from a checkout's src/ (an editable install)". - SHOULD 5: `sweep_incoming` removes an `.incoming-*` directory nothing has touched for an hour before a clone is made, never following a link and never refusing over one it cannot remove; tested with one two hours old (removed) and one fresh (kept). - SHOULD 6: the pyproject comment says only what the package job shows. - Nits: `test_a_ref_that_is_a_tag_and_a_branch_is_the_tag` (a lightweight tag over a branch of its name, and an annotated tag over one) and `test_a_tool_killed_by_a_signal_exits_128_plus_the_signal` (POSIX: SIGTERM is 143). The stub sends itself `$STUB_SIGNAL` where it is set. Each new test fails on its mutation: `--depth 2` (shallow 'true'), the sweep removed (leftover kept), branches before tags, the peeled pattern dropped, and the raw status returned (241). Co-Authored-By: Claude Opus 5.5 --- pyproject.toml | 5 +- src/openreposhape/cli.py | 2 +- src/openreposhape/standard.py | 40 +++++++++- tests/test_openreposhape_package.py | 116 ++++++++++++++++++++++++---- 4 files changed, 143 insertions(+), 20 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index e2b69c0c..7e1a364b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,8 +24,9 @@ # depend on that day. 1.27.0 is the newest hatchling that still runs on # Python 3.9, the oldest Python this package supports (1.28.0 and later need # 3.10). Hatchling, because `force-include` below puts `setup-project.py` -# into the package declaratively, and its wheels are reproducible from -# `SOURCE_DATE_EPOCH`. +# into the package declaratively. Its wheels take their file times from +# `SOURCE_DATE_EPOCH`, and the `package` job builds twice on one runner at +# the commit's time and compares the two files; that is all it shows. requires = ["hatchling==1.27.0"] build-backend = "hatchling.build" diff --git a/src/openreposhape/cli.py b/src/openreposhape/cli.py index b0c958b0..b92e9cd9 100644 --- a/src/openreposhape/cli.py +++ b/src/openreposhape/cli.py @@ -74,7 +74,7 @@ computed by the clone's own scripts/repo_shape.py), so a project scaffolded from it pins exactly the release you installed. `--version` names the release, its commit, its digest and the clone. A build that is not a release -follows main; installed from a checkout of the standard, it uses that +follows main; run from a checkout's src/ (an editable install), it uses that checkout as it stands. $OPENREPOSHAPE_REPO owner/name, URL or path to clone (default opensoft/openRepoShape) diff --git a/src/openreposhape/standard.py b/src/openreposhape/standard.py index 57a471fb..6dbe40b1 100644 --- a/src/openreposhape/standard.py +++ b/src/openreposhape/standard.py @@ -414,9 +414,13 @@ def remote_commit(url: str, repo: str, ref: str, env: dict) -> str: A TAG FIRST, then a branch, then the name as given (`HEAD`, or a full `refs/...`): `git checkout ` in a fresh clone resolves it in that order, and the two must not disagree. An annotated tag is read through - to its commit (`^{}`). + to its commit (`^{}`), and that line is asked for BY NAME: `git + ls-remote` given a name prints a tag's peeled line only when a pattern + names it too, and the tag's own line names a tag object, not a commit. + (A ref here is letters, digits, `.`, `_`, `/` and `-`, so neither + pattern holds a wildcard.) """ - proc = git(["ls-remote", "--", url, ref], env) + proc = git(["ls-remote", "--", url, ref, f"{ref}^{{}}"], env) if proc.returncode != 0: raise Refusal( f"could not ask {repo} which commit {ref} is (`git ls-remote` " @@ -450,6 +454,31 @@ def move_into_place(incoming: Path, final: Path) -> None: time.sleep(0.2) +#: Where a clone is made before it is moved into place, and how long one +#: may sit untouched before a later run takes it for the leftover of a run +#: that was killed mid-clone. +INCOMING_PREFIX = ".incoming-" +INCOMING_STALE_SECONDS = 3600 + + +def sweep_incoming(cache: Path) -> None: + """Remove what a KILLED run left: an `.incoming-*` directory nothing has + touched for an hour. A run that fails, or is interrupted with Ctrl-C, + removes its own (`clone_into`'s `finally`); one ended by SIGTERM, + SIGHUP or SIGKILL cannot, and its half-made clone would otherwise sit in + the cache for good. An hour, so a clone another run is making right now + is left alone. A link is never followed, and a directory that cannot be + removed is left for the next run: this is tidying, never a refusal.""" + cutoff = time.time() - INCOMING_STALE_SECONDS + for leftover in cache.glob(INCOMING_PREFIX + "*"): + try: + if (not leftover.is_symlink() and leftover.is_dir() + and leftover.stat().st_mtime < cutoff): + remove_tree(leftover) + except OSError: + continue + + def clone_into(cache: Path, url: str, repo: str, wanted: Target, commit: str, env: dict) -> Path: """A full clone of `url` at `commit` (or at `wanted.ref`, resolved by the @@ -458,11 +487,14 @@ def clone_into(cache: Path, url: str, repo: str, wanted: Target, CLONED BESIDE ITS PLACE AND MOVED IN WHOLE. A clone that failed half way, or a run interrupted, leaves nothing at `/` for the next run to mistake for a clone; and of two runs cloning at once, the one that - moves second uses the first one's and discards its own. + moves second uses the first one's and discards its own. What a killed + run left beside it is swept first (`sweep_incoming`). """ try: cache.mkdir(parents=True, exist_ok=True) - incoming = Path(tempfile.mkdtemp(prefix=".incoming-", dir=str(cache))) + sweep_incoming(cache) + incoming = Path(tempfile.mkdtemp(prefix=INCOMING_PREFIX, + dir=str(cache))) except OSError as exc: raise Refusal(f"could not make a directory under {cache}, where " f"this command keeps its clones of the standard: " diff --git a/tests/test_openreposhape_package.py b/tests/test_openreposhape_package.py index 4369a7b4..a262fee8 100644 --- a/tests/test_openreposhape_package.py +++ b/tests/test_openreposhape_package.py @@ -43,8 +43,10 @@ import os import re import shutil +import signal import subprocess import sys +import time from pathlib import Path import pytest @@ -116,7 +118,8 @@ #: Every stub: one line, `STUB {json}`, naming the file, the ref it was #: committed at, its arguments, where it ran, what it finds of #: git-filter-repo, the first PATH entry and the hand-over variable; then -#: `$STUB_EXIT`, so a test sees an exit status travel back. +#: `$STUB_SIGNAL` sent to itself where it is set, or `$STUB_EXIT`, so a test +#: sees an exit status travel back. STUB = '''\ import json, os, shutil, sys print("STUB " + json.dumps({ @@ -125,6 +128,8 @@ "path0": os.environ.get("PATH", "").split(os.pathsep)[0], "self_bootstrap": os.environ.get("OPENREPOSHAPE_SELF_BOOTSTRAP")}), flush=True) +if os.environ.get("STUB_SIGNAL"): + os.kill(os.getpid(), int(os.environ["STUB_SIGNAL"])) sys.exit(int(os.environ.get("STUB_EXIT", "0"))) ''' @@ -190,23 +195,29 @@ class PythonCommand: `command` mean nothing here, because this command fetches no `setup.sh` and has no file of its own to run. - Called, `$OPENREPOSHAPE_REPO` names a repository that does not exist, - unless the test names one: beside this checkout nothing is cloned, and a - run that failed to see the checkout would fail on disk rather than ask - github.com where main is.""" + Either way `$OPENREPOSHAPE_REPO` names a repository that does not exist, + unless the test names one, so a run that reached for a clone it should + not need fails on disk rather than asking github.com where main is; and + either way it runs in the test's own directory, so a scaffold that lost + its `--into` lands there and never in this checkout.""" usage_lines = (USAGE_FIRST, *LAUNCHER_USAGE[1:]) def __init__(self, tmp_path: Path) -> None: self.tmp_path = tmp_path + def nowhere(self, env: dict | None) -> dict: + """`env` over a `$OPENREPOSHAPE_REPO` that does not exist.""" + return {"OPENREPOSHAPE_REPO": str(self.tmp_path / "nowhere.git"), + **(env or {})} + def __call__(self, *args: str, home: Path | None = None, env: dict | None = None, setup_sh: Path | None = None, command: Path | None = None) -> subprocess.CompletedProcess: - nowhere = {"OPENREPOSHAPE_REPO": str(self.tmp_path / "nowhere.git")} return run_package(SOURCE, "openreposhape", args, package_env(self.tmp_path, home, - {**nowhere, **(env or {})})) + self.nowhere(env)), + cwd=self.tmp_path) def detached(self, *args: str, home: Path | None = None, env: dict | None = None, @@ -215,8 +226,8 @@ def detached(self, *args: str, home: Path | None = None, if not site.is_dir(): installed_copy(self.tmp_path / "package") return run_package(site, "openreposhape", args, - package_env(self.tmp_path, home, env, - detached=True), + package_env(self.tmp_path, home, + self.nowhere(env), detached=True), cwd=cwd or self.tmp_path) @@ -411,7 +422,9 @@ def test_version_names_the_release_and_clones_nothing(tmp_path, def test_version_says_an_unstamped_build_follows_main(cmd): - result = cmd.detached("--version") + """`$OPENREPOSHAPE_REPO` set EMPTY is unset, the shell's + `${VAR:-default}`: the default repository, named.""" + result = cmd.detached("--version", env={"OPENREPOSHAPE_REPO": ""}) assert result.returncode == 0, result.stderr lines = result.stdout.splitlines() assert lines[0] == "openRepoShape 0.0.0 (opensoft/openRepoShape @ main)" @@ -444,7 +457,9 @@ def test_the_install_line_keeps_the_launchers_description_column(cmd): def test_a_bare_run_asks_nothing_of_git_gh_or_curl(cmd, tmp_path): """The usage, exit 1, and NO CALL: a `git`, a `gh` and a `curl` that log every call stand first on PATH (a POSIX script and a `.cmd` each), - and the log must not exist afterwards.""" + and the log must not exist afterwards. DETACHED, where the standard is + a clone and so a git call away, with the repository nowhere: a bare run + that reached for its standard would log a `git ls-remote` here.""" log = tmp_path / "calls.log" fakes = tmp_path / "fakes" fakes.mkdir() @@ -455,7 +470,8 @@ def test_a_bare_run_asks_nothing_of_git_gh_or_curl(cmd, tmp_path): posix.chmod(0o755) (fakes / f"{name}.cmd").write_bytes( f'@echo {name} %* >> "{log}"\r\n@exit /b 1\r\n'.encode("utf-8")) - result = cmd(env={"PATH": f"{fakes}{os.pathsep}{os.environ['PATH']}"}) + result = cmd.detached( + env={"PATH": f"{fakes}{os.pathsep}{os.environ['PATH']}"}) assert result.returncode == 1, result.stdout + result.stderr assert USAGE_FIRST in result.stdout.splitlines() assert not log.exists(), log.read_text(encoding="utf-8") @@ -500,7 +516,8 @@ def test_a_standard_with_no_doctor_is_refused_by_name(cmd, tmp_path): def test_version_names_the_ref_set_for_the_run(cmd): - result = cmd.detached("--version", env={"OPENREPOSHAPE_REF": "v1.2.3"}) + result = cmd.detached("--version", env={"OPENREPOSHAPE_REF": "v1.2.3", + "OPENREPOSHAPE_REPO": ""}) assert result.returncode == 0, result.stderr assert result.stdout.splitlines()[0] == \ "openRepoShape 0.0.0 (opensoft/openRepoShape @ v1.2.3)" @@ -533,6 +550,47 @@ def test_a_stamped_run_clones_its_commit_once_and_runs_from_it( assert git("rev-parse", "HEAD", cwd=clone).stdout.strip() == commit +def test_the_clone_is_full(tmp_path, shape_standard): + """A FULL clone, never a shallow one: the doctor's currency row compares + commits, and `update-shape.py` reads the standard's history. Cloned + through a `file://` URL, because git ignores `--depth` in a clone from a + plain path, so only a URL would show a shallow one: every commit of the + standard is there, the stamped commit's parent and the other branch's + included.""" + result = stamped_run( + tmp_path, shape_standard, ["--doctor", "."], + env={"OPENREPOSHAPE_REPO": shape_standard.bare.as_uri()}) + assert result.returncode == 0, result.stdout + result.stderr + clone = cache_dir(tmp_path) / shape_standard.stamp()["commit"] + assert git("rev-parse", "--is-shallow-repository", + cwd=clone).stdout.strip() == "false" + for ref, commit in shape_standard.commits.items(): + assert git("cat-file", "-e", commit, cwd=clone, + check=False).returncode == 0, f"{ref} ({commit}) is missing" + + +def test_a_clone_a_killed_run_left_is_swept_once_an_hour_has_passed( + tmp_path, shape_standard): + """A run killed mid-clone leaves its `.incoming-*` directory beside the + clones. The next run that clones removes one nothing has touched for an + hour, and leaves a newer one alone: another run may be cloning into it + right now.""" + cache = cache_dir(tmp_path) + old, new = cache / ".incoming-old", cache / ".incoming-new" + for leftover in (old, new): + (leftover / ".git").mkdir(parents=True) + (leftover / ".git" / "HEAD").write_text("ref: refs/heads/main\n", + encoding="utf-8") + two_hours_ago = time.time() - 2 * 3600 + os.utime(old, (two_hours_ago, two_hours_ago)) + result = stamped_run(tmp_path, shape_standard, ["--doctor", "."]) + assert result.returncode == 0, result.stdout + result.stderr + assert not old.exists(), "the killed run's leftover was not swept" + assert (new / ".git" / "HEAD").is_file(), "a fresh one was swept" + assert sorted(path.name for path in cache.iterdir()) == sorted( + [new.name, shape_standard.stamp()["commit"]]) + + def test_a_clone_whose_digest_is_not_the_stamped_one_is_refused( tmp_path, shape_standard): """The digest is the clone's own `repo_shape.tree_digest`, compared with @@ -615,6 +673,24 @@ def test_the_variable_overrides_the_stamp_for_one_run(tmp_path, "instead of the commit above; no digest is checked" in version.stdout +def test_a_ref_that_is_a_tag_and_a_branch_is_the_tag(tmp_path): + """A name that is a tag AND a branch is the tag, which is what `git + checkout` in a fresh clone makes of it; an annotated tag is read through + to its commit (`^{}`), never taken for one.""" + standard = make_standard(tmp_path / "both") + commits = standard.commits + git("branch", PINNED, commits[OTHER], cwd=standard.bare) + git("tag", "-a", "-m", "annotated", "annotated", commits[OTHER], + cwd=standard.bare) + git("branch", "annotated", commits["main"], cwd=standard.bare) + site = installed_copy(tmp_path / "package", standard.stamp()) + for ref, at in ((PINNED, PINNED), ("annotated", OTHER)): + result = stamped_run(tmp_path, standard, ["--doctor", "."], + site=site, env={"OPENREPOSHAPE_REF": ref}) + assert result.returncode == 0, result.stdout + result.stderr + assert stub_report(result.stdout)["at"] == at, ref + + def test_an_unstamped_build_follows_main(tmp_path): """Each run asks the repository where main is: a commit landing there is the next run's standard, in a clone of its own.""" @@ -666,6 +742,20 @@ def test_each_tool_command_runs_its_file_with_every_argument_unchanged( assert Path(report["cwd"]).resolve() == typed_in.resolve() +@pytest.mark.skipif(os.name == "nt", reason="a POSIX signal's exit status: " + "Windows ends a process with the status it is handed") +def test_a_tool_killed_by_a_signal_exits_128_plus_the_signal( + tmp_path, shape_standard): + """As a shell reports it: a tool ended by SIGTERM exits 143, never the + -15 `subprocess` reports, which an exit status would wrap to 241.""" + result = stamped_run(tmp_path, shape_standard, ["--json"], + entry="shape_doctor", + env={"STUB_SIGNAL": str(int(signal.SIGTERM))}) + assert result.returncode == 128 + signal.SIGTERM, ( + result.stdout + result.stderr) + assert stub_report(result.stdout)["tool"] == "shape-doctor.py" + + def test_a_scaffold_hands_over_to_the_clones_setup_project(tmp_path, shape_standard): """setup.sh's hand-over from a clone it made: `--into` where the From 6179bd780bb9dc3127946f30415030977501fe8b Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 16:00:23 +0000 Subject: [PATCH 21/25] tests/test_repo_hygiene.py: every module of src/openreposhape/ is pure ASCII (#563) RV590's nit: the package's `openRepoShape` prints its usage and every refusal to the console the command was typed in, which on a stock Windows is PowerShell 5.1's, so its modules are held to `setup-project.py`'s rule by a new test parametrized over the directory. The existing test is unchanged. Co-Authored-By: Claude Opus 5.5 --- tests/test_repo_hygiene.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index d1e35997..5ded8060 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -1475,6 +1475,21 @@ def test_setup_project_py_is_pure_ascii(name): f"`grep -nP '[^\\x00-\\x7F]' {name}`") +@pytest.mark.parametrize( + "path", sorted((REPO / "src" / "openreposhape").glob("*.py")), + ids=lambda p: p.name) +def test_the_package_is_pure_ascii(path): + """The openreposhape package (#563), held to `setup-project.py`'s rule + for the same reason: its `openRepoShape` prints its usage and every + refusal to whatever console the command was typed in, and on a stock + Windows that is PowerShell 5.1's. Its wheel carries `setup-project.py` + as well, which the test above already holds.""" + name = path.relative_to(REPO).as_posix() + assert path.read_bytes().isascii(), ( + f"{name} must be pure ASCII; find the offending line with " + f"`grep -nP '[^\\x00-\\x7F]' {name}`") + + #: THE ONE LINE THAT INSTALLS THE COMMANDS THIS REPOSITORY NO LONGER SHIPS #: (#92). `park` and `resume` moved to opensoft/openRepoTools, and what is #: left here is a pointer at its installer: printed by `openRepoShape From d090f0c50ea3669d95c796e3623bd183fa0d54bc Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 16:14:28 +0000 Subject: [PATCH 22/25] src/openreposhape/standard.py, tests/test_openreposhape_package.py: an annotated tag named in full, `refs/tags/`, in $OPENREPOSHAPE_REF runs its commit (#563) RV590's re-check of 6179bd78: `remote_commit` looked a full name up as given, never as `^{}`, so `OPENREPOSHAPE_REF=refs/tags/` took the tag object for the commit and every run was refused ("... should be this command's clone of the standard at , and it is at "). The docstring says a full `refs/...` name works, and the bash launcher accepts that spelling. The lookup now tries `^{}` before ``; `git ls-remote` already prints that line, because the peeled pattern asks for it. `test_a_ref_that_is_a_tag_and_a_branch_is_the_tag` gains the case `refs/tags/annotated`, which failed before the fix with that refusal. Co-Authored-By: Claude Opus 5.5 --- src/openreposhape/standard.py | 12 ++++++------ tests/test_openreposhape_package.py | 6 ++++-- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/src/openreposhape/standard.py b/src/openreposhape/standard.py index 6dbe40b1..af803327 100644 --- a/src/openreposhape/standard.py +++ b/src/openreposhape/standard.py @@ -414,11 +414,11 @@ def remote_commit(url: str, repo: str, ref: str, env: dict) -> str: A TAG FIRST, then a branch, then the name as given (`HEAD`, or a full `refs/...`): `git checkout ` in a fresh clone resolves it in that order, and the two must not disagree. An annotated tag is read through - to its commit (`^{}`), and that line is asked for BY NAME: `git - ls-remote` given a name prints a tag's peeled line only when a pattern - names it too, and the tag's own line names a tag object, not a commit. - (A ref here is letters, digits, `.`, `_`, `/` and `-`, so neither - pattern holds a wildcard.) + to its commit (`^{}`), named short or in full (`refs/tags/`), and + that line is asked for BY NAME: `git ls-remote` given a name prints a + tag's peeled line only when a pattern names it too, and the tag's own + line names a tag object, not a commit. (A ref here is letters, digits, + `.`, `_`, `/` and `-`, so neither pattern holds a wildcard.) """ proc = git(["ls-remote", "--", url, ref, f"{ref}^{{}}"], env) if proc.returncode != 0: @@ -432,7 +432,7 @@ def remote_commit(url: str, repo: str, ref: str, env: dict) -> str: sha, _, name = line.partition("\t") found[name.strip()] = sha.strip().lower() for name in (f"refs/tags/{ref}^{{}}", f"refs/tags/{ref}", - f"refs/heads/{ref}", ref): + f"refs/heads/{ref}", f"{ref}^{{}}", ref): if COMMIT_RE.fullmatch(found.get(name, "")): return found[name] return "" diff --git a/tests/test_openreposhape_package.py b/tests/test_openreposhape_package.py index a262fee8..99c753ff 100644 --- a/tests/test_openreposhape_package.py +++ b/tests/test_openreposhape_package.py @@ -676,7 +676,8 @@ def test_the_variable_overrides_the_stamp_for_one_run(tmp_path, def test_a_ref_that_is_a_tag_and_a_branch_is_the_tag(tmp_path): """A name that is a tag AND a branch is the tag, which is what `git checkout` in a fresh clone makes of it; an annotated tag is read through - to its commit (`^{}`), never taken for one.""" + to its commit (`^{}`), never taken for one, by its short name and by + its full `refs/tags/` name alike.""" standard = make_standard(tmp_path / "both") commits = standard.commits git("branch", PINNED, commits[OTHER], cwd=standard.bare) @@ -684,7 +685,8 @@ def test_a_ref_that_is_a_tag_and_a_branch_is_the_tag(tmp_path): cwd=standard.bare) git("branch", "annotated", commits["main"], cwd=standard.bare) site = installed_copy(tmp_path / "package", standard.stamp()) - for ref, at in ((PINNED, PINNED), ("annotated", OTHER)): + for ref, at in ((PINNED, PINNED), ("annotated", OTHER), + ("refs/tags/annotated", OTHER)): result = stamped_run(tmp_path, standard, ["--doctor", "."], site=site, env={"OPENREPOSHAPE_REF": ref}) assert result.returncode == 0, result.stdout + result.stderr From 1c8a708565e723cd3741486a71521f230f82b05a Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Thu, 15 Oct 2026 20:37:13 +0000 Subject: [PATCH 23/25] .github/workflows/tests.yml, tests/test_openreposhape_package.py: the package job's comment and the package tests' docstring say the suite runs on Linux since #597 (#563) Comment and docstring only. #597 removed tests-macos and tests-windows, so the package job's comment no longer speaks of "the three jobs above" and "all three runners", and names the three systems the release workflow installs on. The package tests' docstring no longer says its cases run on a Windows job: CI runs the module on Linux, and no CI job runs it on Windows. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/tests.yml | 10 ++++++---- tests/test_openreposhape_package.py | 7 +++---- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 527a3b65..77cf6bb2 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -131,14 +131,16 @@ jobs: # THE openreposhape PACKAGE, BUILT AND INSTALLED AS A PERSON INSTALLS IT # (#563). A JOB OF ITS OWN, AND NO PYTEST IN IT: the build backend # `pyproject.toml` pins (hatchling) and git-filter-repo, the package's one - # dependency, both come from PyPI, and the suite in the three jobs above + # dependency, both come from PyPI, and the suite in the `tests` job above # never touches the network. What can be checked offline is checked there, - # on all three runners (`tests/test_openreposhape_package.py`); this job is - # what only a real build and a real pipx can say. + # on Linux like the rest of the suite since #597 + # (`tests/test_openreposhape_package.py`); this job is what only a real + # build and a real pipx can say. # # UBUNTU, ON THE OLDEST PYTHON THE PACKAGE SUPPORTS, 3.9. The wheel is pure # Python, one file for every system; the release workflow (#563's next - # pull request) installs it on all three before it uploads anything. + # pull request) installs it on Linux, macOS and Windows before it uploads + # anything. package: runs-on: ubuntu-latest steps: diff --git a/tests/test_openreposhape_package.py b/tests/test_openreposhape_package.py index 99c753ff..5273f1bb 100644 --- a/tests/test_openreposhape_package.py +++ b/tests/test_openreposhape_package.py @@ -9,10 +9,9 @@ so `the_python_command`, autouse here and only here, points both at the package's `openRepoShape` for each test this module collects: one case, one rule, both commands, and not a line of the launcher's module changed for -it. This module carries no Windows skip, so those cases run on the Windows -job too, where the launcher's own module skips. The four launcher cases that -run its FILE (from stdin, or by its path) are re-stated below for this -command. +it. CI runs this module on Linux, as it runs the whole suite since #597, +and no CI job runs it on Windows. The four launcher cases that run its FILE +(from stdin, or by its path) are re-stated below for this command. `PythonCommand`, called, runs the package from this checkout's `src/`, where it sits BESIDE a checkout of the standard and uses this one, as the From 4410da00ad100977ba8d8e8108786625fde6e6cf Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 16 Oct 2026 09:49:51 +0000 Subject: [PATCH 24/25] tests/test_install_sh.py, tests/test_openreposhape_command.py: the two symlink refusal cases spell paths as Git Bash does, so they pass in tests-git-bash (#563, #585) Lane openRepoShape-5's work, applied as lane 5 wrote it: #596 (#585) landed first, so #590, landing second, carries these hunks. In both cases the refusal is matched against `bash_path(link)`, the launcher's case hands `OPENREPOSHAPE_BIN_DIR` over as `bash_path(bin_dir)`, and a link read back has Windows' `\\?\` prefix stripped before it is compared. `bash_path` is #596's, in tests/conftest.py, and returns the path as it is off Windows. Lane 5's evidence: on Windows (probe run 38769166462, the merge of 01e8c2c with this branch) the four cases failed without the hunks and both suites gave 117 passed with them; on Linux, 117 passed. Here, on Linux: the two suites give 117 passed, with and without Path.write_text writing CRLF. Co-Authored-By: Claude Opus 5.5 --- tests/test_install_sh.py | 6 ++++-- tests/test_openreposhape_command.py | 9 +++++---- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/tests/test_install_sh.py b/tests/test_install_sh.py index 28f158b2..2c69f66c 100644 --- a/tests/test_install_sh.py +++ b/tests/test_install_sh.py @@ -671,12 +671,14 @@ def test_a_command_that_is_a_symlink_is_refused_before_any_request( result = net.run() assert result.returncode == 2, result.stdout + result.stderr said = " ".join(result.stderr.split()) - assert f"{link} is a symbolic link, so nothing was installed" in said, said + assert f"{bash_path(link)} is a symbolic link, so nothing was installed" \ + in said, said assert "pipx upgrade openreposhape" in said assert "pipx uninstall openreposhape" in said assert net.calls() == [], net.calls() assert link.is_symlink() - assert link.readlink() == packaged + # Windows reads a link back with `\\?\` ahead of its target. + assert str(link.readlink()).removeprefix("\\\\?\\") == str(packaged) if not dangling: assert packaged.read_bytes() == original assert sorted(path.name for path in net.bin.iterdir()) == \ diff --git a/tests/test_openreposhape_command.py b/tests/test_openreposhape_command.py index 35c2dcd4..4f6f2ea6 100644 --- a/tests/test_openreposhape_command.py +++ b/tests/test_openreposhape_command.py @@ -407,17 +407,18 @@ def test_install_refuses_a_target_that_is_a_symlink(tmp_path, dangling): link = bin_dir / "openRepoShape" link.symlink_to(packaged) result = run_cmd("--install", home=tmp_path, - env={"OPENREPOSHAPE_BIN_DIR": str(bin_dir), + env={"OPENREPOSHAPE_BIN_DIR": bash_path(bin_dir), "OPENREPOSHAPE_REF": "v9.9.9", "OPENREPOSHAPE_RECORD_REF": "1"}) assert result.returncode == 2, result.stdout + result.stderr said = " ".join(result.stderr.split()) - assert f"{link} is a symbolic link, so --install will not copy this " \ - "command through it" in said, said + assert f"{bash_path(link)} is a symbolic link, so --install will not " \ + "copy this command through it" in said, said assert "pipx upgrade openreposhape" in said assert "pipx uninstall openreposhape" in said assert link.is_symlink() - assert os.readlink(link) == str(packaged) + # Windows reads a link back with `\\?\` ahead of its target. + assert os.readlink(link).removeprefix("\\\\?\\") == str(packaged) if not dangling: assert packaged.read_bytes() == original assert not packaged.with_name("openRepoShape.ref").exists() From 70955e4dd3861f0f58c146807bac796cd4dba744 Mon Sep 17 00:00:00 2001 From: Brett Heap Date: Fri, 16 Oct 2026 09:54:00 +0000 Subject: [PATCH 25/25] .github/workflows/tests.yml: the header says what is the `package` job's own, that what it checks needs PyPI, not that it alone reaches PyPI (#563) Comment only, three lines for three. `tests` and `tests-git-bash` both pip-install pytest 8.3.4, and `tests` git-filter-repo too, so "the one job that reaches PyPI" was not true. What is the package job's own is that it builds the wheel and installs it with pipx, which needs the build backend and git-filter-repo from PyPI. The job's own comment below says only that those two come from PyPI and that the suite in `tests` never touches the network, which holds, and is unchanged. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/tests.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 266c62c7..7928afaf 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -12,9 +12,9 @@ name: tests # below runs the launcher's and the installer's suites, and nothing else, on # Windows under Git Bash. # -# The `package` job at the end is the one job that reaches PyPI, and it runs -# no pytest: building the openreposhape wheel and installing it with pipx -# need the build backend and git-filter-repo from there (#563). +# The `package` job at the end runs no pytest. It builds the openreposhape +# wheel and installs it with pipx, so what it checks needs PyPI (the build +# backend, git-filter-repo; #563). The other jobs take only test tools there. on: push: