diff --git a/AGENTS.md b/AGENTS.md index 44a31c7a..23e390bc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,13 +25,13 @@ types `yes` at its prompt; you never answer it for them. No terminal: section 2. **Never answer an install prompt.** The preflight asks `Type yes to continue:` for each thing it offers to run — an install, `gh auth login`, `gh auth setup-git`. The human types it; you never install software or log in for them. +The one-line install (`install.sh`) is theirs to run: relay it, never run it. **Never pass `--allow-upstream-org` on your own initiative.** It guards only -`--org opensoft` itself — scaffolding into opensoft's own namespace, the -upstream owner of openRepoShape. If setup.sh refuses because the organisation -you were given resolves to opensoft, STOP and relay what it said; the fix is -almost always an explicit `--org `. Pass `--allow-upstream-org` -only if the human asks for it in those words. +`--org opensoft`, the namespace of openRepoShape's upstream owner. If setup.sh +refuses because the organisation you were given resolves to opensoft, STOP and +relay what it said; the fix is almost always an explicit `--org `. +Pass `--allow-upstream-org` only if the human asks for it in those words. **Never bypass an organisation ruleset.** A ruleset that refuses a direct push to `main` is doing its job; the exit is in the tool's own failure message (for diff --git a/README.md b/README.md index 36453f70..d1641310 100644 --- a/README.md +++ b/README.md @@ -102,10 +102,10 @@ is on no `PATH` by default, and its installer finishes by printing the line that puts it there — run that line, or the next shell cannot find `brew`. Then `brew install gh`, and `brew install git-filter-repo` only if you are ADOPTING an existing repository rather than scaffolding a new one. The stock `/bin/bash` is -3.2 and is enough: `setup.sh` and the `openRepoShape` command are written to use -no syntax newer than that, though CI runs on Linux only and no run of it checks -them against bash 3.2. That the login shell is zsh changes nothing — both entry -points are run under `bash` explicitly. +3.2 and is enough: `setup.sh`, `install.sh` and the `openRepoShape` command are +written to use no syntax newer than that, though CI runs on Linux only and no +run of it checks them against bash 3.2. That the login shell is zsh changes +nothing — all three entry points are run under `bash` explicitly. @@ -174,14 +174,14 @@ run the four steps below INSIDE the WSL shell. > opens with a preflight that names what is missing and, for `gh`, its login and > its credential helper, prints the command for THIS machine and asks once — > `Type yes to continue:`, the same words it asks everything else with — then -> runs it and carries on. Nothing is installed without that typed `yes`; `--yes` -> does not answer it, and with no terminal attached there is no offer at all. -> Homebrew is the one install never run for you: on a Mac without `brew` the -> preflight names and stops there. To check a machine and -> stop, creating nothing, run `openRepoShape --preflight` — `py setup-project.py -> --preflight` on Windows — which exits 0 when everything is present and 1 when -> it is not (`openRepoShape --preflight` exits 2 when `git` or a Python 3.9 or -> newer is missing). +> runs it and carries on. The preflight installs nothing without that typed +> `yes`; `--yes` does not answer it, and with no terminal attached there is no +> offer at all. Homebrew is the one install never run for you: on a Mac without +> `brew` the preflight names and stops there. To check a +> machine and stop, creating nothing, run `openRepoShape --preflight` — +> `py setup-project.py --preflight` on Windows — which exits 0 when everything +> is present and 1 when it is not (`openRepoShape --preflight` exits 2 when +> `git` or a Python 3.9 or newer is missing). **1. Install the GitHub CLI**, `gh`. The official instructions are at (and ): @@ -225,7 +225,22 @@ offer to configure git's credential helper, or run `gh auth setup-git`: the first commits go up with a plain `git push` over HTTPS, and without it the three repositories are created and the push fails. -**3. Install the `openRepoShape` command**, once: +**3. Install the `openRepoShape` command**, once, in one line: + +```sh +curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +``` + +`install.sh` is attached to every release and installs the newest, with no `gh` +and no login. Its command is PINNED to the release tag, recorded in +`openRepoShape.ref` beside it: `setup.sh` and the standard are fetched at that +tag, not at main. It installs `git-filter-repo` with `pipx` where it can, or +prints the command, and ends with `openRepoShape --preflight`. `install.sh` +itself never runs `sudo`; the preflight it ends with offers what is missing, +each behind a typed `yes` at your terminal. `| bash -s -- v1.0.1` names a +release; the same line again upgrades. + +By hand, the same command at main, which records no ref and follows main: ```sh gh api repos/opensoft/openRepoShape/contents/openRepoShape \ @@ -239,7 +254,7 @@ curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRep | bash -s -- --install ``` -It installs ONE command into `~/.local/bin`, idempotently — a second run +Each way installs ONE command into `~/.local/bin`, idempotently — a second run prints `unchanged` — and prints the `export PATH=…` line if that directory is not on `PATH`. The estate verbs `park` and `resume`, for [carrying in-flight work](#carrying-in-flight-work-to-another-workstation), are @@ -380,7 +395,13 @@ records as whose act the election was, and it defaults to the `gh` login. #### Install the command -Two ways in, the short one first. Install `openRepoShape` as a command: +Two ways in, the short one first. Install the command as step 3 above does: + +```sh +curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +``` + +or by hand, at main, through the API: ```sh gh api repos/opensoft/openRepoShape/contents/openRepoShape \ @@ -1394,6 +1415,7 @@ scripts/path_classify.py the classifier over the path policy scripts/shape_materialize.py the ONE materializer, used by both tools scripts/validate-repository-naming.py openRepoShape installs itself as a command; runs setup.sh +install.sh the one-line install: a release, pinned setup.sh the shim: find an interpreter, hand over setup-project.py THE FLOW, and the Windows way in (no bash there) scaffold-project.py creates the three repositories and the pins diff --git a/docs/cli.md b/docs/cli.md index 2bc93189..91681c5b 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -152,10 +152,22 @@ opensoft/openRepoTools', and its own installer places them: curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoTools/main/openRepoTools | bash -s -- --install $OPENREPOSHAPE_REPO owner/name to fetch from (default opensoft/openRepoShape) - $OPENREPOSHAPE_REF the ref to fetch it at (default main) + $OPENREPOSHAPE_REF the ref to fetch it at (default: the ref --install + recorded, else main) $OPENREPOSHAPE_ORG the organisation, when --org is not given $OPENREPOSHAPE_BIN_DIR where --install puts it (default ~/.local/bin) $OPENREPOSHAPE_SETUP_SH run this setup.sh on disk instead of fetching one + $OPENREPOSHAPE_RECORD_REF 1: --install records $OPENREPOSHAPE_REF (below) + +`--install` RECORDS A REF ONLY WHEN ASKED: with $OPENREPOSHAPE_RECORD_REF=1 +it writes $OPENREPOSHAPE_REF to `openRepoShape.ref` beside the command it +places, which is what the one-line installer (install.sh) does with the +release tag it installs. The installed command then stays on that ref: it +fetches setup.sh at it, setup.sh clones the standard at it (`--shape-ref`), +and `--doctor` checks its clone out at it. $OPENREPOSHAPE_REF set for one run +still wins, and `--version` prints the ref in force. A command re-installing +itself keeps its own record; any other install records nothing, removes a +record already there, and follows main. `--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` over (default `.`) and prints one row per check and ONE verdict: diff --git a/docs/handbook.html b/docs/handbook.html index a1add8af..59ec34d6 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -472,7 +472,7 @@ - + @@ -494,11 +494,11 @@

openRepoShape

convention, as data and as running code.

The fastest path — macOS, Linux, WSL2

-
curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRepoShape \
-    | bash -s -- --install
-openRepoShape --preflight
+          
curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash
 openRepoShape <Project> --org <your-org>
-

That first line installs ONE command into ~/.local/bin. The +

That first line installs ONE command into ~/.local/bin: the + newest release, pinned to its tag, with git-filter-repo where pipx + can install it, and it ends with openRepoShape --preflight. The estate verbs park and resume, for carrying in-flight work to another workstation, are opensoft/openRepoTools’ and have a one-liner of @@ -609,7 +609,7 @@

The one thing to read before electing it

- +

Setup

Start here

@@ -687,11 +687,11 @@

What this machine needs

brew install gh    # macOS, or Linux with Homebrew

Then brew install git-filter-repo only if you are ADOPTING an existing repository rather than scaffolding a new one.

-

The stock /bin/bash is 3.2 and is enough: setup.sh and the - openRepoShape command are written to use no syntax newer than that, though - CI runs on Linux only and no run of it checks them against bash 3.2. That the login - shell is zsh changes nothing — both entry points are run under bash - explicitly.

+

The stock /bin/bash is 3.2 and is enough: setup.sh, + install.sh and the openRepoShape command are written to use no + syntax newer than that, though CI runs on Linux only and no run of it checks them against + bash 3.2. That the login shell is zsh changes nothing — all three entry points are + run under bash explicitly.

The tool offers the rest

The preflight names what is missing and, for gh, its login and its credential helper, prints the command for THIS machine and asks once — Type yes to continue: — @@ -775,11 +775,23 @@

Make one machine setting first

Install the openRepoShape command, once

Install the openRepoShape command, once, inside WSL

Fetch the flow, in the directory the new project should land in

+
curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash
+

install.sh is attached to every release and + installs the newest, with no gh and no login. Its command is PINNED to the + release tag, recorded in openRepoShape.ref beside it: setup.sh + and the standard are fetched at that tag, not at main. It installs + git-filter-repo with pipx where it can, or prints the command, + and ends with openRepoShape --preflight. install.sh itself never + runs sudo; the preflight it ends with offers what is missing, each behind a + typed yes at your terminal. | bash -s -- v1.0.1 names a + release; the same line again upgrades.

+

By hand, the same command at main, which records no ref and follows main:

gh api repos/opensoft/openRepoShape/contents/openRepoShape \
     -H 'Accept: application/vnd.github.raw' | bash -s -- --install

Or, where raw downloads are not blocked, the same bytes over HTTPS:

curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRepoShape \
     | bash -s -- --install
+

By hand, the same command at main, which records no ref and follows main:

curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoShape/main/openRepoShape \
     | bash -s -- --install

Or, where raw downloads are blocked, the same bytes through the API:

@@ -794,13 +806,13 @@

Fetch the flow, in the directory the new project should an exhausted stdin, so it cannot ask for the one yes before three repositories exist — it refuses instead. Downloading the file first leaves stdin attached to the terminal you are typing at.

-

It installs ONE command into ~/.local/bin, +

Each way installs ONE command into ~/.local/bin, idempotently — a second run prints unchanged — and prints the export PATH=… line if that directory is not on PATH. The estate verbs park and resume, for carrying in-flight work, are opensoft/openRepoTools’ and have an install line of their own.

-

It installs ONE command into ~/.local/bin, +

Each way installs ONE command into ~/.local/bin, idempotently — a second run prints unchanged — and prints the export PATH=… line if that directory is not on PATH. It is one file at the repository root and implements nothing: @@ -954,8 +966,11 @@

Joining a family

Consent rules, on every platform

    -
  • Every action that changes the machine or the account is a typed yes, at - the same prompt: Type yes to continue:.
  • +
  • Every action the tool offers that changes the machine or the account is a typed + yes, at the same prompt: Type yes to continue:. install.sh + asks nothing before its own two acts, because running it is your act: it places the + command and installs git-filter-repo with pipx where it can, and itself + never runs sudo. The preflight it ends with asks before each offer, as above.
  • --yes skips the scaffold question only. It does not answer an install offer, gh auth login or gh auth setup-git.
  • With no terminal attached there is no offer at all.
  • @@ -1027,7 +1042,7 @@

    What you have afterwards — the same on every platform

    - +

    Setup

    If it fails part-way

    @@ -1062,7 +1077,7 @@

    If it fails part-way

    - + @@ -1120,7 +1135,7 @@

    Reading private legs in CI

    - + @@ -1356,7 +1371,7 @@

    The InkRouter example

    - + @@ -1653,7 +1668,7 @@

    Is this repository compliant? openRepoShape --doctor

    - + @@ -1808,7 +1823,7 @@

    The degrade rule

    - +

    Reference

    Layout of this repository

    @@ -1822,6 +1837,7 @@

    Layout of this repository

    scripts/shape_materialize.py
    the ONE materializer, used by both tools
    scripts/validate-repository-naming.py
    answers the naming question for one name
    openRepoShape
    installs itself as a command; runs setup.sh
    +
    install.sh
    the one-line install: a release, pinned
    setup.sh
    the shim: find an interpreter, hand over
    setup-project.py
    THE FLOW, and the Windows way in (no bash there)
    scaffold-project.py
    creates the three repositories and the pins
    @@ -1871,7 +1887,8 @@

    For AI assistants

  • Never answer an install prompt. The preflight asks Type yes to continue: for each thing it offers to run — an install, gh auth login, gh auth setup-git. The human types it; you never - install software or log in for them.
  • + install software or log in for them. The one-line install (install.sh) is + theirs to run: relay it, never run it.
  • Never pass --allow-upstream-org on your own initiative. If setup.sh refuses because the organisation you were given resolves to opensoft, STOP and relay what it said; the fix is almost always an explicit --org <their-org>.
  • diff --git a/docs/release-notes/v1.0.1.md b/docs/release-notes/v1.0.1.md new file mode 100644 index 00000000..9acb9734 --- /dev/null +++ b/docs/release-notes/v1.0.1.md @@ -0,0 +1,134 @@ +# openRepoShape v1.0.1 + +v1.0.1 is v1.0.0 plus one change: openRepoShape installs in one line, and the +command it installs stays on the release tag it was installed from (#564). + +## What 1.0.1 adds + +- **The one-line install.** `install.sh` is attached to every release, so + this line always installs the newest one: + + ```sh + curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash + ``` + + It reads which release is the newest from github.com's `releases/latest` + redirect, with no `gh` and no login. It fetches that release's + `openRepoShape` and installs it into `~/.local/bin`, or + `$OPENREPOSHAPE_BIN_DIR`, pinned to the release tag. It installs + `git-filter-repo` 2.47.0 with `pipx` when `pipx` is there and + `git-filter-repo` is not, and otherwise prints the one command that + installs it. It ends with `openRepoShape --preflight`. `install.sh` itself + never runs `sudo`; the preflight it ends with offers what is missing, each + behind a typed `yes` at your terminal. Every download `install.sh` makes + is HTTPS only, redirects included, at TLS 1.2 or newer. It exits 0 when the + command is + installed and the preflight passed, 1 when the command is installed but the + preflight did not pass or the command does not stay on the release tag, and + 2 when it installed nothing. +- **An installed command that stays on its release tag.** With + `OPENREPOSHAPE_RECORD_REF=1`, which `install.sh` sets, `openRepoShape + --install` records `$OPENREPOSHAPE_REF` in `openRepoShape.ref` beside the + command it places. The installed command then fetches `setup.sh` at that + ref, `setup.sh` clones the standard at it (`--shape-ref`), and `--doctor` + checks its clone out at it, and so does a symlink to the command. In + v1.0.0 an installed command fetched main on every run unless + `$OPENREPOSHAPE_REF` was exported. `$OPENREPOSHAPE_REF` set for one run + still overrides the record, and `openRepoShape --version` prints the ref in + force. The installed command is byte for byte the release's own file. + +## Install and upgrade + +- **The newest release:** the line above. Run it again to upgrade to the + newest release. +- **A named release:** + `curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash -s -- v1.0.1`, + or `bash install.sh v1.0.1` from a checkout or a download. +- **What is installed:** `openRepoShape --version` prints + `openRepoShape (opensoft/openRepoShape @ v1.0.1)`. +- **By hand, at main:** the README's + `gh api repos/opensoft/openRepoShape/contents/openRepoShape … | bash -s -- --install` + line still installs main's command, which records no ref and follows main. +- **Platforms:** macOS (the stock `/bin/bash` 3.2 is enough), Linux and WSL2. + Git Bash on Windows is meant to work, as #564 asks, but nothing has run it + yet: the launcher's and the installer's tests skip on the Windows job. + #585 (v1.1.0) runs them under Git Bash. Native Windows without Git Bash + still starts with `py setup-project.py`, as the README says. +- **A project's pin** on the standard works as before. Follow + [How to pin 1.0.0](v1.0.0.md#how-to-pin-100) with the v1.0.1 release + commit. + +## A script that runs `--install` + +`openRepoShape --install` records a ref only when it is asked to, with +`OPENREPOSHAPE_RECORD_REF=1`. A script that exports `$OPENREPOSHAPE_REF` +around `--install` for its own reasons gets the install it got from v1.0.0: +nothing is recorded, a record already there is removed, and the command +follows main. To pin the command a script installs, set +`OPENREPOSHAPE_RECORD_REF=1` and set `$OPENREPOSHAPE_REF` to the commit or +tag the command should stay on. + +## Known limits + +v1.0.1 changes nothing else, so its known limits are v1.0.0's: +[Known limits in v1.0.0](v1.0.0.md#known-limits). + +## Before tagging (remove at tag time) + + + +The one-line install works only once `install.sh` is an asset of the release +GitHub marks as latest. Nothing in this repository attaches it, so the +release is made in this order, and each step is safe to stop after: + +1. **Land and prove the release commit.** Squash-land #577 into + `release/v1.0`; that squash commit is ``. Confirm that its + tree is the tree of the PR head that passed `tests`, `tests-macos` and + `tests-windows`: + + ```sh + git rev-parse ^{tree} ^{tree} + ``` + + The two are equal when `release/v1.0` had no other commits. If they + differ, run `gh workflow run tests.yml --ref release/v1.0 -R opensoft/openRepoShape` + and go on only when all three jobs pass on ``. A push to + `release/v1.0` runs no CI by itself: `tests.yml` runs on a push to main + only. +2. **Tag that commit, annotated, and push the tag:** + + ```sh + git tag -a v1.0.1 -m "openRepoShape v1.0.1" + git push origin refs/tags/v1.0.1 + ``` + +3. **Create the release with `install.sh` attached, in one command,** from a + checkout at `` and with this section cut from a copy of + these notes: + + ```sh + gh release create v1.0.1 install.sh --verify-tag --latest \ + --title "openRepoShape v1.0.1" --notes-file -R opensoft/openRepoShape + ``` + + `--verify-tag` refuses rather than tag whatever the default branch holds. + gh makes a draft, uploads `install.sh`, then publishes it, so the newest + release is never without the file. +4. **Check the published asset, by redirect and by bytes:** + + ```sh + curl -fsSI --proto '=https' --proto-redir '=https' \ + https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh + curl -fsSL --proto '=https' --proto-redir '=https' \ + https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh \ + | cmp - <(git show v1.0.1:install.sh) + ``` + + The first must answer with a `location:` naming the v1.0.1 asset, + `…/releases/download/v1.0.1/install.sh`; the second, run in bash or zsh, + must print nothing and exit 0. +5. **Never attach `install.sh` to v1.0.0.** Its tree has no `install.sh`, and + its `openRepoShape` cannot record a ref, so the installer would exit 1 over + a command that follows main. **Publish a later patch on an older line with + `--latest=false`**, so that `releases/latest`, and with it the one-line + install, keeps naming the newest line. diff --git a/install.sh b/install.sh new file mode 100755 index 00000000..453507bb --- /dev/null +++ b/install.sh @@ -0,0 +1,409 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 +# +# install.sh — openRepoShape in one line, pinned to a release (#564): +# +# curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +# +# A named release instead of the newest: `… | bash -s -- v1.0.1`. From a +# checkout or a download, the same: `bash install.sh []`. Running the +# same line again upgrades to the newest release. This file is attached to +# every GitHub release, which is what makes that URL always the newest one. +# +# FOUR STEPS, IN THIS ORDER, EACH SAYING IN A LINE WHAT IT DID: +# +# 1. THE RELEASE: the one named, or the newest, read from the redirect +# github.com answers `releases/latest` with. That needs no `gh`, no token +# and no login, because the repository is public; `gh api` is asked only +# when the redirect names no release and `gh` is installed. +# 2. THAT RELEASE'S `openRepoShape`, fetched into a temporary directory, +# checked to be the command, and installed by its own `--install` with +# `$OPENREPOSHAPE_REF` set to the release tag and +# `$OPENREPOSHAPE_RECORD_REF=1`, which asks it to record that tag beside +# the command it places; the command then stays on it. +# 3. GIT-FILTER-REPO, which `adopt-project.py` needs and nothing else does: +# said when it is there; installed with `pipx` when pipx is there and it +# is not; otherwise the one command that installs it, printed. +# 4. THE INSTALLED COMMAND'S `--preflight`, LAST, and its verdict. +# +# install.sh ITSELF NEVER RUNS sudo, AND INSTALLS NOTHING BUT THESE TWO: the +# command and, through pipx, git-filter-repo. The preflight it ends with +# offers what is missing, each behind a typed `yes` at the person's own +# terminal, exactly as `openRepoShape --preflight` always has, and some of +# those offers are `sudo` commands: that is the preflight's, asked first. +# +# EXIT STATUS: +# 0 installed, pinned to the release tag, and the preflight passed; +# 1 installed, but the preflight did not pass, or the installed command +# does not stay on the release tag (one older than v1.0.1 cannot record +# it): the lines above say which; +# 2 refused before anything was installed: a version that is not a ref, no +# release found, a fetch that failed, or a file that is not the command. +# git-filter-repo never changes it: only adopting needs it, and +# `adopt-project.py` refuses with its own install command where it is missing. +# +# EVERYTHING RUNS FROM `main`, CALLED ON THE LAST LINE, INSIDE BRACES. Under +# `curl … | bash` bash reads this script from the pipe as it goes, so a +# download cut off part-way would run whatever had arrived. Inside a function +# nothing runs until the last line has arrived; the braces make that line one +# command that bash parses whole, so a download cut inside it runs nothing +# either; and by then the pipe holds nothing a child could read in its place. +# +# BASH 3.2, which is /bin/bash on every Mac: no `${x,,}`, no `declare -A`, +# no `mapfile`; an empty array never meets `set -u`. + +set -euo pipefail + +REPO="${OPENREPOSHAPE_REPO:-opensoft/openRepoShape}" + +#: The version CI installs (`.github/workflows/tests.yml`), so what this +#: installs is what the adoption suites ran against. +FILTER_REPO_PIN="git-filter-repo==2.47.0" + +WORKDIR="" + +#: What `latest_from_redirect` and `latest_from_gh` found, and when they did +#: not, why. Globals rather than output, because a function run inside +#: `$(…)` is a subshell and could say only one of the two. +FOUND="" +FAULT="" + +say() { printf '%s\n' "$*"; } +die() { printf '\nREFUSED: %s\n' "$1" >&2; exit "${2:-2}"; } + +usage() { + cat <<'USAGE' +install.sh: install openRepoShape, pinned to a release. + + curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash + curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash -s -- v1.0.1 + bash install.sh [] + +With no it installs the newest release. It fetches that release's +`openRepoShape`, installs it into ~/.local/bin pinned to the release tag, +installs git-filter-repo with pipx when pipx is there and git-filter-repo is +not (or prints the command that installs it), and ends with `openRepoShape +--preflight`. install.sh itself never runs sudo; the preflight it ends with +offers what is missing, each behind a typed `yes` at your terminal. Run the +same line again to upgrade. + +Exit 0: installed, pinned, and the preflight passed. 1: installed, but the +preflight did not pass, or the command does not stay on the release tag. 2: +refused, and nothing was installed. + + $OPENREPOSHAPE_REPO owner/name to install from (default opensoft/openRepoShape) + $OPENREPOSHAPE_BIN_DIR where the command goes (default ~/.local/bin) +USAGE +} + +# THE REPOSITORY, AS AN OWNER/NAME, because every URL below is built from it. +# The launcher also takes a URL or a path in this variable, for the clone +# setup.sh makes; there is no release at either, so this refuses them by +# name. A control character is refused before anything is echoed. +checked_repo() { + case "$REPO" in + *[![:print:]]*) die "\$OPENREPOSHAPE_REPO carries a control character; it must be an owner/name on github.com, such as opensoft/openRepoShape." ;; + esac + case "$REPO" in + [A-Za-z0-9]*/[A-Za-z0-9._-]*) ;; + *) die "\$OPENREPOSHAPE_REPO is '$REPO', which is not an owner/name on github.com, such as opensoft/openRepoShape: this installer builds github.com URLs from it." ;; + esac + case "$REPO" in + */*/* | *[!A-Za-z0-9._/-]* | */. | */..) die "\$OPENREPOSHAPE_REPO is '$REPO', which is not an owner/name on github.com, such as opensoft/openRepoShape: this installer builds github.com URLs from it." ;; + esac +} + +# THE LAUNCHER'S `checked_ref`, TRANSCRIBED: the version reaches a URL here +# and `git checkout` once the command runs, so it is held to the same three +# checks in the same order (length, then a control character, then the +# shape, which is the only one that echoes it), plus the empty value. Not +# shared, because this file arrives alone down a pipe. +checked_version() { + local ref="$1" what="$2" + [ -n "$ref" ] || die "$what is empty. Name a release, such as v1.0.1, or none for the newest." + if [ "${#ref}" -gt 255 ]; then + die "$what is ${#ref} characters long; a release tag, branch or commit is at most 255 characters." + fi + case "$ref" in + *[![:print:]]*) die "$what carries a control character, which is not a release tag, branch or commit." ;; + esac + case "$ref" in + [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) die "$what is '$ref', which is not a release tag, branch or commit: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'." ;; + esac +} + +# THE NEWEST RELEASE, FROM THE REDIRECT. `releases/latest` answers 302 with +# `location: …/releases/tag/`; a repository with no release yet answers +# with `…/releases`, which names no tag and is no answer. The header name is +# matched in any case (HTTP/2 sends it in lower case, HTTP/1.1 need not), +# without `${x,,}`, which bash 3.2 does not have, and a relative location is +# read the same way as a whole URL. +# +# EVERY curl HERE IS HTTPS ONLY, REDIRECTS INCLUDED, AT TLS 1.2 OR NEWER +# (SonarCloud shell:S6506), with the flags written out on the curl line +# itself, never gathered into an array, so a reader and the analyser both +# see them there. `tests/test_install_sh.py` holds every curl line to it. +latest_from_redirect() { + local url="$1" headers line value status=0 + FOUND="" + command -v curl >/dev/null 2>&1 || { + FAULT="curl is not installed" + return 1 + } + headers="$(curl -fsSI --proto '=https' --proto-redir '=https' --tlsv1.2 "$url" 2>/dev/null)" || status=$? + if [ "$status" -ne 0 ]; then + FAULT="curl failed, exit $status" + return 1 + fi + while IFS= read -r line; do + line="${line%$'\r'}" + case "$line" in + [Ll][Oo][Cc][Aa][Tt][Ii][Oo][Nn]:*) + value="${line#*:}" + while :; do + case "$value" in + ' '* | $'\t'*) value="${value#?}" ;; + *) break ;; + esac + done + case "$value" in + */releases/tag/?*) + FOUND="${value##*/releases/tag/}" + return 0 + ;; + esac + case "$value" in + *[![:print:]]*) FAULT="it redirects to a location that names no release" ;; + *) FAULT="it redirects to $value, which names no release" ;; + esac + return 1 + ;; + esac + done <<<"$headers" + FAULT="it answered with no location" + return 1 +} + +# THE FALLBACK, AND ONLY WHERE `gh` IS INSTALLED: `gh api` needs a login even +# for a public repository, which is why it is not the first way. +latest_from_gh() { + local status=0 + FOUND="" + command -v gh >/dev/null 2>&1 || { + FAULT="gh is not installed" + return 1 + } + FOUND="$(gh api "repos/$REPO/releases/latest" --jq .tag_name 2>/dev/null)" || status=$? + if [ "$status" -ne 0 ] || [ -z "$FOUND" ]; then + FAULT="gh failed, exit $status" + return 1 + fi +} + +# THE COMMAND, AND NOT A PAGE THAT SAYS 200. A captive portal, a proxy's +# error page or a renamed file is refused here, before anything runs it: +# the launcher's first line, and the `INSTALLABLES=(` line that +# `tests/test_repo_hygiene.py` holds it to carrying. +looks_like_the_launcher() { + local line first=1 + while IFS= read -r line || [ -n "$line" ]; do + if [ "$first" -eq 1 ]; then + [ "$line" = "#!/usr/bin/env bash" ] || return 1 + first=0 + fi + case "$line" in "INSTALLABLES=("*) return 0 ;; esac + done <"$1" + return 1 +} + +# THE ONE COMMAND THAT INSTALLS IT HERE, for a machine with no pipx, PRINTED +# AND NEVER RUN, in POSIX shell: Homebrew's where `brew` is; where `apt-get` +# is, pipx from apt and then git-filter-repo through it, because Debian 12 +# and Ubuntu 23.04 on refuse `pip install --user` into the system Python +# (externally-managed-environment); and otherwise pip, under whichever Python +# name this machine has. Always at the version CI runs, except brew's. +filter_repo_command() { + local candidate + if command -v brew >/dev/null 2>&1; then + printf '%s' "brew install git-filter-repo" + return 0 + fi + if command -v apt-get >/dev/null 2>&1; then + printf '%s' "sudo apt-get install -y pipx && pipx install $FILTER_REPO_PIN" + return 0 + fi + for candidate in python3 python; do + if command -v "$candidate" >/dev/null 2>&1; then + printf '%s' "$candidate -m pip install --user $FILTER_REPO_PIN" + return 0 + fi + done + printf '%s' "python3 -m pip install --user $FILTER_REPO_PIN" +} + +filter_repo_step() { + local found status=0 + if found="$(command -v git-filter-repo 2>/dev/null)" && [ -n "$found" ]; then + say "git-filter-repo: already installed ($found)" + return 0 + fi + if command -v git >/dev/null 2>&1 && + git filter-repo --version >/dev/null 2>&1; then + say "git-filter-repo: already installed (git filter-repo --version answers)" + return 0 + fi + if command -v pipx >/dev/null 2>&1; then + say "git-filter-repo: not installed; installing it with pipx, because adopt-project.py needs it: pipx install $FILTER_REPO_PIN" + pipx install "$FILTER_REPO_PIN" || status=$? + if [ "$status" -ne 0 ]; then + say "git-filter-repo: pipx could not install it (exit $status). Before adopting a repository, install it: pipx install $FILTER_REPO_PIN" + elif command -v git-filter-repo >/dev/null 2>&1; then + say "git-filter-repo: installed with pipx" + else + say "git-filter-repo: installed with pipx, into a directory that is not on PATH yet: run \`pipx ensurepath\` and open a new shell" + fi + return 0 + fi + say "git-filter-repo: not installed, and there is no pipx to install it with. Only adopt-project.py needs it; before adopting a repository, run:" + say " $(filter_repo_command)" +} + +main() { + local version="" asked launcher_url bin_dir target shown status=0 pinned=1 + + # NOTHING A CHILD READS IS THIS SCRIPT. Piped, stdin is the pipe the + # script came down; a terminal stays a terminal, which is where the + # preflight asks before it offers anything. + [ -t 0 ] || exec /dev/null 2>&1 || + die "curl is not installed, and this installer fetches with it. Install curl, or install by hand with: gh api repos/$REPO/contents/openRepoShape -H 'Accept: application/vnd.github.raw' | bash -s -- --install" + + # 1. THE RELEASE. + if [ -n "$version" ]; then + checked_version "$version" "the version" + say "release: $version, as named" + else + asked="https://github.com/$REPO/releases/latest" + if latest_from_redirect "$asked"; then + version="$FOUND" + else + local redirect_fault="$FAULT" + latest_from_gh || die "could not find the newest release of $REPO, so nothing was installed. Both ways failed: + curl -fsSI $asked ($redirect_fault) + gh api repos/$REPO/releases/latest ($FAULT) + Name a release instead, as bash install.sh v1.0.1 or as: + curl -fsSL https://github.com/$REPO/releases/latest/download/install.sh | bash -s -- v1.0.1" + version="$FOUND" + asked="gh api repos/$REPO/releases/latest" + fi + checked_version "$version" "the newest release's tag" + say "release: $version, the newest release of $REPO ($asked)" + fi + + # 2. THAT RELEASE'S COMMAND, FETCHED, CHECKED AND INSTALLED BY ITSELF. + WORKDIR="$(mktemp -d "${TMPDIR:-/tmp}/openRepoShape-install.XXXXXXXX")" || + die "could not make a temporary directory under ${TMPDIR:-/tmp}. Set \$TMPDIR to a directory you can write to." + trap 'rm -rf -- "$WORKDIR"' EXIT + launcher_url="https://raw.githubusercontent.com/$REPO/$version/openRepoShape" + curl -fsSL --proto '=https' --proto-redir '=https' --tlsv1.2 "$launcher_url" -o "$WORKDIR/openRepoShape" || status=$? + # WHICH FAILURE IT WAS DECIDES WHAT TO SAY. curl's exit 22 is the server + # answering with an error, which for a raw URL means no such file at that + # ref; anything else is curl getting no answer at all (DNS, a proxy, TLS, + # a timeout), where asking whether the release exists would send the + # person looking in the wrong place. + case "$status" in + 0) [ -s "$WORKDIR/openRepoShape" ] || + die "fetched $launcher_url, but it was empty, so nothing was installed." ;; + 22) die "could not fetch $launcher_url: the server answered with an error (curl exit 22), so nothing was installed. Is $version a release of $REPO? https://github.com/$REPO/releases" ;; + *) die "could not fetch $launcher_url: curl got no answer (curl exit $status), so nothing was installed. Check this machine's network, DNS and proxy, then run the line again." ;; + esac + looks_like_the_launcher "$WORKDIR/openRepoShape" || + die "$launcher_url is not the openRepoShape command (its first line is not #!/usr/bin/env bash, or it has no INSTALLABLES line), so nothing was installed." + say "fetch: $launcher_url" + + bin_dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" + target="$bin_dir/openRepoShape" + say "install: OPENREPOSHAPE_REF=$version OPENREPOSHAPE_RECORD_REF=1 openRepoShape --install" + OPENREPOSHAPE_REF="$version" OPENREPOSHAPE_RECORD_REF=1 \ + "${BASH:-bash}" "$WORKDIR/openRepoShape" --install || + die "the $version openRepoShape's --install refused (exit $?); its own refusal above says why." + + # THE PIN, CHECKED BY ASKING THE INSTALLED COMMAND, with the variable + # unset: a release older than v1.0.1 installs without recording a ref and + # would follow main, which is the drift this installer exists to end. + shown="$(unset OPENREPOSHAPE_REF; "${BASH:-bash}" "$target" --version 2>/dev/null)" || shown="" + case "$shown" in + *"@ $version)") ;; + *) + pinned=0 + say "install: $target does not stay on $version: it says '${shown:-nothing}' to --version. A release older than v1.0.1 cannot record the tag it was installed at: install v1.0.1 or newer, or export OPENREPOSHAPE_REF=$version in your shell profile to keep it there." + ;; + esac + if [ -n "${OPENREPOSHAPE_REF:-}" ] && [ "$OPENREPOSHAPE_REF" != "$version" ]; then + say "install: \$OPENREPOSHAPE_REF is set in this shell, by your shell profile or by hand, and while it is, every run uses it instead of $version. Remove it from your shell profile to run $version." + fi + + # 3. GIT-FILTER-REPO. + filter_repo_step + + # 4. THE PREFLIGHT, LAST, THROUGH THE INSTALLED COMMAND AND ITS RECORD, so + # its setup.sh is the release's. Where the command did not record the tag, + # the tag is handed to this one run instead, so that a command which + # follows main is still not checked with main's setup.sh. + say "preflight: $target --preflight" + status=0 + if [ "$pinned" -eq 1 ]; then + ( + unset OPENREPOSHAPE_REF + "${BASH:-bash}" "$target" --preflight + ) || status=$? + else + OPENREPOSHAPE_REF="$version" "${BASH:-bash}" "$target" --preflight || status=$? + fi + + # EXIT 1 IS THE PREFLIGHT'S "not ready", which names what is missing; any + # other status is a run that did not get that far (setup.sh could not be + # fetched, or there is no Python to run it), and its REFUSED line says so. + case "$status" in + 0) ;; + 1) + say "done: openRepoShape $version is installed at $target, but the preflight did not pass (exit 1); it names what is missing above." + exit 1 + ;; + *) + say "done: openRepoShape $version is installed at $target, but the preflight did not run to the end (exit $status); the REFUSED line above says why." + exit 1 + ;; + esac + if [ "$pinned" -eq 0 ]; then + say "done: openRepoShape $version is installed at $target and this machine is ready, but the command does not stay on $version (see above)." + exit 1 + fi + say "done: openRepoShape $version is installed at $target, pinned to the release tag, and this machine is ready. Run the same line again to upgrade to the newest release." + exit 0 +} + +# `${1+"$@"}`, setup.sh's own spelling: no arguments is the one-line run, and +# that has to survive `set -u` under every bash this has to run on. In +# BRACES, so the call is one compound command that bash reads to its closing +# brace before running any of it: a download cut anywhere inside this line +# is a syntax error, and runs nothing. +{ main ${1+"$@"}; } diff --git a/openRepoShape b/openRepoShape index 46cab822..2ab98fb7 100755 --- a/openRepoShape +++ b/openRepoShape @@ -3,7 +3,12 @@ # # openRepoShape — the short way in: `openRepoShape Atlas --org `. # -# Install it once, from anywhere, with no checkout and no clone: +# Install it once, from anywhere, with no checkout and no clone. The newest +# release, pinned to its tag (`install.sh`, #564): +# +# curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/download/install.sh | bash +# +# Or this file at main, by hand, which follows main: # # gh api repos/opensoft/openRepoShape/contents/openRepoShape \ # -H 'Accept: application/vnd.github.raw' | bash -s -- --install @@ -28,9 +33,31 @@ set -euo pipefail # setup.sh reads the same variable as something to `git clone` — see # shape_clone_url below: one name, both uses. REPO="${OPENREPOSHAPE_REPO:-opensoft/openRepoShape}" -REF="${OPENREPOSHAPE_REF:-main}" SELF="${BASH_SOURCE[0]:-$0}" +# THE REF, AND WHETHER IT WAS NAMED (#564). A NAMED ref is one somebody +# chose: setup.sh is told it (`--shape-ref`) and `--doctor` checks its clone +# out at it, where an unnamed `main` leaves both to the default branch. Two +# things name one, and the first wins: `$OPENREPOSHAPE_REF`, set for a run, +# and the ref `--install` RECORDED beside this file (`.ref`, read by +# `read_recorded_ref` below, once `ref_fault` exists to check it). With +# neither, it is `main`, unnamed, exactly as before there was a record. +REF="${OPENREPOSHAPE_REF:-main}" +REF_NAMED=0 +[ -z "${OPENREPOSHAPE_REF:-}" ] || REF_NAMED=1 + +#: 1 when the ref in force came from the record, which is what lets a +#: command re-installing itself keep it with no opt-in (see `record_ref`). +REF_FROM_RECORD=0 + +#: Why the record beside this file was refused, said once the arguments are +#: read, so that `--help` still answers past a broken record. +RECORD_FAULT="" + +#: The record's name is the command's own plus this, so the record of +#: `~/.local/bin/openRepoShape` is `~/.local/bin/openRepoShape.ref`. +RECORD_SUFFIX=".ref" + PROJECT="" ORG="" MODE="run" @@ -96,10 +123,22 @@ opensoft/openRepoTools', and its own installer places them: curl -fsSL https://raw.githubusercontent.com/opensoft/openRepoTools/main/openRepoTools | bash -s -- --install $OPENREPOSHAPE_REPO owner/name to fetch from (default opensoft/openRepoShape) - $OPENREPOSHAPE_REF the ref to fetch it at (default main) + $OPENREPOSHAPE_REF the ref to fetch it at (default: the ref --install + recorded, else main) $OPENREPOSHAPE_ORG the organisation, when --org is not given $OPENREPOSHAPE_BIN_DIR where --install puts it (default ~/.local/bin) $OPENREPOSHAPE_SETUP_SH run this setup.sh on disk instead of fetching one + $OPENREPOSHAPE_RECORD_REF 1: --install records $OPENREPOSHAPE_REF (below) + +`--install` RECORDS A REF ONLY WHEN ASKED: with $OPENREPOSHAPE_RECORD_REF=1 +it writes $OPENREPOSHAPE_REF to `openRepoShape.ref` beside the command it +places, which is what the one-line installer (install.sh) does with the +release tag it installs. The installed command then stays on that ref: it +fetches setup.sh at it, setup.sh clones the standard at it (`--shape-ref`), +and `--doctor` checks its clone out at it. $OPENREPOSHAPE_REF set for one run +still wins, and `--version` prints the ref in force. A command re-installing +itself keeps its own record; any other install records nothing, removes a +record already there, and follows main. `--doctor [] [--json] [--placement-plan ]` runs `shape-doctor.py` over (default `.`) and prints one row per check and ONE verdict: @@ -168,6 +207,10 @@ workdir() { # curl is the fallback for a machine with no `gh`. Each attempt writes to a # file and is checked before its bytes are used, so a half-written first # attempt is never mistaken for a fetch. +# +# HTTPS ONLY, REDIRECTS INCLUDED, TLS 1.2 OR NEWER (#564, SonarCloud +# shell:S6506): the flags are written out on the curl line itself, never +# gathered into an array, so a reader and the analyser both see them there. fetch_from_repo() { local path="$1" dest="$2" if command -v gh >/dev/null 2>&1 && @@ -177,7 +220,8 @@ fetch_from_repo() { return 0 fi if command -v curl >/dev/null 2>&1 && - curl -fsSL "https://raw.githubusercontent.com/$REPO/$REF/$path" \ + curl -fsSL --proto '=https' --proto-redir '=https' --tlsv1.2 \ + "https://raw.githubusercontent.com/$REPO/$REF/$path" \ >"$dest" 2>/dev/null && [ -s "$dest" ]; then return 0 fi @@ -295,9 +339,67 @@ machine_report() { say " run: openRepoShape --preflight (offers what is missing; each install asks first)" } +# THE REF THIS INSTALL RAN AT, WRITTEN BESIDE THE COMMAND IT PLACED (#564). +# Without it an installed command read `$OPENREPOSHAPE_REF` at every run and +# fell back to `main`, so a command installed from a release fetched the next +# day's main: the release it was installed from lasted until main moved. +# +# A FILE BESIDE THE COMMAND, NEVER AN EDIT TO IT. The placed command stays +# byte for byte the file it was installed from, so `cmp` against the release's +# own `openRepoShape` still says whether it IS that release: the `(unchanged)` +# verdict `install_commands` gives, and any copy vendored and checked by +# sha256 elsewhere, compare the command's bytes and never see the record. A +# command copied away without its record follows `main` again, and says so in +# `--version`. +# +# RECORDED ONLY WHEN ASKED (#564, the review on #577). `$OPENREPOSHAPE_REF` +# alone records nothing, because a script that sets it around `--install` +# for its own reasons is not asking for a pin: workBenches sets it to a value +# that names no ref at all, to make any fetch fail loudly, and a record of +# that would break every later run on every machine it installs to. So the +# ref in force is written only when `$OPENREPOSHAPE_RECORD_REF` is 1, as the +# one-line installer sets it, or when it came from the record beside the copy +# being run, so a pinned command re-installing itself stays pinned. Any other +# install is an install with no ref: it records nothing, and REMOVES a record +# already there, which makes an install at main follow main however the +# command was installed before. Every outcome is one line, and nothing to +# record and no record is no line at all. The record's old value is echoed +# only when it is a ref this command would accept, because a hand-edited +# file is not this command's to print. +record_ref() { + local record="$1" recording="$2" was="" + if [ -f "$record" ]; then + was="$(< "$record")" || was="" + [ -z "$(ref_fault "$was" "it")" ] || was="another ref" + fi + if [ "$recording" -eq 0 ]; then + [ -e "$record" ] || return 0 + rm -f -- "$record" + say "openRepoShape: no longer pinned to ${was:-another ref}; it follows main, or \$OPENREPOSHAPE_REF where set (removed $record)" + elif [ "$was" = "$REF" ]; then + say "openRepoShape: already pinned to $REF in $record (unchanged)" + else + printf '%s\n' "$REF" >"$record" + say "openRepoShape: pinned to $REF${was:+ (was $was)}, recorded in $record" + fi +} + install_commands() { local dir="${OPENREPOSHAPE_BIN_DIR:-$HOME/.local/bin}" - local name target source verb placed=0 + local name target source verb placed=0 recording=0 record + if [ "$REF_FROM_RECORD" -eq 1 ] || + { [ "$REF_NAMED" -eq 1 ] && [ "${OPENREPOSHAPE_RECORD_REF:-}" = 1 ]; }; then + recording=1 + fi + # THE RECORD SITS BESIDE THE FILE, symlinks followed, as it is read. + record="$(resolved "$dir/openRepoShape")$RECORD_SUFFIX" + # BOTH CHECKED BEFORE ANYTHING IS COLLECTED OR PLACED: a ref about to be + # written down and read back by every later run, and a record path this + # install could neither write nor remove. A refusal after the copy would + # leave a new command beside an old record. + [ "$recording" -eq 0 ] || checked_ref "$REF" + [ ! -e "$record" ] || [ -f "$record" ] || + die "$record is not a file, so this install could neither record its ref there nor remove it. Move it away, then run this again." collect_commands mkdir -p "$dir" for name in "${INSTALLABLES[@]}"; do @@ -314,6 +416,9 @@ install_commands() { placed=$((placed + 1)) say "$name: $verb" done + # The record belongs to the one command that reads it, and is written + # only once that command is in place. + record_ref "$record" "$recording" # THE COUNT, out loud, and kept at one installable (#92): it costs a line # and is what keeps $INSTALLABLES honest — a person reading N lines cannot # tell whether an N+1th was meant to be there. The sentence that made @@ -361,7 +466,8 @@ find_python() { # A CHECKOUT OF THE STANDARD, made the way `setup.sh` self-bootstraps one: # `git clone` of `shape_clone_url`, into the temporary directory `workdir` -# owns, checked out at $OPENREPOSHAPE_REF when one was named. Sets $SHAPE_DIR. +# owns, checked out at $REF when one was named: by $OPENREPOSHAPE_REF, or by +# the record `--install` left beside this file (#564). Sets $SHAPE_DIR. # # WITHOUT `--depth 1`, WHICH IS THE ONE DEVIATION FROM setup.sh AND IS FORCED. # A shallow clone has exactly ONE commit, and the doctor's shape-currency row @@ -385,7 +491,7 @@ shape_checkout() { die "could not clone $REPO into a temporary directory, so there is no standard to check against. \`--doctor\` needs one: run it from a checkout of openRepoShape, or make the clone work." - if [ -n "${OPENREPOSHAPE_REF:-}" ]; then + if [ "$REF_NAMED" -eq 1 ]; then checked_ref "$REF" # ` --`, never `-- `: a `--` BEFORE the argument makes git # read it as a PATHSPEC and try to restore a file of that name from @@ -404,17 +510,93 @@ shape_checkout() { # CHARACTER next, because a newline in it could forge a line of output before # this script has said a word; and only then the SHAPE, which is safe to echo # back because it has passed both. +# +# WHOSE REF IT IS IS AN ARGUMENT (#564): `$OPENREPOSHAPE_REF` by default, or +# the record `--install` wrote. `ref_fault` SAYS what is wrong and refuses +# nothing, so the record can be checked as it is read and refused only once +# the arguments are, after `--help` has had its chance; `checked_ref` is the +# refusing spelling. An EMPTY ref is a fault too, which no caller could reach +# before the record: an empty variable is no ref at all, while an empty file +# is one. +ref_fault() { + local ref="$1" what="$2" + if [ -z "$ref" ]; then + printf '%s' "$what is empty, which is not a branch, tag or commit this tool will pass to \`git checkout\`." + elif [ "${#ref}" -gt 255 ]; then + printf '%s' "$what is ${#ref} characters long; a branch, tag or commit this tool will pass to \`git checkout\` is at most 255 characters." + else + case "$ref" in + *[![:print:]]*) printf '%s' "$what carries a control character, which is not a branch, tag or commit this tool will pass to \`git checkout\`." ;; + [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) printf '%s' "$what is '$ref', which is not a branch, tag or commit this tool will pass to \`git checkout\`: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'." ;; + esac + fi +} + checked_ref() { - local ref="$1" - if [ "${#ref}" -gt 255 ]; then - die "\$OPENREPOSHAPE_REF is ${#ref} characters long; a branch, tag or commit this tool will pass to \`git checkout\` is at most 255 characters." + local fault + fault="$(ref_fault "$1" "${2:-\$OPENREPOSHAPE_REF}")" + [ -z "$fault" ] || die "$fault" +} + +# A PATH WITH ITS SYMLINKS FOLLOWED (#564, the review on #577). The record +# sits beside the FILE `--install` placed, so a link to the command from +# anywhere else (`~/bin/openRepoShape -> ~/.local/bin/openRepoShape`) must +# read the record there, not look for one beside the link. `readlink` with +# no `-f`, which macOS before 12.3 lacks, one link at a time and at most 40 +# of them, the kernel's own limit, so a loop of links ends. Where `readlink` +# fails or is missing the path is used as it is, which is where the record +# was looked for before. Git Bash makes copies for `ln -s` unless told +# otherwise, and a copy is no link. +resolved() { + local path="$1" link hops=0 + while [ -L "$path" ] && [ "$hops" -lt 40 ]; do + link="$(readlink -- "$path" 2>/dev/null)" && [ -n "$link" ] || break + case "$link" in + /*) path="$link" ;; + *) path="$(dirname -- "$path")/$link" ;; + esac + hops=$((hops + 1)) + done + printf '%s' "$path" +} + +# THE RECORD `--install` LEFT BESIDE THIS FILE, READ ONCE, BEFORE ANY RUN +# (#564). Only when no `$OPENREPOSHAPE_REF` was set, which is what lets that +# variable win for a run, and only beside a FILE: a script arriving on stdin +# has no place of its own to have been installed at. CHECKED HERE, as it is +# read, rather than where the ref is used: it is a file on disk that every run +# reads, `--version` prints it, and a value nobody typed is not echoed until +# it has passed. A record that fails is NOT refused here but kept in +# `$RECORD_FAULT`, and `refuse_bad_record` refuses with it once the arguments +# have been read: `--help` answers past a broken record, and every run that +# would use the ref refuses, `--version` first. `$(< file)` is bash's own +# read, so no `cat` is needed; a record that is not a readable FILE (a +# directory, say) is a fault in the same words as the rest, never a raw bash +# error. +RECORD_WAY_OUT="Run the installer again to record a release, or delete that file to follow main." +read_recorded_ref() { + local record text fault + [ "$REF_NAMED" -eq 0 ] && [ -f "$SELF" ] || return 0 + record="$(resolved "$SELF")$RECORD_SUFFIX" + [ -e "$record" ] || return 0 + if [ ! -f "$record" ] || [ ! -r "$record" ]; then + RECORD_FAULT="$record is not a file this command can read, so it cannot read the ref \`--install\` recorded there. Remove it, then run the installer again to record a release, or leave it removed to follow main." + return 0 fi - case "$ref" in - *[![:print:]]*) die "\$OPENREPOSHAPE_REF carries a control character, which is not a branch, tag or commit this tool will pass to \`git checkout\`." ;; - esac - case "$ref" in - [!A-Za-z0-9]* | *..* | *.lock | *[!A-Za-z0-9._/-]*) die "\$OPENREPOSHAPE_REF is '$ref', which is not a branch, tag or commit this tool will pass to \`git checkout\`: a ref starts with a letter or digit, uses only letters, digits, '.', '_', '/', '-', has no '..' and does not end in '.lock'." ;; - esac + text="$(< "$record")" + fault="$(ref_fault "$text" "the ref recorded in $record")" + if [ -n "$fault" ]; then + RECORD_FAULT="$fault $RECORD_WAY_OUT" + return 0 + fi + REF="$text" + REF_NAMED=1 + REF_FROM_RECORD=1 +} +read_recorded_ref + +refuse_bad_record() { + [ -z "$RECORD_FAULT" ] || die "$RECORD_FAULT" } # `--doctor`: RUN THE STANDARD'S OWN SCRIPT, never a reimplementation here. @@ -515,7 +697,7 @@ run_doctor() { while [ $# -gt 0 ]; do case "$1" in -h | --help) usage; exit 0 ;; - --version) say "openRepoShape ($REPO @ $REF)"; exit 0 ;; + --version) refuse_bad_record; say "openRepoShape ($REPO @ $REF)"; exit 0 ;; # TWO MODES IS A REFUSAL, NEVER LAST-WINS. `--doctor --install` # used to enter the install path and silently drop the diagnosis, and # the other order dropped the install (Copilot, PR #96). Each of these @@ -622,6 +804,10 @@ while [ $# -gt 0 ]; do esac done +# A BROKEN RECORD IS REFUSED HERE, after `--help` has had its chance and +# before any run that would use the ref it should have named (#564). +refuse_bad_record + if [ "$MODE" = "doctor" ]; then # ONE PATH AND THE DOCTOR'S TWO FLAGS, NOTHING ELSE. Every other # argument this command knows is about a run that CREATES something, and @@ -699,7 +885,8 @@ ARGS=() [ -z "$PROJECT" ] || ARGS+=(--project "$PROJECT") # Fetched at $REF, so setup.sh self-bootstraps its checkout at $REF too: # otherwise the script that runs and the standard it clones are two commits. -[ -z "${OPENREPOSHAPE_REF:-}" ] || ARGS+=(--shape-ref "$REF") +# Whenever the ref was NAMED, by the variable or by the record (#564). +[ "$REF_NAMED" -eq 0 ] || ARGS+=(--shape-ref "$REF") ARGS+=(${PASSTHROUGH[@]+"${PASSTHROUGH[@]}"}) # $OPENREPOSHAPE_SETUP_SH is the OFFLINE TEST PATH: a setup.sh already on disk diff --git a/tests/readme-cap.d/564.txt b/tests/readme-cap.d/564.txt new file mode 100644 index 00000000..7a500425 --- /dev/null +++ b/tests/readme-cap.d/564.txt @@ -0,0 +1,21 @@ ++22 +2026-10-15 +Twenty-two lines for the one-line install, v1.0.1 (#564), carried into main +for v1.1.0 by Brett Heap's ruling 35. On release/v1.0 #577 raised the cap the +old way, 1422 -> 1444, in the test; here the same lines are this file. +Brett Heap: "we need it to be more simple to install ... now and after, we +need a simple install process". Step 3 of the quick start now LEADS with +`curl -fsSL …/releases/latest/download/install.sh | bash`, and the worked +example's "Install the command" with the same line: ten lines for the two +blocks and the blank lines around them (+10). Nine, step 3's paragraph and the +blank after it, say what that line does that the old two did not, each a thing +a reader would otherwise meet unannounced on their own machine: it needs no +`gh` and no login, the command is PINNED to the release tag, where that is +recorded and what it changes, it installs git-filter-repo with pipx or prints +the command, it ends in the preflight, `install.sh` itself never runs `sudo` +while each of the preflight's offers waits for a typed `yes`, how to name a +release and how to upgrade (+9). Two re-introduce the existing lines as the +by-hand way, at main, so neither is lost (+2), and one is the Layout row for +`install.sh` (+1). The review on #577 folded a separate "Pinned" paragraph +into step 3's and left the exit statuses to `install.sh --help` and the +release notes, which took the raise from 28 lines to 22. diff --git a/tests/test_install_sh.py b/tests/test_install_sh.py new file mode 100644 index 00000000..34c7de5a --- /dev/null +++ b/tests/test_install_sh.py @@ -0,0 +1,724 @@ +# SPDX-License-Identifier: Apache-2.0 +"""`install.sh`: openRepoShape in one line, pinned to a release (#564). + +OFFLINE, ON A `$PATH` THE TEST OWNS ENTIRELY. Each run's `$PATH` holds the +real programs the installer and the command it installs need (`bash`, +`mktemp`, `cp` and the rest of `INSTALL_NEEDS`), and FAKES for everything +that would reach the network or install something: `curl`, `gh`, `pipx`, +and, where a test asks for them, `git-filter-repo`, `git` and `brew`. +Restricted rather than prepended, because the machine running this may have +git-filter-repo and pipx installed for real, and which of the three +git-filter-repo branches runs must be the test's choice, not the machine's. + +EVERY FAKE IS ONE PROGRAM. A two-line shell launcher per name hands its +arguments to `FAKE` below, which appends the call to `calls.log` and answers +from `net.json`: the redirect github.com answers `releases/latest` with, the +files raw.githubusercontent.com serves, the tag `gh api` reports, and where +`pipx` puts what it installs. The `setup.sh` it serves is another launcher +into the same program, so the preflight the installer ends with is LOGGED, +with the arguments the installed command handed it: `--shape-ref ` +is where the pin shows. + +THE COMMAND SERVED IS THIS CHECKOUT'S `openRepoShape`, so the install the +installer runs is the real one. Every run sets `GH_TOKEN=bogus` and an empty +`GH_CONFIG_DIR`, and its `TMPDIR` is a directory each test checks is empty +again afterwards. + +WINDOWS_SKIP, for the reason `tests/test_openreposhape_command.py` gives: the +installer and the command are bash, run where `bash` is a bash, and on +Windows the way in is `setup-project.py`. +""" + +from __future__ import annotations + +import json +import re +import shlex +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +from conftest import REPO, WINDOWS_SKIP +from test_openreposhape_command import COMMAND, INSTALL_NEEDS, command_env + +INSTALLER = REPO / "install.sh" + +#: Bash by ABSOLUTE path: `subprocess` resolves a bare argv[0] against the +#: child's `$PATH`, and the one each run builds holds only what it chose. +BASH = shutil.which("bash") + +pytestmark = [pytest.mark.skipif(BASH is None, reason="install.sh is bash"), + WINDOWS_SKIP] + +SLUG = "opensoft/openRepoShape" +RELEASE = "v9.9.9" +FILTER_REPO = "git-filter-repo==2.47.0" + +#: The flags every curl in the installer and the launcher carries, as the +#: fakes log them once the shell has taken the quotes off (SonarCloud +#: shell:S6506: HTTPS only, redirects included, TLS 1.2 or newer). +HTTPS_ONLY = "--proto =https --proto-redir =https --tlsv1.2" + + +def latest(slug: str = SLUG) -> str: + return f"https://github.com/{slug}/releases/latest" + + +def tag_page(tag: str, slug: str = SLUG) -> str: + return f"https://github.com/{slug}/releases/tag/{tag}" + + +def raw(name: str, ref: str = RELEASE, slug: str = SLUG) -> str: + return f"https://raw.githubusercontent.com/{slug}/{ref}/{name}" + + +#: THE ONE PROGRAM BEHIND EVERY FAKE. `curl -fsSI` answers a configured +#: redirect with the headers github.com sends, CRLF and all, and `curl +#: -fsSL` a configured file, to `-o` or to stdout; anything else is curl's +#: own exit 22. `gh` reports a release only when one is configured. `pipx` +#: places a `git-filter-repo` launcher where it is told to, or fails. Every +#: call is logged first, so what was NOT asked is as plain as what was. +FAKE = r''' +import json +import shlex +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +NET = json.loads((HERE / "net.json").read_text(encoding="utf-8")) +NAME, ARGS = sys.argv[1], sys.argv[2:] +with open(HERE / "calls.log", "a", encoding="utf-8") as log: + log.write(" ".join([NAME] + ARGS) + "\n") + + +def refuse(message): + sys.stderr.write(message + "\n") + return 22 if NAME == "curl" else 1 + + +def launcher(directory, name): + path = Path(directory) / name + path.write_text("#!/bin/sh\nexec %s %s %s \"$@\"\n" % ( + shlex.quote(sys.executable), shlex.quote(str(Path(__file__))), name), + encoding="utf-8") + path.chmod(0o755) + + +def curl(): + url = next((arg for arg in ARGS if arg.startswith("https://")), "") + if url in NET.get("unreachable", []): + sys.stderr.write("curl: (6) Could not resolve host\n") + return 6 + if "-fsSI" in ARGS: + location = NET["redirects"].get(url) + if location is None: + return refuse("curl: (22) The requested URL returned error: 404") + sys.stdout.write("HTTP/2 302 \r\n%s: %s\r\n" + "content-length: 0\r\n\r\n" + % (NET.get("location_header", "location"), location)) + return 0 + served = NET["files"].get(url) + if served is None: + return refuse("curl: (22) The requested URL returned error: 404") + data = Path(served).read_bytes() + if "-o" in ARGS: + Path(ARGS[ARGS.index("-o") + 1]).write_bytes(data) + else: + sys.stdout.buffer.write(data) + return 0 + + +def gh(): + if ARGS[:1] == ["api"] and ARGS[1].endswith("/releases/latest") \ + and NET.get("gh_latest"): + print(NET["gh_latest"]) + return 0 + return refuse("fake gh: no answer for: " + " ".join(ARGS)) + + +def pipx(): + if NET.get("pipx_installs_into") is None: + return refuse("fake pipx: could not install " + " ".join(ARGS)) + launcher(NET["pipx_installs_into"], "git-filter-repo") + return 0 + + +def git(): + if ARGS == ["filter-repo", "--version"] and NET.get("git_filter_repo"): + print("a40bce548d2c") + return 0 + return refuse("fake git: no answer for: " + " ".join(ARGS)) + + +def setup_sh(): + print("fake preflight: " + " ".join(ARGS)) + return NET.get("preflight_exit", 0) + + +sys.exit({"curl": curl, "gh": gh, "pipx": pipx, "git": git, + "setup.sh": setup_sh}.get(NAME, lambda: 0)()) +''' + +#: A RELEASE OLDER THAN v1.0.1, as far as the installer can tell: the +#: launcher's first line and its `INSTALLABLES` line, an `--install` that +#: copies itself and records nothing, and a `--version` that therefore says +#: `main`. It is all the installer asks of a command. +OLD_LAUNCHER = """#!/usr/bin/env bash +INSTALLABLES=(openRepoShape) +case "$1" in +--install) mkdir -p "$OPENREPOSHAPE_BIN_DIR" && cp "$0" "$OPENREPOSHAPE_BIN_DIR/openRepoShape" ;; +--version) printf 'openRepoShape (%s @ %s)\\n' "$OPENREPOSHAPE_REPO" "${OPENREPOSHAPE_REF:-main}" ;; +--preflight) printf 'old preflight at %s\\n' "${OPENREPOSHAPE_REF:-main}" ;; +esac +""" + + +class Net: + """The fakes, what they answer, the `$PATH` that holds them, and a run. + + `fakes` names the programs on that `$PATH` that answer through `FAKE`; + `serve` adds a release's two files; `config` is written to `net.json` + at every `run`, so a test changes an answer between two runs by + changing it here. + """ + + def __init__(self, tmp_path: Path, fakes=("curl", "gh", "pipx")): + self.tmp_path = tmp_path + self.root = tmp_path / "net" + self.root.mkdir() + (self.root / "fake.py").write_text(FAKE, encoding="utf-8") + self.path = tmp_path / "path" + self.path.mkdir() + for name in (*INSTALL_NEEDS, "bash"): + real = shutil.which(name) + assert real, f"this machine has no {name}, which the install needs" + (self.path / name).symlink_to(real) + for name in fakes: + self.fake(name) + self.temporary = tmp_path / "tmp" + self.temporary.mkdir() + self.bin = tmp_path / ".local" / "bin" + self.config = {"redirects": {latest(): tag_page(RELEASE)}, + "files": {}, "pipx_installs_into": str(self.path)} + self.serve(RELEASE) + + def fake(self, name: str, directory: Path | None = None) -> Path: + """A launcher for `name` into `FAKE`, on this `$PATH` by default.""" + path = (directory or self.path) / name + path.write_text( + f"#!/bin/sh\nexec {shlex.quote(sys.executable)} " + f"{shlex.quote(str(self.root / 'fake.py'))} {name} \"$@\"\n", + encoding="utf-8") + path.chmod(0o755) + return path + + def serve(self, ref: str, slug: str = SLUG, + launcher: Path = COMMAND) -> None: + """`openRepoShape` and `setup.sh` at `ref`, as raw URLs answer.""" + served = self.root / "served" / ref + served.mkdir(parents=True, exist_ok=True) + self.config["files"][raw("openRepoShape", ref, slug)] = str(launcher) + self.config["files"][raw("setup.sh", ref, slug)] = str( + self.fake("setup.sh", served)) + + def run(self, *args: str, piped: bool = False, env: dict | None = None, + script: str | None = None) -> subprocess.CompletedProcess: + """`bash install.sh `, or `… | bash -s -- ` when piped; + `script` pipes that text in place of the installer's own.""" + (self.root / "net.json").write_text(json.dumps(self.config), + encoding="utf-8") + gh_config = self.tmp_path / "gh-config" + gh_config.mkdir(exist_ok=True) + piped = piped or script is not None + argv = [BASH, "-s", "--", *args] if piped \ + else [BASH, str(INSTALLER), *args] + return subprocess.run( + argv, capture_output=True, text=True, check=False, + input=(script or INSTALLER.read_text(encoding="utf-8")) + if piped else "", + cwd=str(self.tmp_path), start_new_session=True, + env=command_env(home=self.tmp_path, setup_sh=None, env={ + "PATH": str(self.path), "GH_TOKEN": "bogus", + "GH_CONFIG_DIR": str(gh_config), + "TMPDIR": str(self.temporary), **(env or {})})) + + def calls(self) -> list: + log = self.root / "calls.log" + return log.read_text(encoding="utf-8").splitlines() \ + if log.exists() else [] + + def installed(self, result, ref: str = RELEASE) -> None: + """The command is placed, byte for byte, with `ref` recorded beside + it, and the run left nothing in `$TMPDIR`.""" + target = self.bin / "openRepoShape" + assert target.read_bytes() == COMMAND.read_bytes(), ( + result.stdout + result.stderr) + assert (self.bin / "openRepoShape.ref").read_text( + encoding="utf-8") == f"{ref}\n" + assert list(self.temporary.iterdir()) == [] + + def refused(self, result) -> None: + """Exit 2, nothing placed, no preflight run, nothing left behind.""" + assert result.returncode == 2, result.stdout + result.stderr + assert not self.bin.exists(), "a refused install placed something" + assert not any(call.startswith("setup.sh") for call in self.calls()) + assert list(self.temporary.iterdir()) == [] + + +@pytest.fixture +def net(tmp_path): + return Net(tmp_path) + + +def preflight_call(ref: str = RELEASE) -> str: + """How the preflight reaches setup.sh through a command pinned to `ref`.""" + return f"setup.sh --shape-ref {ref} --preflight" + + +# --- the one line ------------------------------------------------------------ + +def test_the_one_line_installs_the_newest_release_pinned_to_it(net): + """`curl … | bash`: the newest release, from the redirect alone. + + One line per step, in order, and the calls behind them in the same + order: the redirect, the release's command, then the preflight, LAST, + through the installed command, whose `setup.sh` is fetched at the + release and handed `--shape-ref` for it. `gh` is never asked for a + release when the redirect named one. + """ + result = net.run(piped=True) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + target = net.bin / "openRepoShape" + steps = [ + f"release: {RELEASE}, the newest release of {SLUG} ({latest()})", + f"fetch: {raw('openRepoShape')}", + f"install: OPENREPOSHAPE_REF={RELEASE} OPENREPOSHAPE_RECORD_REF=1 " + "openRepoShape --install", + f"openRepoShape: pinned to {RELEASE}, recorded in {target}.ref", + "git-filter-repo: ", + f"preflight: {target} --preflight", + f"done: openRepoShape {RELEASE} is installed at {target}, pinned to " + "the release tag, and this machine is ready. Run the same line again " + "to upgrade to the newest release.", + ] + lines = result.stdout.splitlines() + at = [next(i for i, line in enumerate(lines) if line.startswith(step)) + for step in steps] + assert at == sorted(at), result.stdout + calls = net.calls() + assert calls[0] == f"curl -fsSI {HTTPS_ONLY} {latest()}" + assert calls[1].startswith( + f"curl -fsSL {HTTPS_ONLY} {raw('openRepoShape')} -o ") + assert f"curl -fsSL {HTTPS_ONLY} {raw('setup.sh')}" in calls + assert calls[-1] == preflight_call(), calls + assert not any("releases/latest" in call for call in calls[1:]), calls + + +@pytest.mark.parametrize("piped", [False, True], ids=["file", "piped"]) +def test_a_named_release_is_installed_and_no_newest_is_asked_for(net, piped): + """`bash install.sh v9.9.8` and `… | bash -s -- v9.9.8`: that release, + and no question about which one is newest.""" + net.serve("v9.9.8") + result = net.run("v9.9.8", piped=piped) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result, "v9.9.8") + assert "release: v9.9.8, as named" in result.stdout.splitlines() + assert not any("-fsSI" in call for call in net.calls()), net.calls() + assert net.calls()[-1] == preflight_call("v9.9.8") + + +def test_gh_answers_when_the_redirect_names_no_release(net): + """A redirect to `/releases` names no tag (what github.com answers before + a repository's first release); `gh api` is asked next, where installed.""" + net.config["redirects"][latest()] = f"https://github.com/{SLUG}/releases" + net.config["gh_latest"] = RELEASE + result = net.run() + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"release: {RELEASE}, the newest release of {SLUG} (gh api " \ + f"repos/{SLUG}/releases/latest)" in result.stdout + + +@pytest.mark.parametrize("location, fakes, redirect_fault, gh_fault", [ + pytest.param(f"https://github.com/{SLUG}/releases", ("curl",), + f"it redirects to https://github.com/{SLUG}/releases, which " + "names no release", "gh is not installed", id="no-release"), + pytest.param(None, ("curl", "gh"), "curl failed, exit 22", + "gh failed, exit 1", id="both-failed"), +]) +def test_no_release_found_names_each_way_that_failed( + tmp_path, location, fakes, redirect_fault, gh_fault): + """Each way names its URL or command and why it gave no answer, and the + way out is to name a release; nothing is fetched or installed.""" + net = Net(tmp_path, fakes) + if location is None: + del net.config["redirects"][latest()] + else: + net.config["redirects"][latest()] = location + result = net.run() + net.refused(result) + assert f"could not find the newest release of {SLUG}" in result.stderr + assert f"curl -fsSI {latest()} ({redirect_fault})" in result.stderr + assert f"gh api repos/{SLUG}/releases/latest ({gh_fault})" \ + in result.stderr + assert "bash install.sh v1.0.1" in result.stderr + assert not any("-fsSL" in call for call in net.calls()) + + +@pytest.mark.parametrize("args, said", [ + pytest.param(("v1..v2",), "the version is 'v1..v2'", id="a-range"), + pytest.param(("x.lock",), "the version is 'x.lock'", id="a-lock-file"), + pytest.param(("/etc",), "the version is '/etc'", id="a-path"), + pytest.param(("v1\nforged",), "the version carries a control character", + id="two-lines"), + pytest.param(("a" * 256,), "the version is 256 characters long", + id="over-long"), + pytest.param(("-x",), "unknown option '-x'", id="an-option"), + pytest.param(("-x\nforged",), "an option carries a control character", + id="an-option-of-two-lines"), + pytest.param(("v1.0.1", "v1.0.2"), "two versions given", id="two"), +]) +def test_a_version_that_is_not_a_ref_is_refused_before_any_request( + net, args, said): + """Checked the way the launcher checks a ref, before any URL is built + from it; a control character and the over-long value never echoed.""" + result = net.run(*args) + net.refused(result) + assert said in result.stderr + assert net.calls() == [] + assert "forged" not in result.stderr + assert "a" * 256 not in result.stderr + + +def test_a_failed_fetch_names_its_url_and_installs_nothing(net): + """An error from the server asks whether the release exists.""" + del net.config["files"][raw("openRepoShape")] + result = net.run() + net.refused(result) + assert f"could not fetch {raw('openRepoShape')}: the server answered " \ + "with an error (curl exit 22), so nothing was installed. Is " \ + f"{RELEASE} a release of {SLUG}?" in result.stderr + + +def test_a_fetch_with_no_answer_points_at_the_network_not_the_release(net): + """No answer at all (DNS, a proxy, TLS) is not a question about the + release, and the refusal does not send anyone looking there.""" + net.config["unreachable"] = [raw("openRepoShape")] + result = net.run() + net.refused(result) + assert f"could not fetch {raw('openRepoShape')}: curl got no answer " \ + "(curl exit 6), so nothing was installed. Check this machine's " \ + "network, DNS and proxy" in result.stderr + assert "a release of" not in result.stderr + + +def test_a_page_that_is_not_the_command_is_refused(net): + """A proxy's or a portal's page can arrive with a 200; it is refused + before anything runs it.""" + page = net.root / "page.html" + page.write_text("Sign in\n", + encoding="utf-8") + net.config["files"][raw("openRepoShape")] = str(page) + result = net.run() + net.refused(result) + assert f"{raw('openRepoShape')} is not the openRepoShape command" \ + in result.stderr + + +# --- git-filter-repo --------------------------------------------------------- + +def present_on_path(net): + net.fake("git-filter-repo") + return f"git-filter-repo: already installed ({net.path}/git-filter-repo)" + + +def answering_git(net): + net.fake("git") + net.config["git_filter_repo"] = True + return "git-filter-repo: already installed (git filter-repo --version " \ + "answers)" + + +def pipx_onto_path(net): + return "git-filter-repo: installed with pipx" + + +def pipx_off_path(net): + elsewhere = net.tmp_path / "pipx-bin" + elsewhere.mkdir() + net.config["pipx_installs_into"] = str(elsewhere) + return "git-filter-repo: installed with pipx, into a directory that is " \ + "not on PATH yet: run `pipx ensurepath` and open a new shell" + + +def pipx_failing(net): + net.config["pipx_installs_into"] = None + return f"git-filter-repo: pipx could not install it (exit 1). Before " \ + f"adopting a repository, install it: pipx install {FILTER_REPO}" + + +def no_pipx(net): + (net.path / "pipx").unlink() + return f" python3 -m pip install --user {FILTER_REPO}" + + +def no_pipx_but_brew(net): + no_pipx(net) + net.fake("brew") + net.fake("apt-get") + return " brew install git-filter-repo" + + +def no_pipx_but_apt(net): + no_pipx(net) + net.fake("apt-get") + return f" sudo apt-get install -y pipx && pipx install {FILTER_REPO}" + + +@pytest.mark.parametrize("arrange", [ + present_on_path, answering_git, pipx_onto_path, pipx_off_path, + pipx_failing, no_pipx, no_pipx_but_brew, no_pipx_but_apt]) +def test_git_filter_repo_is_said_installed_or_printed(net, arrange): + """The three branches, and the ways each can end. + + PRESENT, on `$PATH` or as `git filter-repo`: said, and pipx is never + run. NOT PRESENT AND PIPX IS THERE: said first, then `pipx install` at + CI's version, then what came of it. NEITHER: the one command for this + machine, printed and never run: brew's where brew is, pipx from apt where + apt-get is (Debian 12 and Ubuntu 23.04 on refuse `pip install --user`), + and pip otherwise. Nothing runs `sudo` or `apt-get`. None changes the + exit status, and the step sits between the install and the preflight. + """ + expected = arrange(net) + result = net.run() + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + lines = result.stdout.splitlines() + assert expected in lines, result.stdout + ran_pipx = [call for call in net.calls() if call.startswith("pipx")] + if arrange in (pipx_onto_path, pipx_off_path, pipx_failing): + assert ran_pipx == [f"pipx install {FILTER_REPO}"], net.calls() + assert "git-filter-repo: not installed; installing it with pipx, "\ + "because adopt-project.py needs it: pipx install " \ + f"{FILTER_REPO}" in lines + assert net.calls().index(ran_pipx[0]) < \ + net.calls().index(preflight_call()) + else: + assert ran_pipx == [], net.calls() + if arrange in (no_pipx, no_pipx_but_brew, no_pipx_but_apt): + assert lines[lines.index(expected) - 1] == ( + "git-filter-repo: not installed, and there is no pipx to install " + "it with. Only adopt-project.py needs it; before adopting a " + "repository, run:") + assert lines.index(expected) < lines.index( + f"preflight: {net.bin}/openRepoShape --preflight") + assert not any(call.split()[0] in ("apt-get", "brew", "sudo") + for call in net.calls()), net.calls() + + +# --- the preflight, the exit status, and running it again -------------------- + +@pytest.mark.parametrize("status, said", [ + pytest.param(1, "but the preflight did not pass (exit 1); it names what " + "is missing above.", id="not-ready"), + pytest.param(2, "but the preflight did not run to the end (exit 2); the " + "REFUSED line above says why.", id="did-not-run"), +]) +def test_a_preflight_that_does_not_pass_exits_one_with_the_command_placed( + net, status, said): + """Exit 1 is the preflight's own "not ready", which names what is + missing; anything else stopped before it could say.""" + net.config["preflight_exit"] = status + result = net.run() + assert result.returncode == 1, result.stdout + result.stderr + net.installed(result) + assert f"done: openRepoShape {RELEASE} is installed at " \ + f"{net.bin}/openRepoShape, {said}" in result.stdout + + +def test_a_variable_in_the_shell_is_named_and_the_preflight_runs_the_pin(net): + """`$OPENREPOSHAPE_REF` exported in the person's shell overrides the pin + on every run while it stays set, so the installer says so; the release + is still what is recorded, and the preflight still runs through it.""" + result = net.run(env={"OPENREPOSHAPE_REF": "v0.0.1"}) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert "install: $OPENREPOSHAPE_REF is set in this shell, by your " \ + "shell profile or by hand, and while it is, every run uses it " \ + f"instead of {RELEASE}. Remove it from your shell profile to run " \ + f"{RELEASE}." in result.stdout + assert net.calls()[-1] == preflight_call() + + +def test_the_same_line_again_changes_nothing_and_then_upgrades(net): + """Idempotent while the newest release is the same one; the next + release, once it is the newest, replaces the record and says which it + replaced.""" + assert net.run().returncode == 0 + again = net.run() + assert again.returncode == 0, again.stdout + again.stderr + assert f"openRepoShape: already installed at {net.bin}/openRepoShape " \ + "(unchanged)" in again.stdout + assert f"openRepoShape: already pinned to {RELEASE} in " \ + f"{net.bin}/openRepoShape.ref (unchanged)" in again.stdout + + net.config["redirects"][latest()] = tag_page("v10.0.0") + net.serve("v10.0.0") + upgrade = net.run() + assert upgrade.returncode == 0, upgrade.stdout + upgrade.stderr + net.installed(upgrade, "v10.0.0") + assert f"openRepoShape: pinned to v10.0.0 (was {RELEASE}), recorded in " \ + f"{net.bin}/openRepoShape.ref" in upgrade.stdout + + +def test_a_release_whose_command_records_no_ref_exits_one(net): + """A release before v1.0.1 installs a command that follows main. The + installer asks the installed command, says so, and exits 1; and the + preflight it ends with is still handed the release's tag, so it is not + main's setup.sh that checks the machine.""" + old = net.root / "old-openRepoShape" + old.write_text(OLD_LAUNCHER, encoding="utf-8") + net.serve(RELEASE, launcher=old) + result = net.run(env={"OPENREPOSHAPE_BIN_DIR": str(net.bin), + "OPENREPOSHAPE_REPO": SLUG}) + assert result.returncode == 1, result.stdout + result.stderr + assert f"install: {net.bin}/openRepoShape does not stay on {RELEASE}: " \ + f"it says 'openRepoShape ({SLUG} @ main)' to --version." \ + in result.stdout + assert "A release older than v1.0.1 cannot record the tag it was " \ + "installed at: install v1.0.1 or newer, or export " \ + f"OPENREPOSHAPE_REF={RELEASE} in your shell profile to keep it " \ + "there." in result.stdout + assert f"old preflight at {RELEASE}" in result.stdout.splitlines() + assert result.stdout.splitlines()[-1].endswith( + f"but the command does not stay on {RELEASE} (see above).") + + +@pytest.mark.parametrize("header, location", [ + pytest.param("Location", tag_page(RELEASE), id="upper-case"), + pytest.param("location", f"/{SLUG}/releases/tag/{RELEASE}", + id="relative"), +]) +def test_the_redirect_is_read_in_any_case_and_relative(net, header, location): + """HTTP/1.1 may send `Location:`, and a location may be relative; both + name the release the same way.""" + net.config["location_header"] = header + net.config["redirects"][latest()] = location + result = net.run() + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"release: {RELEASE}, the newest release of {SLUG} ({latest()})" \ + in result.stdout.splitlines() + + +def test_openreposhape_bin_dir_is_where_the_command_and_its_record_go(net): + """The installer places, pins and preflights the command where the + launcher puts it, and leaves `~/.local/bin` alone.""" + default = net.bin + net.bin = net.tmp_path / "elsewhere" + result = net.run(env={"OPENREPOSHAPE_BIN_DIR": str(net.bin)}) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert f"preflight: {net.bin}/openRepoShape --preflight" \ + in result.stdout.splitlines() + assert not default.exists() + + +#: A curl that RUNS, in either script: tab-indented, alone or as the value +#: of an assignment. Usage text and refusals carry curl lines too, indented +#: with spaces, and are printed, never run. +CURL_RUN = re.compile(r"\t+(\w+=\"\$\()?curl -fsS") + + +@pytest.mark.parametrize("name", ["install.sh", "openRepoShape"]) +def test_every_curl_that_runs_is_https_only_redirects_included(name): + """SonarCloud shell:S6506, held here rather than found there: each curl + line refuses any protocol but HTTPS, for its redirects too, written out + on the line itself.""" + runs = [line for line in (REPO / name).read_text( + encoding="utf-8").splitlines() if CURL_RUN.match(line)] + assert runs, f"{name} runs no curl; this guard has nothing to hold" + for line in runs: + assert "--proto '=https' --proto-redir '=https' --tlsv1.2" in line, ( + f"{name}: a curl that is not HTTPS only:\n{line}") + + +#: Where a download can stop in the last line. Right after the word `main` +#: is the one the braces are for: unbraced, that line would run `main` with +#: none of the arguments, and install the newest release where one was named. +CUTS = [ + pytest.param(lambda text: text[:text.rindex("{ main")], id="before-it"), + pytest.param(lambda text: text[:text.rindex(" ${1+")], id="after-main"), + pytest.param(lambda text: text[:text.rindex(" }")], id="before-the-brace"), +] + + +@pytest.mark.parametrize("cut", CUTS) +def test_a_download_cut_off_in_the_last_line_runs_nothing(net, cut): + """Everything runs from `main`, called on the last line inside braces: + a script that stops anywhere in or before that line asks for nothing, + installs nothing and leaves nothing behind.""" + result = net.run("v9.9.8", + script=cut(INSTALLER.read_text(encoding="utf-8"))) + assert net.calls() == [] + assert result.stdout == "" + assert not net.bin.exists() + + +# --- the repository it installs from ---------------------------------------- + +def test_openreposhape_repo_names_where_everything_comes_from(net): + """A fork named once is where the release, the command and the + preflight's setup.sh all come from, and the installed command says so.""" + fork = "acme/openRepoShape" + net.config["redirects"] = {latest(fork): tag_page(RELEASE, fork)} + net.config["files"] = {} + net.serve(RELEASE, fork) + result = net.run(env={"OPENREPOSHAPE_REPO": fork}) + assert result.returncode == 0, result.stdout + result.stderr + net.installed(result) + assert all(SLUG not in call for call in net.calls()), net.calls() + assert f"curl -fsSL {HTTPS_ONLY} {raw('setup.sh', RELEASE, fork)}" \ + in net.calls() + + +@pytest.mark.parametrize("value", [ + "https://github.com/acme/openRepoShape.git", "acme", "acme/x/y", + "/srv/openRepoShape"]) +def test_a_repository_that_is_not_an_owner_and_name_is_refused(net, value): + result = net.run(env={"OPENREPOSHAPE_REPO": value}) + net.refused(result) + assert f"$OPENREPOSHAPE_REPO is '{value}', which is not an owner/name " \ + "on github.com" in result.stderr + assert net.calls() == [] + + +# --- what the file says about itself ------------------------------------------ + +def test_help_prints_the_one_line_and_exits_zero(tmp_path): + result = subprocess.run( + [BASH, str(INSTALLER), "--help"], capture_output=True, text=True, + check=False, input="", + env=command_env(home=tmp_path, setup_sh=None, + env={"GH_TOKEN": "bogus", + "GH_CONFIG_DIR": str(tmp_path)})) + assert result.returncode == 0, result.stderr + assert "curl -fsSL https://github.com/opensoft/openRepoShape/releases/" \ + "latest/download/install.sh | bash" in result.stdout + + +def test_it_installs_the_git_filter_repo_ci_runs(): + """What the installer puts on a machine is what the adoption suites ran + against: the pin it names is every pin `.github/workflows/tests.yml` + installs.""" + [pinned] = re.findall(r'^FILTER_REPO_PIN="(git-filter-repo==[^"]+)"$', + INSTALLER.read_text(encoding="utf-8"), re.M) + ci = re.findall(r"git-filter-repo==[0-9.]+", ( + REPO / ".github" / "workflows" / "tests.yml").read_text( + encoding="utf-8")) + assert ci, "tests.yml installs no git-filter-repo" + assert set(ci) == {pinned}, (pinned, ci) diff --git a/tests/test_openreposhape_command.py b/tests/test_openreposhape_command.py index b0a1222a..5e495477 100644 --- a/tests/test_openreposhape_command.py +++ b/tests/test_openreposhape_command.py @@ -34,7 +34,7 @@ import pytest -from conftest import REPO, WINDOWS_SKIP +from conftest import REPO, WINDOWS_SKIP, git COMMAND = REPO / "openRepoShape" SETUP = REPO / "setup.sh" @@ -98,7 +98,8 @@ def command_env(home: Path | None = None, env: dict | None = None, """ environ = dict(os.environ) for name in ("OPENREPOSHAPE_ORG", "OPENREPOSHAPE_REF", "OPENREPOSHAPE_REPO", - "OPENREPOSHAPE_BIN_DIR", "OPENREPOSHAPE_SETUP_SH"): + "OPENREPOSHAPE_BIN_DIR", "OPENREPOSHAPE_SETUP_SH", + "OPENREPOSHAPE_RECORD_REF"): environ.pop(name, None) if setup_sh is not None: environ["OPENREPOSHAPE_SETUP_SH"] = str(setup_sh) @@ -113,7 +114,8 @@ def command_env(home: Path | None = None, env: dict | None = None, def run_cmd(*args: str, home: Path | None = None, env: dict | None = None, - setup_sh: Path | None = SETUP) -> subprocess.CompletedProcess: + setup_sh: Path | None = SETUP, + command: Path = COMMAND) -> subprocess.CompletedProcess: """The command, run from its file, with that environment. `input=""` means stdin is a pipe rather than a terminal, which is what the @@ -121,8 +123,11 @@ def run_cmd(*args: str, home: Path | None = None, env: dict | None = None, controlling terminal either (#492): setup asks at that terminal when stdin is not one, and the terminal pytest was started from is nobody's answer. + + `command` is this checkout's file unless a test names an INSTALLED copy, + which is how the record `--install` leaves beside one is read (#564). """ - return subprocess.run(["bash", str(COMMAND), *args], capture_output=True, + return subprocess.run(["bash", str(command), *args], capture_output=True, text=True, check=False, input="", env=command_env(home, env, setup_sh), start_new_session=True) @@ -1021,3 +1026,297 @@ def test_install_from_stdin_fetches_itself_into_a_live_workdir(offline_github, assert stat.S_IMODE(target.stat().st_mode) == 0o755, name assert target.read_bytes() == (REPO / name).read_bytes(), name assert f"{name}: installed at" in result.stdout + + +# --- the ref `--install` records, and the installed command runs (#564) ----- + +#: A ref for an install to record. Nothing fetches it: every install below +#: runs from this checkout's file, which copies its own bytes, and every run +#: after it is offline. `OTHER` is what `$OPENREPOSHAPE_REF` names for one +#: run, over the record. +PINNED = "v9.9.9" +OTHER = "other" + + +def install_at(tmp_path, ref=None, command=COMMAND, opt_in=True): + """`--install` from `command` into `/bin`, with + `$OPENREPOSHAPE_REF` set to `ref` when one is given, and asked to record + it (`$OPENREPOSHAPE_RECORD_REF=1`) unless `opt_in` is false; the result, + and the path of the command it places.""" + bin_dir = tmp_path / "bin" + env = {"OPENREPOSHAPE_BIN_DIR": str(bin_dir)} + if ref is not None: + env["OPENREPOSHAPE_REF"] = ref + if opt_in: + env["OPENREPOSHAPE_RECORD_REF"] = "1" + return (run_cmd("--install", home=tmp_path, env=env, command=command), + bin_dir / "openRepoShape") + + +def record_of(installed: Path) -> Path: + """Where `--install` records the ref of the command at `installed`.""" + return installed.with_name(installed.name + ".ref") + + +def version_of(installed: Path, env: dict | None = None) -> str: + result = run_cmd("--version", command=installed, env=env) + assert result.returncode == 0, result.stderr + return result.stdout.strip() + + +def test_install_records_the_ref_beside_a_command_it_leaves_unchanged( + tmp_path): + """THE RECORD IS A FILE BESIDE THE COMMAND, AND THE COMMAND IS THE FILE. + + The placed bytes must still be the bytes it was installed from, because + everything that asks whether an installed command IS a release compares + bytes: this installer's own `(unchanged)` verdict, and a copy vendored + elsewhere and checked by sha256 against the release's own file. So the + ref goes in `openRepoShape.ref`, said on its own line between the + command's line and the count, and `--version` then names it with no + variable set, while the variable still wins for the one run it is set + for. + """ + result, installed = install_at(tmp_path, PINNED) + assert result.returncode == 0, result.stderr + record = record_of(installed) + assert record.read_bytes() == f"{PINNED}\n".encode("ascii") + assert installed.read_bytes() == COMMAND.read_bytes(), ( + "the command must stay byte for byte the file it was installed from") + assert result.stdout.splitlines()[:3] == [ + f"openRepoShape: installed at {installed}", + f"openRepoShape: pinned to {PINNED}, recorded in {record}", + f"openRepoShape: 1 of 1 placed in {installed.parent}"] + assert version_of(installed) == \ + f"openRepoShape (opensoft/openRepoShape @ {PINNED})" + assert version_of(installed, {"OPENREPOSHAPE_REF": OTHER}) == \ + f"openRepoShape (opensoft/openRepoShape @ {OTHER})" + + +def test_an_install_with_no_ref_records_nothing_and_follows_main(tmp_path): + """No ref at install time is the install as it always was: the same + lines, no record, and `main`, unnamed, in `--version`.""" + result, installed = install_at(tmp_path) + assert result.returncode == 0, result.stderr + assert not record_of(installed).exists() + assert "pinned" not in result.stdout + assert version_of(installed) == \ + "openRepoShape (opensoft/openRepoShape @ main)" + + +def test_installing_again_keeps_moves_or_drops_the_record(tmp_path): + """Idempotent at the same ref, moved by a new one, dropped by none. + + A second install at the same ref changes neither file and says so twice. + A newer ref replaces the record and names the one it replaced, which is + what an upgrade looks like. The installed command re-installing itself + with no variable set keeps ITS ref, because that is the ref it runs at. + And an install with no ref at all removes the record, so a command put + back at main follows main however it was installed before. + """ + install_at(tmp_path, PINNED) + same, installed = install_at(tmp_path, PINNED) + record = record_of(installed) + assert same.returncode == 0, same.stderr + assert f"openRepoShape: already pinned to {PINNED} in {record} " \ + "(unchanged)" in same.stdout + assert same.stdout.count("(unchanged)") == 2, same.stdout + + newer, _ = install_at(tmp_path, "v10.0.0") + assert f"openRepoShape: pinned to v10.0.0 (was {PINNED}), recorded in " \ + f"{record}" in newer.stdout + itself, _ = install_at(tmp_path, command=installed) + assert "openRepoShape: already pinned to v10.0.0" in itself.stdout + assert record.read_text(encoding="utf-8") == "v10.0.0\n" + + dropped, _ = install_at(tmp_path) + assert dropped.returncode == 0, dropped.stderr + assert "openRepoShape: no longer pinned to v10.0.0; it follows main" \ + in dropped.stdout + assert not record.exists() + assert version_of(installed).endswith("@ main)") + + +def test_a_ref_that_is_not_one_is_refused_before_anything_is_placed( + tmp_path): + """The ref is about to be written down and read by every later run, so + it is checked first, the way the doctor checks it before a checkout.""" + result, installed = install_at(tmp_path, "bad..ref") + assert result.returncode == 2, result.stdout + result.stderr + assert "$OPENREPOSHAPE_REF is 'bad..ref'" in result.stderr + assert not installed.parent.exists(), "a refused install placed something" + + +@pytest.mark.parametrize("text, shown", [ + pytest.param(b"", "is empty", id="empty"), + pytest.param(b"-x\n", "is '-x'", id="an-option"), + pytest.param(b"v1..v2\n", "is 'v1..v2'", id="a-range"), + pytest.param(b"v1\nforged\n", "carries a control character", + id="two-lines"), + pytest.param(b"a" * 256 + b"\n", "is 256 characters long", + id="over-long"), +]) +def test_a_record_that_is_not_a_ref_is_refused_by_name(tmp_path, text, shown): + """A record is a file on disk that every run reads, so it is checked + when it is read, before `--version` can print it, and the refusal names + the file and the two ways out. The control character and the over-long + value are refused without being echoed. `$OPENREPOSHAPE_REF`, set, is + the way past it for one run, because then the record is not read.""" + _result, installed = install_at(tmp_path, PINNED) + record_of(installed).write_bytes(text) + result = run_cmd("--version", command=installed) + assert result.returncode == 2, result.stdout + result.stderr + assert f"the ref recorded in {record_of(installed)} {shown}" \ + in result.stderr + assert "Run the installer again to record a release, or delete that " \ + "file to follow main." in result.stderr + assert "forged" not in result.stderr + assert "a" * 256 not in result.stderr + assert result.stdout == "" + assert run_cmd("--help", command=installed).returncode == 0, ( + "--help must answer past a broken record") + assert version_of(installed, {"OPENREPOSHAPE_REF": OTHER}).endswith( + f"@ {OTHER})") + + +def test_a_ref_set_without_the_opt_in_is_never_recorded(tmp_path): + """`$OPENREPOSHAPE_REF` alone is not a request for a pin (the review on + #577). A script that sets it around `--install` for its own reasons + (workBenches sets one that names no ref, so that any fetch fails loudly) + gets the install it always got: the same bytes, no record, `main` in + `--version`, and no check of a value it never meant to be read. A record + already there is removed, exactly as by an install with no ref.""" + install_at(tmp_path, PINNED) + result, installed = install_at( + tmp_path, "pinned-by-workBenches-no-fetch", opt_in=False) + assert result.returncode == 0, result.stderr + assert installed.read_bytes() == COMMAND.read_bytes() + assert not record_of(installed).exists() + assert f"no longer pinned to {PINNED}" in result.stdout + assert version_of(installed).endswith("@ main)") + + again, _ = install_at(tmp_path, "bad..ref", opt_in=False) + assert again.returncode == 0, again.stderr + assert "pinned" not in again.stdout + assert not record_of(installed).exists() + + +#: Links to the installed command from `/links`: each pair is a link's +#: name and what it points at, `None` meaning the command's absolute path, +#: and the run goes through the LAST one. +LINKS = [ + pytest.param([("ors", "../bin/openRepoShape")], id="relative"), + pytest.param([("ors", None)], id="absolute"), + pytest.param([("ors", "../bin/openRepoShape"), ("again", "ors")], + id="a-chain"), +] + + +@pytest.mark.parametrize("links", LINKS) +def test_a_link_to_the_installed_command_reads_its_record(tmp_path, links): + """The record sits beside the FILE `--install` placed, so a symlink to + the command from another directory follows the link, or a chain of them, + to read it.""" + _result, installed = install_at(tmp_path, PINNED) + directory = tmp_path / "links" + directory.mkdir() + for name, points_at in links: + (directory / name).symlink_to(points_at or installed) + through = directory / links[-1][0] + assert version_of(through) == \ + f"openRepoShape (opensoft/openRepoShape @ {PINNED})" + + +def test_a_record_that_is_a_directory_is_refused_in_the_usual_form(tmp_path): + """Not a raw bash error, by every run that reads it and by an install + that would write or remove it, which places nothing.""" + _result, installed = install_at(tmp_path, PINNED) + record = record_of(installed) + record.unlink() + record.mkdir() + for result in (run_cmd("--version", command=installed), + install_at(tmp_path, PINNED)[0]): + assert result.returncode == 2, result.stdout + result.stderr + assert result.stderr.startswith(f"\nREFUSED: {record} is not a file") + assert "Is a directory" not in result.stderr + assert installed.read_bytes() == COMMAND.read_bytes() + + +#: WHICH REF EACH LATER RUN USES: recorded at install, then overridden by the +#: variable for one run, then neither. The third column is what the run must +#: be handed; `None` is `main`, unnamed, which nothing is told. +RECORDED_RUNS = [ + pytest.param(PINNED, None, PINNED, id="the-record"), + pytest.param(PINNED, OTHER, OTHER, id="the-variable-over-the-record"), + pytest.param(None, None, None, id="no-record"), +] + + +@pytest.mark.parametrize("recorded, variable, expected", RECORDED_RUNS) +def test_setup_sh_clones_the_standard_at_the_ref_in_force( + tmp_path, recorded, variable, expected): + """`--shape-ref` reaches setup.sh whenever a ref was NAMED, and the + record names one: the setup.sh this command fetched at that ref then + clones the standard at the same ref, not at main.""" + _result, installed = install_at(tmp_path, recorded) + stub = tmp_path / "setup.sh" + stub.write_text(STUB_SETUP_SH, encoding="utf-8") + result = run_cmd("--preflight", command=installed, setup_sh=stub, + env=None if variable is None + else {"OPENREPOSHAPE_REF": variable}) + assert result.returncode == 0, result.stdout + result.stderr + match = re.search(rf"{STUB_MARKER} in .+ args: (.*)", result.stdout) + assert match, result.stdout + result.stderr + forwarded = match.group(1).split() + if expected is None: + assert "--shape-ref" not in forwarded, forwarded + else: + at = forwarded.index("--shape-ref") + assert forwarded[at + 1] == expected, forwarded + + +def standard_at_three_refs(tmp_path) -> Path: + """A bare repository whose `shape-doctor.py` says which ref it is. + + Three commits: the `PINNED` tag, the `OTHER` branch and `main`, each + with a doctor that prints its own name and nothing else, so a run + reports which ref the command checked its clone out at. + """ + seed = tmp_path / "seed" + seed.mkdir() + git("init", "-q", "-b", "main", cwd=seed) + + def commit(name): + (seed / "shape-doctor.py").write_text( + f"print('STUB DOCTOR AT {name}')\n", encoding="utf-8") + git("add", "-A", cwd=seed) + git("commit", "-q", "-m", name, cwd=seed) + + commit(PINNED) + git("tag", PINNED, cwd=seed) + git("switch", "-q", "-c", OTHER, cwd=seed) + commit(OTHER) + git("switch", "-q", "main", cwd=seed) + commit("main") + bare = tmp_path / "standard.git" + git("clone", "-q", "--bare", str(seed), str(bare), cwd=tmp_path) + return bare + + +@pytest.mark.parametrize("recorded, variable, expected", RECORDED_RUNS) +def test_the_doctor_checks_its_clone_out_at_the_ref_in_force( + tmp_path, recorded, variable, expected): + """The installed `--doctor` clones the standard, and checks the clone + out at the ref in force: the record, the variable over it, or neither, + which leaves the clone on its default branch.""" + _result, installed = install_at(tmp_path, recorded) + temporary = tmp_path / "tmp" + temporary.mkdir() + env = {"OPENREPOSHAPE_REPO": str(standard_at_three_refs(tmp_path)), + "TMPDIR": str(temporary)} + if variable is not None: + env["OPENREPOSHAPE_REF"] = variable + result = run_cmd("--doctor", str(tmp_path), command=installed, env=env) + assert result.returncode == 0, result.stdout + result.stderr + assert f"STUB DOCTOR AT {expected or 'main'}" in result.stdout, ( + result.stdout + result.stderr) diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index 8d4a4adb..402d39dd 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -68,12 +68,13 @@ "shape_advisory", "conftest", "bootstrap"} #: The bash scripts a person runs BEFORE they have a checkout, or with no -#: checkout in sight at all: the front door itself and the command that -#: fetches it. `park` and `resume` were here from #82 until the carve (#92) -#: and are opensoft/openRepoTools' now, held to these same rules by that -#: repository's own copy of this test. Every one is a shipped executable and -#: is held to the same shebang, mode bit and `set -euo pipefail` rule. -SHIPPED_BASH = ["setup.sh", "openRepoShape"] +#: checkout in sight at all: the front door itself, the command that +#: fetches it, and the one-line installer that places that command pinned to +#: a release (#564). `park` and `resume` were here from #82 until the carve +#: (#92) and are opensoft/openRepoTools' now, held to these same rules by +#: that repository's own copy of this test. Every one is a shipped executable +#: and is held to the same shebang, mode bit and `set -euo pipefail` rule. +SHIPPED_BASH = ["setup.sh", "openRepoShape", "install.sh"] #: The entry point a person runs BEFORE they have a checkout on a machine with #: no bash: the same front door, on an interpreter alone. Held to the same @@ -1408,7 +1409,7 @@ def test_shipped_bash_is_executable_and_fails_loudly(name): @WINDOWS_SKIP @pytest.mark.parametrize("name", SHIPPED_BASH) def test_shipped_bash_parses_under_bash(name): - """`bash -n` on the two shipped bash scripts, everywhere there is a bash. + """`bash -n` on the shipped bash scripts, everywhere there is a bash. SKIPPED ON WINDOWS, AND `shutil.which` IS NOT ENOUGH TO SEE WHY. The `bash` a stock Windows install puts on PATH is @@ -1502,6 +1503,27 @@ def test_the_openrepotools_install_line_is_the_same_everywhere(name): f" {OPENREPOTOOLS_INSTALL}") +#: THE ONE LINE THAT INSTALLS THIS STANDARD (#564), and every file that +#: carries it: the installer it fetches, the command it installs (in its +#: header), the README and its designed reading, and the release notes that +#: introduced it. Pasted by people who cannot check it, like the two lines +#: above, so the copies are held byte for byte. `releases/latest/download/` +#: is what makes it the NEWEST release's `install.sh` every time. +ONE_LINE_INSTALL = ( + "curl -fsSL https://github.com/opensoft/openRepoShape/releases/latest/" + "download/install.sh | bash") +ONE_LINE_INSTALL_IN = ["install.sh", "openRepoShape", "README.md", + "docs/handbook.html", "docs/release-notes/v1.0.1.md"] + + +@pytest.mark.parametrize("name", ONE_LINE_INSTALL_IN) +def test_the_one_line_install_is_the_same_everywhere(name): + text = (REPO / name).read_text(encoding="utf-8") + assert ONE_LINE_INSTALL in text, ( + f"{name} must carry the one-line install byte for byte:\n" + f" {ONE_LINE_INSTALL}") + + #: Where each document describes `openRepoShape --doctor`: the file, the #: line its passage begins with, what ends it, and whether it describes the #: installed form's kept clone. `docs/cli.md` has two: the doctor's purpose in