Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 152 additions & 14 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,32 @@
# failing) on any that don't, since those can only ever be reached by
# `SHAPE_LEGS_TOKEN`. Runs only when the App secrets are present — its
# only consumer is the mint step below, which is guarded the same way.
#
# Each url is read by its exact key: `--name-only` lists the keys, then
# `--get` asks for one, as git reads it. Splitting `--get-regexp`'s
# `key value` lines at the first space gave a section named `members/My
# IRRS` the url `IRRS.url <url>`, which matched neither form and dropped
# that leg from the token (#741). A key holds no line break, so one per
# line is exact; the two steps below that print the urls read them the
# same way. A url holding a line break or a carriage return is skipped
# in all three: it could print a workflow command of its own, and git
# cannot clone it. And each repository is named once, however many
# sections or paths give it.
#
# And only what git reads (#786), by the rules the doctor and `siblings.py`
# read the same file by (#733): only a `submodule.<name>.url` key counts;
# a section whose name is empty or holds a `..` component is ignored, as is
# one with no `path`; a value that begins with `-` is ignored, that value
# alone, so an earlier one stands; nothing is read while `.gitmodules` is
# unmerged; and a file git refuses whole (one it cannot parse, a `path`,
# `url`, `ignore`, `update` or `branch` with no value, an `update` other
# than none, checkout, rebase or merge, a `shallow` that is no boolean, a
# `fetchRecurseSubmodules` that is neither a boolean nor on-demand) fails
# the step with its cause and the key or line, since git reads none of it
# and a repository named from it would be one git never fetches. The two
# steps below that print the urls read by the same key rules; the fetch
# step refuses a file git refuses, and the failing step is never reached
# with one.
- name: leg repository names from .gitmodules
id: leg-names
if: env.SHAPE_LEGS_APP_SET == 'true'
Expand All @@ -64,8 +90,52 @@
run: |
set -e
legs=""
if [ -f .gitmodules ]; then
while IFS= read -r url; do
if [ -f .gitmodules ] && [ -z "$(git ls-files -u -- .gitmodules)" ]; then
# git reads none of a file it refuses, and neither does this (#786). Whatever
# of the file is printed is workflow-command data: % and CR are escaped.
esc() { sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"; }
if ! err=$(git config --file .gitmodules --list 2>&1 >/dev/null); then
err=$(printf '%s\n' "$err" | sed -n '$s/^fatal: //p' | esc)
printf '%s\n' "::error::.gitmodules holds a line git cannot parse${err:+ ($err)}, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the line git stops at."
exit 1
fi
cause() { printf '%s (%s)\n' "$1" "$(printf '%s' "$key" | esc)"; }
us=$(printf '\001')
refusal=$(git config -z --file .gitmodules --list | tr '\000\n' '\n\001' | while IFS= read -r rec; do
case $rec in *"$us"*) value=${rec#*"$us"}; bare= ;; *) value=; bare=y ;; esac
key=${rec%%"$us"*}
section=${key%.*}
case $section in submodule.?*) ;; *) continue ;; esac
case /${section#submodule.}/ in *[/\\]..[/\\]*) continue ;; esac

Check failure on line 109 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M0&open=AaG7SvXisER9q1doP1M0&pullRequest=348
field=${key##*.}
case $field in

Check failure on line 111 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M1&open=AaG7SvXisER9q1doP1M1&pullRequest=348
path|url|ignore|update|branch)
[ -z "$bare" ] || { case $field in ignore|update) a=an ;; *) a=a ;; esac; cause "$a $field key with no value"; break; } ;;
esac
case $field in

Check failure on line 115 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M2&open=AaG7SvXisER9q1doP1M2&pullRequest=348
update)
case $value in none|checkout|rebase|merge) ;; *) cause "an update that is not none, checkout, rebase or merge"; break ;; esac ;;
shallow)
git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a shallow that is not a boolean"; break; } ;;
fetchrecursesubmodules)
[ "$value" = on-demand ] || git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a fetchRecurseSubmodules that is neither a boolean nor on-demand"; break; } ;;
esac
done || true)
if [ -n "$refusal" ]; then
printf '%s\n' "::error::.gitmodules holds $refusal, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the key and line git stops at."
exit 1
fi
while IFS= read -r key; do
case $key in submodule.?*.url) ;; *) continue ;; esac
name=${key#submodule.}
name=${name%.url}
case /$name/ in *[/\\]..[/\\]*) continue ;; esac

Check failure on line 132 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M3&open=AaG7SvXisER9q1doP1M3&pullRequest=348
# A value beginning with - is ignored, that value alone: git keeps an earlier one.
url=$(git config --file .gitmodules --get -- "$key" '^([^-]|$)') || continue
git config --file .gitmodules --get -- "submodule.$name.path" '^([^-]|$)' >/dev/null || continue
# A url holding a line break could print a workflow command of its own.
case $url in *'
'*|*"$(printf '\r')"*) continue ;; esac
[ -n "$url" ] || continue
case "$url" in
https://github.com/*) repo="${url#https://github.com/}" ;;
Expand All @@ -76,6 +146,7 @@
owner="${repo%%/*}"
name="${repo#*/}"
if [ "$owner" = "$REPO_OWNER" ]; then
case ",$legs," in *",$name,"*) continue ;; esac
if [ -z "$legs" ]; then
legs="$name"
else
Expand All @@ -84,7 +155,7 @@
else
echo "::warning::leg $owner/$name is owned by a different account ($owner) than this repository ($REPO_OWNER); a GitHub App installation token is per-owner, so it is excluded from the App-minted token here and can only be reached via SHAPE_LEGS_TOKEN."
fi
done < <(git config --file .gitmodules --get-regexp '\.url$' | cut -d' ' -f2- || true)
done < <(git config --file .gitmodules --name-only --get-regexp '\.url$' || true)
fi
echo "legs=$legs" >> "$GITHUB_OUTPUT"
echo "legs for the App-minted token: $legs"
Expand Down Expand Up @@ -135,7 +206,9 @@
# carry: `https://github.com/...` and SSH `git@github.com:...`. With no
# credential configured this falls back to a plain, unauthenticated
# `git submodule update --init --recursive` — the same command that
# already ran here, unchanged, for a project whose legs are public.
# already ran here, unchanged, for a project whose legs are public. A fetch
# that fails over a `.gitmodules` git refuses is no such degrade: no
# credential cures it, so the step names the cause and fails (#786).
- name: fetch the legs (submodules)
id: submodules
env:
Expand Down Expand Up @@ -169,10 +242,60 @@
echo "legs_available=true" >> "$GITHUB_OUTPUT"
else
echo "legs_available=false" >> "$GITHUB_OUTPUT"
readable=
if [ -f .gitmodules ] && [ -z "$(git ls-files -u -- .gitmodules)" ]; then readable=y; fi
if [ -n "$readable" ]; then
# git reads none of a file it refuses, and neither does this (#786). Whatever
# of the file is printed is workflow-command data: % and CR are escaped.
esc() { sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"; }
if ! err=$(git config --file .gitmodules --list 2>&1 >/dev/null); then
err=$(printf '%s\n' "$err" | sed -n '$s/^fatal: //p' | esc)
printf '%s\n' "::error::.gitmodules holds a line git cannot parse${err:+ ($err)}, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the line git stops at."
exit 1
fi
cause() { printf '%s (%s)\n' "$1" "$(printf '%s' "$key" | esc)"; }
us=$(printf '\001')
refusal=$(git config -z --file .gitmodules --list | tr '\000\n' '\n\001' | while IFS= read -r rec; do
case $rec in *"$us"*) value=${rec#*"$us"}; bare= ;; *) value=; bare=y ;; esac
key=${rec%%"$us"*}
section=${key%.*}
case $section in submodule.?*) ;; *) continue ;; esac
case /${section#submodule.}/ in *[/\\]..[/\\]*) continue ;; esac

Check failure on line 263 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M4&open=AaG7SvXisER9q1doP1M4&pullRequest=348
field=${key##*.}
case $field in

Check failure on line 265 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M5&open=AaG7SvXisER9q1doP1M5&pullRequest=348
path|url|ignore|update|branch)
[ -z "$bare" ] || { case $field in ignore|update) a=an ;; *) a=a ;; esac; cause "$a $field key with no value"; break; } ;;
esac
case $field in

Check failure on line 269 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M6&open=AaG7SvXisER9q1doP1M6&pullRequest=348
update)
case $value in none|checkout|rebase|merge) ;; *) cause "an update that is not none, checkout, rebase or merge"; break ;; esac ;;
shallow)
git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a shallow that is not a boolean"; break; } ;;
fetchrecursesubmodules)
[ "$value" = on-demand ] || git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a fetchRecurseSubmodules that is neither a boolean nor on-demand"; break; } ;;
esac
done || true)
if [ -n "$refusal" ]; then
printf '%s\n' "::error::.gitmodules holds $refusal, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the key and line git stops at."
exit 1
fi
fi
echo "::warning::submodule checkout failed (git exit $status) using credential source '$legs_credential'. If the legs are PRIVATE or internal sibling repositories, the default GITHUB_TOKEN cannot read them. Configure a GitHub App (SHAPE_LEGS_APP_ID + SHAPE_LEGS_APP_PRIVATE_KEY, preferred) or set SHAPE_LEGS_TOKEN — a fine-grained PAT — with contents:read on the legs, as a repository or organisation secret. Leg URLs:"
git config --file .gitmodules --get-regexp '\.url$' | while read -r _ url; do
echo "::warning:: $url"
done
if [ -n "$readable" ]; then
git config --file .gitmodules --name-only --get-regexp '\.url$' | awk '!seen[$0]++' | while IFS= read -r key; do
case $key in submodule.?*.url) ;; *) continue ;; esac
name=${key#submodule.}
name=${name%.url}
case /$name/ in *[/\\]..[/\\]*) continue ;; esac

Check failure on line 289 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M7&open=AaG7SvXisER9q1doP1M7&pullRequest=348
# A value beginning with - is ignored, that value alone: git keeps an earlier one.
url=$(git config --file .gitmodules --get -- "$key" '^([^-]|$)') || continue
git config --file .gitmodules --get -- "submodule.$name.path" '^([^-]|$)' >/dev/null || continue
# A url holding a line break could print a workflow command of its own.
case $url in *'
'*|*"$(printf '\r')"*) continue ;; esac
echo "::warning:: $url"
done
fi
fi

- uses: actions/setup-python@v5
Expand Down Expand Up @@ -209,15 +332,30 @@
# a credential that IS configured (App or PAT) that still could not
# check out the legs (wrong scope, missing contents:read grant, an
# incomplete App installation, a leg URL problem) is a finding a human
# must act on, so this fails the job — the only case that does. The
# root checkout above never depended on either credential, so a
# failure here can only mean the credential named below cannot read one
# of the LEG repositories.
# must act on, so this fails the job — the only case that does, but for a
# `.gitmodules` git refuses, which the fetch step above fails whatever the
# credential, since none cures it. The root checkout above never depended
# on either credential, so a failure here can only mean the credential
# named below cannot read one of the LEG repositories.
- name: fail — a leg credential is configured but the legs still could not be fetched
if: steps.submodules.outputs.legs_available != 'true' && (env.SHAPE_LEGS_TOKEN_SET == 'true' || env.SHAPE_LEGS_APP_SET == 'true')
run: |
echo "::error::A leg credential is configured (source: ${{ steps.submodules.outputs.legs_credential }}) but 'git submodule update --init --recursive' still failed. The root checkout does not use this token — it reads with the workflow's default token — so this means the '${{ steps.submodules.outputs.legs_credential }}' credential cannot read one or more of the following LEG repositories (wrong token/App scope, a missing contents:read grant, an incomplete App installation, or a leg URL problem):"
git config --file .gitmodules --get-regexp '\.url$' | while read -r _ url; do
echo "::error:: $url"
done
if [ -f .gitmodules ] && [ -z "$(git ls-files -u -- .gitmodules)" ]; then
# No refusal here: the fetch step above exits 1 on a file git refuses, so
# this step is never reached with one.
git config --file .gitmodules --name-only --get-regexp '\.url$' | awk '!seen[$0]++' | while IFS= read -r key; do
case $key in submodule.?*.url) ;; *) continue ;; esac
name=${key#submodule.}
name=${name%.url}
case /$name/ in *[/\\]..[/\\]*) continue ;; esac

Check failure on line 351 in .github/workflows/validate.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoTools&issues=AaG7SvXisER9q1doP1M8&open=AaG7SvXisER9q1doP1M8&pullRequest=348
# A value beginning with - is ignored, that value alone: git keeps an earlier one.
url=$(git config --file .gitmodules --get -- "$key" '^([^-]|$)') || continue
git config --file .gitmodules --get -- "submodule.$name.path" '^([^-]|$)' >/dev/null || continue
# A url holding a line break could print a workflow command of its own.
case $url in *'
'*|*"$(printf '\r')"*) continue ;; esac
echo "::error:: $url"
done
fi
exit 1
16 changes: 8 additions & 8 deletions contracts/shape-pin.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,36 +27,36 @@ pin_role: shape
source_repository: opensoft/openRepoShape
materialization: copied

commit: "190133d5e704e380c430f87ca9c4e31746de0012"
commit: "50178305e4229275d18e71c2152403b4e9d3004e"
revision_kind: commit

digest_algorithm: sha256
digest_definition: sorted-ls-tree-r-v1
digests:
tree_sha256: "7a66972c7de193c050f77f97bffa13d8543c28d74f61ea210644e65aa3f37912"
tree_sha256: "204d73aabf2379554600234d4b36dc08d3c3a1da422395dec44637b5ca7a5184"

verify_pin: scripts/validate-pins.py

# The copies that live in THIS repository, each with the sha256 of its bytes
# as written by the scaffold. Written by `scaffold-project.py`; do not hand-edit.
files:
- path: scripts/validate-pins.py
sha256: "09820c5dd29355bdbbd3c82bb6197f31e7a0a234e8beaaa5940862d09c32597f"
sha256: "7acc22fc632a046aec692dd34550ab65fef6eaf4b89278f1bb5e6b73b860e2e7"
- path: scripts/validate-manifest.py
sha256: "7f36200f55562ab3e9588c00aaaefb0763b74f979b07d4688b70be38b1236adc"
sha256: "bc60bd26fc6f3104bf499f80ba694afc102458e32ab57092d127aa59980dfdda"
- path: scripts/bootstrap.py
sha256: "d2bd1cfc8515088980df66c8adda3fdaea908e7e2eba6f94062c1b9e65610995"
sha256: "89ff92e5e469003fb5a1f13a5adbe5be2f9c2913ad152eb3f49e8545f0a8c3cc"
- path: Makefile
sha256: "6d65dd06e12564a7c018a72a5a2e3c80b999d7026ce5643e77ca8dd6e9254dfa"
- path: shape/.gitattributes
sha256: "e2b88cf25928c5a58bb2acaae1779d934c262273db708d0aa6dfb01591b15baa"
- path: .github/workflows/validate.yml
sha256: "fb8c0360863ad953af7825f3be01ed27f8cf1269e5cab19dc0d6c5423161b487"
sha256: "58726aabbec255f79ce629a5f60529efa28c64c5d0d24384a2f676ff70b06a92"
- path: AGENTS-shape.md
sha256: "84f8bf1afb83d7cd5bd556f81a3bfc01319f00ff87c33a95e98cf5d1fada5fee"
- path: scripts/repo_shape.py
sha256: "be5358aad1384f723e321bb089e1c04bfb552157f922554e9f872fae9372a904"
sha256: "2ae28ea2af0fbf508cea28c1f8136afdf7737588de0130e510e1ad6ef14c108e"
- path: scripts/validate-repository-naming.py
sha256: "527273cf0b8ada97166c7d60e2247c2a44e16983bf643843936ce92b19bb0f68"
sha256: "39a57203e2a142335e7b89ea2d69a9979092e70099e954d10e31f3fb9a7a8cec"
- path: contracts/repository-naming.yaml
sha256: "a4eebad21541cbfe9a3e94a0ee667a2d5a7866c3567eea2a000d8339f6f096a4"
4 changes: 2 additions & 2 deletions project.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,11 @@ tracking_branch: "main"
shape:
repository: opensoft/openRepoShape
revision_kind: commit
commit: "190133d5e704e380c430f87ca9c4e31746de0012"
commit: "50178305e4229275d18e71c2152403b4e9d3004e"
digest_algorithm: sha256
digest_definition: sorted-ls-tree-r-v1
digests:
tree_sha256: "7a66972c7de193c050f77f97bffa13d8543c28d74f61ea210644e65aa3f37912"
tree_sha256: "204d73aabf2379554600234d4b36dc08d3c3a1da422395dec44637b5ca7a5184"

# The neutral products this project declares a pin on, by name. OPTIONAL, and
# empty in a fresh scaffold. It is what makes a `<Domainx><Product>` name a
Expand Down
Loading
Loading