validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786)

README.md

@@ -676,7 +676,8 @@ fetch still failed — naming which source it used, meaning that credential
 cannot read one of the LEG repositories, never the root, which no longer
 depends on it — checked via job-level `env: SHAPE_LEGS_APP_SET` /
 `SHAPE_LEGS_TOKEN_SET`, not `secrets` in the step `if:` (disallowed there;
-MedxEHR PR #8, MedxGlass PR #1: zero-job push runs).
+MedxEHR PR #8, MedxGlass PR #1: zero-job push runs). A `.gitmodules` that git
+refuses fails the run too, whatever the credential, naming the key or line.
 
 ### A descendant form is a claim; a claim needs a referent
 

docs/handbook.html

@@ -474,7 +474,7 @@
 
       <!-- reads README.md § Starting a project in a new organisation: blob 45a3f886c0203e91f17a7851cf2c1c71a2681b0c -->
 
-      <!-- reads README.md § The three legs: blob 1d9f5bf98bddd1f70522c86515de3f15900fe141 -->
+      <!-- reads README.md § The three legs: blob 4ec313e8424665e41fc92a4147808e0bd4901d78 -->
 
       <!-- reads README.md § The double pin, and the lockstep invariant: blob 57fffa08ffad5476f8eae73dd3e8494a02e050c9 -->
 
@@ -1126,7 +1126,7 @@ <h2>If it fails part-way</h2>
 
       <!-- reads README.md § Starting a project in a new organisation: blob 45a3f886c0203e91f17a7851cf2c1c71a2681b0c -->
 
-      <!-- reads README.md § The three legs: blob 1d9f5bf98bddd1f70522c86515de3f15900fe141 -->
+      <!-- reads README.md § The three legs: blob 4ec313e8424665e41fc92a4147808e0bd4901d78 -->
 
       <p class="eyebrow">Setup</p>
       <h2>Reading private legs in CI</h2>
@@ -1179,7 +1179,8 @@ <h2>Reading private legs in CI</h2>
         naming and manifest checks and skips <code>validate-pins.py</code> with a warning explaining
         why. It fails outright only if a credential <strong>is</strong> configured and the fetch
         still failed &mdash; meaning that credential cannot read one of the LEG repositories, never
-        the root.</li>
+        the root. A <code>.gitmodules</code> that git refuses fails the run too, whatever the
+        credential, naming the key or line.</li>
       </ul>
       <a class="src" href="https://github.com/opensoft/openRepoShape#reading-private-legs-in-ci-a-github-app-first-shape_legs_token-as-fallback">Full text &mdash; private legs in CI</a>
     </section>
@@ -1188,7 +1189,7 @@ <h2>Reading private legs in CI</h2>
 
       <!-- reads README.md § Starting a project in a new organisation: blob 45a3f886c0203e91f17a7851cf2c1c71a2681b0c -->
 
-      <!-- reads README.md § The three legs: blob 1d9f5bf98bddd1f70522c86515de3f15900fe141 -->
+      <!-- reads README.md § The three legs: blob 4ec313e8424665e41fc92a4147808e0bd4901d78 -->
 
       <p class="eyebrow">Reference</p>
       <h2>The three legs, and naming</h2>

templates/assembly-root/.github/workflows/validate.yml

@@ -67,6 +67,21 @@ jobs:
       # in all three: it could print a workflow command of its own, and git
       # cannot clone it. And each repository is named once, however many
       # sections or paths give it.
+      #
+      # And only what git reads (#786), by the rules the doctor and `siblings.py`
+      # read the same file by (#733): only a `submodule.<name>.url` key counts;
+      # a section whose name is empty or holds a `..` component is ignored, as is
+      # one with no `path`; a value that begins with `-` is ignored, that value
+      # alone, so an earlier one stands; nothing is read while `.gitmodules` is
+      # unmerged; and a file git refuses whole (one it cannot parse, a `path`,
+      # `url`, `ignore`, `update` or `branch` with no value, an `update` other
+      # than none, checkout, rebase or merge, a `shallow` that is no boolean, a
+      # `fetchRecurseSubmodules` that is neither a boolean nor on-demand) fails
+      # the step with its cause and the key or line, since git reads none of it
+      # and a repository named from it would be one git never fetches. The two
+      # steps below that print the urls read by the same key rules; the fetch
+      # step refuses a file git refuses, and the failing step is never reached
+      # with one.
       - name: leg repository names from .gitmodules
         id: leg-names
         if: env.SHAPE_LEGS_APP_SET == 'true'
@@ -75,9 +90,49 @@ jobs:
         run: |
           set -e
           legs=""
-          if [ -f .gitmodules ]; then
+          if [ -f .gitmodules ] && [ -z "$(git ls-files -u -- .gitmodules)" ]; then
+            # git reads none of a file it refuses, and neither does this (#786). Whatever
+            # of the file is printed is workflow-command data: % and CR are escaped.
+            esc() { sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"; }
+            if ! err=$(git config --file .gitmodules --list 2>&1 >/dev/null); then
+              err=$(printf '%s\n' "$err" | sed -n '$s/^fatal: //p' | esc)
+              printf '%s\n' "::error::.gitmodules holds a line git cannot parse${err:+ ($err)}, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the line git stops at."
+              exit 1
+            fi
+            cause() { printf '%s (%s)\n' "$1" "$(printf '%s' "$key" | esc)"; }
+            us=$(printf '\001')
+            refusal=$(git config -z --file .gitmodules --list | tr '\000\n' '\n\001' | while IFS= read -r rec; do
+              case $rec in *"$us"*) value=${rec#*"$us"}; bare= ;; *) value=; bare=y ;; esac
+              key=${rec%%"$us"*}
+              section=${key%.*}
+              case $section in submodule.?*) ;; *) continue ;; esac
+              case /${section#submodule.}/ in *[/\\]..[/\\]*) continue ;; esac
+              field=${key##*.}
+              case $field in
+                path|url|ignore|update|branch)
+                  [ -z "$bare" ] || { case $field in ignore|update) a=an ;; *) a=a ;; esac; cause "$a $field key with no value"; break; } ;;
+              esac
+              case $field in
+                update)
+                  case $value in none|checkout|rebase|merge) ;; *) cause "an update that is not none, checkout, rebase or merge"; break ;; esac ;;
+                shallow)
+                  git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a shallow that is not a boolean"; break; } ;;
+                fetchrecursesubmodules)
+                  [ "$value" = on-demand ] || git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a fetchRecurseSubmodules that is neither a boolean nor on-demand"; break; } ;;
+              esac
+            done || true)
+            if [ -n "$refusal" ]; then
+              printf '%s\n' "::error::.gitmodules holds $refusal, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the key and line git stops at."
+              exit 1
+            fi
             while IFS= read -r key; do
-              url=$(git config --file .gitmodules --get -- "$key") || continue
+              case $key in submodule.?*.url) ;; *) continue ;; esac
+              name=${key#submodule.}
+              name=${name%.url}
+              case /$name/ in *[/\\]..[/\\]*) continue ;; esac
+              # A value beginning with - is ignored, that value alone: git keeps an earlier one.
+              url=$(git config --file .gitmodules --get -- "$key" '^([^-]|$)') || continue
+              git config --file .gitmodules --get -- "submodule.$name.path" '^([^-]|$)' >/dev/null || continue
               # A url holding a line break could print a workflow command of its own.
               case $url in *'
           '*|*"$(printf '\r')"*) continue ;; esac
@@ -151,7 +206,9 @@ jobs:
       # carry: `https://github.com/...` and SSH `git@github.com:...`. With no
       # credential configured this falls back to a plain, unauthenticated
       # `git submodule update --init --recursive` — the same command that
-      # already ran here, unchanged, for a project whose legs are public.
+      # already ran here, unchanged, for a project whose legs are public. A fetch
+      # that fails over a `.gitmodules` git refuses is no such degrade: no
+      # credential cures it, so the step names the cause and fails (#786).
       - name: fetch the legs (submodules)
         id: submodules
         env:
@@ -185,14 +242,60 @@ jobs:
             echo "legs_available=true" >> "$GITHUB_OUTPUT"
           else
             echo "legs_available=false" >> "$GITHUB_OUTPUT"
+            readable=
+            if [ -f .gitmodules ] && [ -z "$(git ls-files -u -- .gitmodules)" ]; then readable=y; fi
+            if [ -n "$readable" ]; then
+              # git reads none of a file it refuses, and neither does this (#786). Whatever
+              # of the file is printed is workflow-command data: % and CR are escaped.
+              esc() { sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"; }
+              if ! err=$(git config --file .gitmodules --list 2>&1 >/dev/null); then
+                err=$(printf '%s\n' "$err" | sed -n '$s/^fatal: //p' | esc)
+                printf '%s\n' "::error::.gitmodules holds a line git cannot parse${err:+ ($err)}, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the line git stops at."
+                exit 1
+              fi
+              cause() { printf '%s (%s)\n' "$1" "$(printf '%s' "$key" | esc)"; }
+              us=$(printf '\001')
+              refusal=$(git config -z --file .gitmodules --list | tr '\000\n' '\n\001' | while IFS= read -r rec; do
+                case $rec in *"$us"*) value=${rec#*"$us"}; bare= ;; *) value=; bare=y ;; esac
+                key=${rec%%"$us"*}
+                section=${key%.*}
+                case $section in submodule.?*) ;; *) continue ;; esac
+                case /${section#submodule.}/ in *[/\\]..[/\\]*) continue ;; esac
+                field=${key##*.}
+                case $field in
+                  path|url|ignore|update|branch)
+                    [ -z "$bare" ] || { case $field in ignore|update) a=an ;; *) a=a ;; esac; cause "$a $field key with no value"; break; } ;;
+                esac
+                case $field in
+                  update)
+                    case $value in none|checkout|rebase|merge) ;; *) cause "an update that is not none, checkout, rebase or merge"; break ;; esac ;;
+                  shallow)
+                    git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a shallow that is not a boolean"; break; } ;;
+                  fetchrecursesubmodules)
+                    [ "$value" = on-demand ] || git -c "x.y=$value" config --type=bool --get x.y >/dev/null 2>&1 || { cause "a fetchRecurseSubmodules that is neither a boolean nor on-demand"; break; } ;;
+                esac
+              done || true)
+              if [ -n "$refusal" ]; then
+                printf '%s\n' "::error::.gitmodules holds $refusal, so git refuses the whole file: it reads none of it, and no repository is named from it. git submodule status names the key and line git stops at."
+                exit 1
+              fi
+            fi
             echo "::warning::submodule checkout failed (git exit $status) using credential source '$legs_credential'. If the legs are PRIVATE or internal sibling repositories, the default GITHUB_TOKEN cannot read them. Configure a GitHub App (SHAPE_LEGS_APP_ID + SHAPE_LEGS_APP_PRIVATE_KEY, preferred) or set SHAPE_LEGS_TOKEN — a fine-grained PAT — with contents:read on the legs, as a repository or organisation secret. Leg URLs:"
-            git config --file .gitmodules --name-only --get-regexp '\.url$' | while IFS= read -r key; do
-              url=$(git config --file .gitmodules --get -- "$key") || continue
-              # A url holding a line break could print a workflow command of its own.
-              case $url in *'
+            if [ -n "$readable" ]; then
+              git config --file .gitmodules --name-only --get-regexp '\.url$' | awk '!seen[$0]++' | while IFS= read -r key; do
+                case $key in submodule.?*.url) ;; *) continue ;; esac
+                name=${key#submodule.}
+                name=${name%.url}
+                case /$name/ in *[/\\]..[/\\]*) continue ;; esac
+                # A value beginning with - is ignored, that value alone: git keeps an earlier one.
+                url=$(git config --file .gitmodules --get -- "$key" '^([^-]|$)') || continue
+                git config --file .gitmodules --get -- "submodule.$name.path" '^([^-]|$)' >/dev/null || continue
+                # A url holding a line break could print a workflow command of its own.
+                case $url in *'
           '*|*"$(printf '\r')"*) continue ;; esac
-              echo "::warning::  $url"
-            done
+                echo "::warning::  $url"
+              done
+            fi
           fi
 
       - uses: actions/setup-python@v5
@@ -229,19 +332,30 @@ jobs:
       # a credential that IS configured (App or PAT) that still could not
       # check out the legs (wrong scope, missing contents:read grant, an
       # incomplete App installation, a leg URL problem) is a finding a human
-      # must act on, so this fails the job — the only case that does. The
-      # root checkout above never depended on either credential, so a
-      # failure here can only mean the credential named below cannot read one
-      # of the LEG repositories.
+      # must act on, so this fails the job — the only case that does, but for a
+      # `.gitmodules` git refuses, which the fetch step above fails whatever the
+      # credential, since none cures it. The root checkout above never depended
+      # on either credential, so a failure here can only mean the credential
+      # named below cannot read one of the LEG repositories.
       - name: fail — a leg credential is configured but the legs still could not be fetched
         if: steps.submodules.outputs.legs_available != 'true' && (env.SHAPE_LEGS_TOKEN_SET == 'true' || env.SHAPE_LEGS_APP_SET == 'true')
         run: |
           echo "::error::A leg credential is configured (source: ${{ steps.submodules.outputs.legs_credential }}) but 'git submodule update --init --recursive' still failed. The root checkout does not use this token — it reads with the workflow's default token — so this means the '${{ steps.submodules.outputs.legs_credential }}' credential cannot read one or more of the following LEG repositories (wrong token/App scope, a missing contents:read grant, an incomplete App installation, or a leg URL problem):"
-          git config --file .gitmodules --name-only --get-regexp '\.url$' | while IFS= read -r key; do
-            url=$(git config --file .gitmodules --get -- "$key") || continue
-            # A url holding a line break could print a workflow command of its own.
-            case $url in *'
+          if [ -f .gitmodules ] && [ -z "$(git ls-files -u -- .gitmodules)" ]; then
+            # No refusal here: the fetch step above exits 1 on a file git refuses, so
+            # this step is never reached with one.
+            git config --file .gitmodules --name-only --get-regexp '\.url$' | awk '!seen[$0]++' | while IFS= read -r key; do
+              case $key in submodule.?*.url) ;; *) continue ;; esac
+              name=${key#submodule.}
+              name=${name%.url}
+              case /$name/ in *[/\\]..[/\\]*) continue ;; esac
+              # A value beginning with - is ignored, that value alone: git keeps an earlier one.
+              url=$(git config --file .gitmodules --get -- "$key" '^([^-]|$)') || continue
+              git config --file .gitmodules --get -- "submodule.$name.path" '^([^-]|$)' >/dev/null || continue
+              # A url holding a line break could print a workflow command of its own.
+              case $url in *'
           '*|*"$(printf '\r')"*) continue ;; esac
-            echo "::error::  $url"
-          done
+              echo "::error::  $url"
+            done
+          fi
           exit 1

templates/assembly-root/README.md

@@ -110,7 +110,9 @@ fetch still failed — naming which source it used. That presence check reads
 job-level `env: SHAPE_LEGS_APP_SET` / `SHAPE_LEGS_TOKEN_SET` booleans rather
 than the `secrets` context directly in the step's `if:` — the `secrets`
 context is not allowed in a step-level `if:` expression, and using it there
-makes GitHub reject the whole workflow file instead of just that step.
+makes GitHub reject the whole workflow file instead of just that step. A
+`.gitmodules` that git refuses fails the run too, whatever the credential,
+naming the key or line.
 
 ## The lockstep invariant