Skip to content

shape-doctor.py: every git the doctor starts runs with GIT_NO_LAZY_FETCH=1, so a read of an object a partial clone lacks fails rather than fetches, and pin_restore names the fetch where a root's partial clone lacks its pin's history (#787) - #791

Merged
brettheap merged 10 commits into
mainfrom
l3-787-no-lazy-fetch
Oct 18, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-3 (openRepoShape-3)
Claim: #787 (comment)

Closes #787.

Stacked on #773; lands after it. The base is main, so until #773 lands this diff also shows #773's commits (l3-754-promisor-next, head 296a6dd6). This PR's own changes are the three commits on top: 152e9c39; 19665420, which moves one function ("Merging with main" below); and 1d774ab3, the NIT round ("Round 2 (NIT)" below).

Lands after the v1.1.0 tag.

Round 2 (NIT) (head 1d774ab3, on 19665420)

RV-f's review (comment) was READY with three NITs. This round takes all three.

  • N1, the partial why. It said "which commit broke the pin cannot be read here", which is wrong where HEAD was moved back without a checkout: there the doctor run after the fetch answers uncommitted. It now says "the pin's history, HEAD's included, is not all in this clone" (partial_history_why, shape-doctor.py:4802, wording at :4820). pin_restore's docstring entry (:3850) says the same.
    • No test held the old words. The partial-root test now asserts the new ones.
    • It gains RV-f's reset --soft case, which answers uncommitted after the fetch (PARTIAL_ROOTS, tests/test_doctor_no_lazy_fetch.py:164).
  • N2, "every git it starts". test_every_git_the_run_starts_has_the_variable (:242) puts a recording git first on PATH (recording, :222). For each git the run starts, it logs the variable's value, the parent's command line and the arguments. Every one has 1, including:
    • the gits validate-pins.py starts, which run_validator launched;
    • Upstream.blob's cat-file blob <commit>:templates/…, which the doctor drives in its own process.
  • N3, other promisor remotes. history_left_out (:4834) returns every promisor remote, not the first. The why adds "where <first>'s branches do not hold it, the same from <name>: git -C … fetch --refetch --no-filter <name>" for each other one (:4825).
    • test_the_why_spells_out_the_fetch_from_each_other_promisor_remote (:200) points origin at another repository and adds the history's remote as an unfetched promisor second.
    • Run as printed, the NEXT leaves the row partial. The why's second line, run as printed, gives committed.
  • The NEXT itself is unchanged: it still fetches the first promisor remote's branches.

Each helper sits beside history_left_out, after remotes_holding, so the branch still merges clean with main bf1f099b.

Mutants (each run against tests/test_doctor_no_lazy_fetch.py):

mutant tests it fails
main not under lazy_fetch_off 5: the three partial-root cases, the N3 test and the N2 test
the variable set to 0 7
the environment not put back 6
no partial answer 4
a failed probe read as "lacks nothing" 1 (tree:0)
run_validator drops the variable 1: the N2 test
round 1's why words 3: the partial-root cases
no other remote's command in the why 1: the N3 test
history_left_out returns the first remote only 1: the N3 test

Results on 1d774ab3, with the same stub gh and command as below:

  • tests/test_doctor_no_lazy_fetch.py, tests/test_doctor_pin_restore_merges.py, tests/test_doctor_partial_clone.py, tests/test_repo_hygiene.py and tests/test_cli_reference.py, together: 162 passed in 204.36s (0:03:24).
  • tests/test_shape_doctor.py -k "pin or partial": 88 passed, 1 skipped, 351 deselected in 217.43s (0:03:37).
  • On the predicted merge of 1d774ab3 into main bf1f099b, tests/test_doctor_no_lazy_fetch.py, tests/test_doctor_pin_restore_merges.py and tests/test_doctor_partial_clone.py: 69 passed in 202.20s (0:03:22).

What was measured first (base 296a6dd6, git 2.43.0)

A probe ran the doctor, --json and the table, under GIT_TRACE2_EVENT, which logs every git process and every fetch child it starts. Each of 14 scenarios ran twice: as is, and with a git shim first on PATH that unsets GIT_NO_LAZY_FETCH, which is the doctor before this change.

A partial clone of the standard: no row fetches today. That covers blob:none and tree:0, behind the pin or pinned at its own HEAD, with the remote in reach or not, plus two full clones. In all ten scenarios:

  • 0 fetches, before and after;
  • the report is byte-identical, stdout and stderr, --json and the table.

The guards from #683 and #754 already cover every read there. For the standard, this change makes the promise hold by construction rather than read by read.

A partial clone of the ROOT whose HEAD commit broke its pin: the doctor fetched.

  • pin_restore walks the pin's history, and today that made 1 lazy fetch (blob:none) or 2 (tree:0): git -c fetch.negotiationAlgorithm=noop fetch origin … --stdin.
  • With the variable alone, the walk read the failed read as "no pin there". The row then said "no commit in this root holds a pin to put back", which is not so. So this PR also changes that row (below).
  • A partial clone of the root with a pin that reads: 0 fetches either way, byte-identical.

What changed (shape-doctor.py)

  • lazy_fetch_off (:10100, the variable's name NO_LAZY_FETCH at :10096) is a context manager. It sets GIT_NO_LAZY_FETCH=1 in this process's environment and puts back what was there on the way out.
  • main runs under it (@lazy_fetch_off(), :10146). Every git child inherits it:
    • the doctor's own reads: git_out (from scripts/repo_shape.py, which is unchanged), git_answer, and its direct subprocess.run of git;
    • update-shape.py's Upstream reads (cat-file, ls-tree, rev-parse, tree_digest), which the doctor loads and drives in this process;
    • the validators run_validator starts with a copy of os.environ, and their gits.
  • No printed line carries it. A NEXT is text for a person's shell, so the partial-clone row's fetch --refetch --no-filter still fetches when pasted.
  • Which git honours it, read from git's source by tag, and said in the docstring (:10128):
    • v2.45.0 reads it in environment.c. v2.45.1 and later, up to v2.55.0 (CI's), read it in promisor-remote.c's fetch_objects, with the warning "lazy fetching disabled; some objects may not be available".
    • The 2024-05-14 security releases for CVE-2024-32465 read it there too: v2.39.4, v2.40.2, v2.41.1, v2.42.2, v2.43.4 and v2.44.1.
    • Ubuntu's 1:2.43.0-1ubuntu7.1 and later carry that patch (CVE-2024-32465.patch), which is why this machine's 2.43.0 honours it. Measured: cat-file of a missing blob gives exit 128, the warning, "could not fetch … from promisor remote", and no fetch child.
    • v2.38.0, v2.40.1, v2.43.0 and v2.44.0 do not read it. There the variable is ignored, harmlessly, and the existing guards keep the doctor from fetching, as before.
  • pin_restore has a new answer, partial (:4062, documented at :3850). Where the walk finds no pin to put back, history_left_out (:4834) asks without fetching whether the root is a partial clone that lacks the pin's history:
    • it runs rev-list --objects --missing=print -n 51 HEAD -- <pin>;
    • a ?<id> line means it lacks the history; so does a failure, which is what a tree:0 clone gives, since that walk cannot read the trees.
    • Where it lacks it, restore is partial and the NEXT is git -C <root> fetch --refetch --no-filter <first promisor remote>. The why (partial_history_why, :4802) says the pin's history, HEAD's included, is not all in this clone, and spells out the same fetch from each other promisor remote (round 2).
    • It reaches the manifest kinds row's files too, since that row uses the same pin_restore.
    • The row stays a FINDING: the pin does not read, whatever the clone holds.
  • The module docstring (:31) says every git the doctor starts runs with the variable.

Pinned copies: none. shape-doctor.py is not copied into projects, and scripts/repo_shape.py and update-shape.py are unchanged.

Docs: no change. README, AGENTS.md and the handbook say the doctor fetches nothing, which now holds by construction. No doc lists pin_restore's restore values, and docs/cli.md takes only the docstring's first line.

Tested (tests/test_doctor_no_lazy_fetch.py, new)

  • test_every_row_from_a_partial_clone_of_the_standard_starts_no_fetch (:88; blob:none and tree:0, remote out of reach and in reach):
    • every row is asked, exit 3;
    • the trace saw the run's gits;
    • no git started a fetch (trace2 child_start);
    • rev-list --objects --all --missing=print is the same before and after.
  • test_the_report_is_the_same_byte_for_byte_with_the_variable_ignored (:131): the full-clone control, plus blob:none and tree:0 partial clones of the standard. --json and the table are run as is and through the shim, and stdout, stderr and the exit are the same.
  • test_a_partial_root_whose_pin_history_is_not_here_names_the_fetch (:172; blob:none, tree:0, and HEAD moved back by reset --soft):
    • no fetch, and restore == "partial";
    • the why says the pin's history, HEAD's included, is not all in this clone;
    • the NEXT is git -C <root> fetch --refetch --no-filter origin;
    • run as printed, the doctor run again says committed, or uncommitted where HEAD was moved back.
  • test_the_why_spells_out_the_fetch_from_each_other_promisor_remote (:200, round 2).
  • test_every_git_the_run_starts_has_the_variable (:242, round 2).
  • test_lazy_fetch_off_sets_the_variable_and_puts_it_back (:266): with the variable unset, and with it set to 0.

Mutants (each run against tests/test_doctor_no_lazy_fetch.py):

mutant killed by
main not under lazy_fetch_off the partial-root test, both filters (it fetched, and answered committed)
the variable set to 0 the partial-root test, both filters, and the unit test
the environment not put back the every-row test (the doctor crashes on its way out) and the unit test
no partial answer the partial-root test, both filters (none)
a failed probe read as "lacks nothing" the partial-root test, tree:0

Rows on tests/test_shape_doctor.py: that file makes no partial clone (--filter occurs 0 times), so the variable cannot change a git answer there.

  • Base 296a6dd6: 436 passed, 3 skipped, 1 xfailed in 327.40s (0:05:27).
  • This head: 436 passed, 3 skipped, 1 xfailed in 331.04s (0:05:31).

Run with a stub gh first on PATH, GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-w787-<group> -o tmp_path_retention_policy=failed <files>

Results:

  • On 152e9c39, the seven files tests/test_doctor_no_lazy_fetch.py, tests/test_doctor_partial_clone.py, tests/test_update_shape_merged_away.py, tests/test_repo_hygiene.py, tests/test_cli_reference.py, tests/test_doctor_pin_restore_merges.py and tests/test_doctor_next_is_a_command.py, together: 230 passed in 175.17s (0:02:55).
  • On 152e9c39, tests/test_shape_doctor.py: 436 passed, 3 skipped, 1 xfailed in 331.04s (0:05:31).
  • On 19665420 (the move), the same seven files: 230 passed in 167.23s (0:02:47).
  • On the predicted merge of 19665420 into main bf1f099b, the same seven files: 236 passed in 183.89s (0:03:03).
  • On the predicted merge, tests/test_shape_doctor.py: 436 passed, 3 skipped, 1 xfailed in 308.14s (0:05:08).
  • CI on 19665420 (run 39021769167, git 2.55.0, on pull/791/merge b558d917): tests gives 11556 passed, 144 skipped, 1 xfailed in 2295.67s (0:38:15). tests-git-bash, package and SonarCloud pass. SonarCloud's analysis of 19665420: 0 open issues, quality gate OK, 0.0% new duplication.

Merging with main

The first push, 152e9c39, conflicted with main bf1f099b in shape-doctor.py, so GitHub ran no CI.

🤖 Generated with Claude Code

Summary by Sourcery

Ensure shape-doctor never lazily fetches missing objects and identify partial clone pin-history gaps with actionable refetch commands.

New Features:

  • Detect partial clones that lack a pin's history and report the required refetch command, including alternative promisor remotes.

Bug Fixes:

  • Prevent all Git processes launched by the doctor and its validators from lazily fetching missing objects during reads.

Enhancements:

  • Preserve and restore the caller's GIT_NO_LAZY_FETCH environment while enforcing lazy-fetch prevention for the doctor run.

Documentation:

  • Update the doctor description to document that all Git processes it starts run with lazy fetching disabled.

Tests:

  • Add coverage verifying no fetches occur in partial clones, reports remain unchanged when the variable is ignored, missing pin history is diagnosed correctly, all child Git processes inherit the setting, and the environment is restored.

Brett Heap and others added 4 commits October 17, 2026 00:57
…lacks objects at, by id, from the promisor remote whose branches hold it, so it no longer loops where the pinned commit sits only on another remote or another branch, and its `why` says where the clone does not hold the commit at all (#754)

RV-f's review of #742, measured with git 2.43:
- S2: the NEXT was `fetch --refetch --no-filter <the first promisor
  remote>`, the remote whole. With the pinned commit on a second promisor
  remote's branches only, it exited 0 and the row came back the same; in
  a `--single-branch` clone whose pin is on another branch, the clone
  still did not hold the commit afterwards. The NEXT now names each
  lacking commit by id, and `remote_holding` picks the promisor remote
  whose tracking refs hold them all (`for-each-ref --contains`, which
  reads commits alone and fetches nothing). Where none does -- always for
  a commit the clone does not hold -- it asks the first, and the `why`
  names every promisor remote.
- S1: where `objects_lacking` is None (the clone does not hold the
  commit), the `why` says so.
- N1: the reason says "at least", since a missing tree hides what is
  under it from `rev-list`.
- N2: the `why` says `--refetch` needs git 2.36 or later.

tests/test_doctor_partial_clone.py: a `--single-branch` partial clone of
a standard whose `main` lacks C1 (exit 3, `lacking == {C1: None}`, C1
still not held), its NEXT run as printed, two promisor remotes with C1
on the second's branches only, and the NEXT and "at least" of the
existing case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hold as absent on every git, so from git 2.45 on it is no longer counted as one object lacked and handed to `for-each-ref --contains`, which fetches it (#754, round 2)

PR #773's CI, on git 2.55.0, failed three tests at setup with "the
clone holds C1". The clone did not hold it. From git 2.45 on,
`rev-list --missing=print` takes a missing tip as missing too: it prints
`?<commit>` and exits 0 (git's rev-list documentation since v2.45.0, "If
some tips passed to the traversal are missing"), where git 2.43, this
machine's, refuses it, `bad object`, exit 128. The tests' `holds` read
exit 0 as "held".

The doctor read the same command. On git 2.45 and later `objects_lacking`
counted the absent commit as one object lacked, not None, so the row said
"lacks at least 1 object" where the clone does not hold the commit at
all, and `remote_holding` asked `for-each-ref --contains` of a commit the
clone lacks, a read that fetches it. A `?<commit>` line for the commit
itself is now None, as the refusal is.

tests/test_doctor_partial_clone.py: `holds` takes both answers as "not
held"; `elsewhere` puts C1 on its `pinned` branch only once
`single_branch`'s clone exists, so no clone flag of any git can bring it
in, and the precondition stays; `objects_lacking` is held to all four
answers it can get, the two for a missing commit included, through the
module, since one machine has one git.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re on different remotes' branches, the NEXT fetches from the remote holding the pinned commit only the commits that remote holds, and the `why` says which remote holds which, so the fetch no longer fails `not our ref` and comes back unchanged (#754, round 3)

RV-f's review of #773 rounds 1 and 2 (SHOULD S2): a partial clone whose
HEAD lacked objects on `origin`'s branches alone, with the pinned commit
on `second`'s alone, got `fetch --refetch --no-filter origin <pinned>
<HEAD>`. git refuses a fetch by id whole where the remote lacks one id,
so the row came back the same, its `why` saying no branch held them.

- `partial_clone_fetch` picks the remote and the commits: the first
  remote whose branches hold every lacking commit some remote holds,
  else the first holding the pinned commit, and only the commits that
  remote holds. The doctor run again names the rest. Where no remote's
  branches hold any, it is the first promisor remote and every commit,
  as before.
- The `why` says where each commit is (`where_lacked`). Where the
  remote is a guess, it spells out the same fetch from each other
  promisor remote as a line to paste (N1).
- `remotes_holding` (was `remote_holding`) lists every remote whose
  branches hold one commit, and is still never asked of a commit the
  clone does not hold.
- SonarCloud S3776 and S3358 (S1): the reason, the `why` and the fetch
  are built by helpers, with no nested conditional, and the nested
  `def which` is gone (N2).

Tests: `test_commits_on_two_remotes_are_fetched_from_each_in_turn` runs
RV-f's case's NEXT twice to an answer, and
`test_the_why_spells_out_the_fetch_from_each_other_promisor_remote`
runs the `why`'s command for a never-fetched `second`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ETCH=1`, so a read of an object a partial clone lacks fails rather than fetches, and `pin_restore` answers `partial` and names the fetch where a root's own partial clone lacks its pin's history (#787)

"It never fetches" held read by read: #683 and #754 ask a partial clone
of the standard with `rev-list --missing=print` before anything reads
it. A read nothing asked about first still fetched. Measured with git
2.43: a partial clone of a ROOT whose HEAD commit broke its pin made one
lazy fetch (`blob:none`) or two (`tree:0`), when `pin_restore` walked
the pin's history.

- `lazy_fetch_off` sets `GIT_NO_LAZY_FETCH=1` in this process's
  environment for the run of `main`, and puts it back after. Every git
  inherits it: the doctor's own reads, `update-shape.py`'s `Upstream`
  as the doctor drives it in-process, the validators and their gits. No
  printed line carries it, so a NEXT `fetch --refetch` still fetches.
  git 2.45.0 and later honour it, and the 2024-05-14 security releases
  back to 2.39.4 (CVE-2024-32465); an older git ignores it, harmlessly.
- With it, `pin_restore`'s walk read a failed read as "no pin there"
  and said `none`: no commit holds a pin to put back, which is not so.
  `history_left_out` asks, without fetching, whether a partial clone
  lacks the pin's history. Where it does, `restore` is `partial`, and
  the NEXT is `git -C <root> fetch --refetch --no-filter <remote>`.
  Run as printed, the doctor run again names the commit (`committed`).

Tests (`tests/test_doctor_no_lazy_fetch.py`):
- no row starts a fetch from a partial clone of the standard (trace2);
- the report is byte-identical with the variable ignored by every git,
  on a full clone and on two partial clones of the standard;
- a partial root whose pin history is not here answers `partial`;
- `lazy_fetch_off` sets the variable and puts it back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 17, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR makes the doctor’s no-fetch guarantee structural by disabling Git lazy fetching for every process it starts, adds a partial pin-restore result with an actionable fetch when a partial root lacks history, improves partial-clone fetch selection and diagnostics, and adds extensive trace- and scenario-based regression tests.

Sequence diagram for no-lazy-fetch doctor execution

sequenceDiagram
    participant User
    participant Doctor as shape-doctor.py
    participant Git as Git child process
    participant Remote as Promisor remote

    User->>Doctor: main(argv)
    activate Doctor
    Doctor->>Doctor: lazy_fetch_off()
    Doctor->>Git: git read with GIT_NO_LAZY_FETCH=1
    Git-->>Doctor: object data or read failure
    Git--x Remote: no lazy fetch
    Doctor->>Git: start validator with inherited environment
    Git-->>Doctor: validator result
    Doctor-->>User: report and optional NEXT command
    Doctor->>Doctor: restore previous environment
    deactivate Doctor
Loading

Sequence diagram for partial pin-history diagnosis

sequenceDiagram
    participant Doctor as pin_restore
    participant Git as Git rev-list
    participant Root as Partial root
    participant User
    participant Remote as Promisor remote

    Doctor->>Git: rev-list --objects --missing=print -n 51 HEAD -- pin
    Git->>Root: inspect pin history with lazy fetching disabled
    Root-->>Git: missing object marker or read failure
    Git-->>Doctor: history is unavailable
    Doctor->>Doctor: history_left_out(root, pin)
    Doctor-->>User: restore=partial and NEXT fetch command
    User->>Remote: git fetch --refetch --no-filter
    Remote-->>Root: restore missing history
    User->>Doctor: run doctor again
    Doctor->>Git: walk restored pin history
    Git-->>Doctor: identify pin commit to restore
Loading

Flow diagram for partial-clone finding and recovery

flowchart TD
    A[Doctor checks pin restore] --> B[pin_restore walk finds no readable pin]
    B --> C[history_left_out runs rev-list --missing=print]
    C --> D{Missing history or failed read?}
    D -- No --> E[Return restore none]
    D -- Yes --> F[Return restore partial]
    F --> G[Print git fetch --refetch --no-filter]
    G --> H[User runs printed fetch]
    H --> I[User runs doctor again]
    I --> J[Doctor reads complete history and names pin commit]
Loading

File-Level Changes

Change Details Files
Disable Git's lazy fetching for the entire doctor execution and restore the caller's environment afterward.
  • Added a context manager that sets GIT_NO_LAZY_FETCH=1 and restores its prior value.
  • Wrapped main so all doctor, loaded helper, validator, and descendant Git processes inherit the setting.
  • Documented Git-version behavior and clarified that printed NEXT commands still fetch when run by the user.
shape-doctor.py
Detect when pin restoration cannot inspect a partial root's history and report a named recovery fetch instead of misclassifying the pin.
  • Added history_left_out using non-fetching rev-list --missing=print probing.
  • Added the partial restore result, explanatory reason, and fetch --refetch --no-filter NEXT command.
  • Kept the condition as a finding and applied it to rows sharing pin_restore.
shape-doctor.py
Make partial-clone diagnostics robust across Git versions, missing commits, missing trees, and multiple promisor remotes.
  • Treat missing commit tips consistently across pre- and post-2.45 Git behavior.
  • Report missing-object counts as lower bounds when missing trees hide descendants.
  • Select fetch sources based on tracked remote branches and name per-remote follow-up fetches when needed.
  • Refactored partial-clone explanations and generated fetch commands to include the relevant commit IDs.
shape-doctor.py
tests/test_doctor_partial_clone.py
Add regression coverage proving the doctor never initiates lazy fetches while preserving existing reports and recovery behavior.
  • Trace Git child processes for standard and broken-pin partial clones with both blob:none and tree:0.
  • Compare JSON/table output with and without the environment variable.
  • Verify lazy_fetch_off environment restoration and partial-root recovery after running the printed fetch.
  • Cover missing commits, multiple promisor remotes, single-branch clones, and version-dependent rev-list output.
tests/test_doctor_no_lazy_fetch.py
tests/test_doctor_partial_clone.py

Assessment against linked issues

Issue Objective Addressed Explanation
#787 Set GIT_NO_LAZY_FETCH=1 for every Git child the doctor starts for its own reads, including reads performed through update-shape.py, while ensuring printed NEXT commands do not inherit the setting. ✅
#787 Keep row answers usable when a partial clone lacks required objects: avoid tracebacks, report the existing cannot-answer outcome where appropriate, and preserve the relevant full-clone and partial-clone behavior. ✅
#787 Test that asking every row of a partial clone with its remote out of reach causes no fetch. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

…rs, after `remotes_holding`, so #791 merges with main, whose `other_sides_copies` (bf1f099) took the place after `pin_restore` (#787)

A move, and nothing else: every line removed is added back unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-17T10:26:57Z) Independent review of #791 by RV-f (opus), on 152e9c39 + 19665420 over #773's 296a6dd6: READY. No MUST, no SHOULD, three NITs. The head, main and #789 (abcad399) did not move during the review.

Rulings on the risks:

  1. The variable leaks into nothing that is printed or must fetch.

    • The only os.environ uses under main are lazy_fetch_off and run_validator's copy for the validators.
    • Every git subprocess the doctor runs is a read.
    • GIT_NO_LAZY_FETCH does not block an explicit fetch: a NEXT run with it exported exits 0, and the missing objects go from 1 to 0.
    • The variable is restored in a finally.
  2. partial is right on every clone type built. The project remote's HEAD breaks the pin.

    root before (296a6dd6) head
    full committed committed, --json byte-identical
    blob:none 1 fetch, committed 0 fetches, partial
    tree:0 3 fetches, committed 0 fetches, partial
    • After the NEXT (fetch --refetch --no-filter <remote>), both partial roots lack 0 objects and answer committed.
    • A promisor remote named upstream, in a --single-branch clone, is named and works.
    • --refetch --no-filter needs git 2.36 or later; the why says so.
  3. On a git that ignores the variable, nothing changes (simulated): 1 and 3 fetches, committed, byte-identical. history_left_out cannot fetch on any git. It reads config, rev-parse HEAD and rev-list --missing=print, and it is reached only after the walk has found nothing.

  4. No other read of a partial root breaks:

    • a dirty root, a detached HEAD and a merge in progress are identical before and after;
    • HEAD moved by reset --soft goes from 1 fetch + uncommitted to 0 fetches + partial, then uncommitted after the NEXT;
    • no traceback anywhere.
  5. With shape-doctor.py: a shape copy renamed on both sides of a merge, DD at its old name, is named gone on both sides with each new name a side's own commit holds the base's blob of it at, never as a merge of hunks git did not merge (#783) #789 at abcad399: the merge-tree is clean, and the committed merge gives 59 passed.

  6. Docs: README.md:1149 and AGENTS.md:214 ("fetches nothing") are now true by construction, and the docstring is right.

NITs:

  • N1: the partial why (shape-doctor.py around :4059-4068) says "which commit broke the pin cannot be read here". Where HEAD moved without a checkout, the answer after the fetch is uncommitted, not a commit. Say "the pin's history, HEAD's included, is not all in this clone" instead.
  • N2: no test proves "every git it starts". Dropping @lazy_fetch_off() is killed only by the two partial-root cases. Add an assert that a validator child, or Upstream, sees GIT_NO_LAZY_FETCH=1.
  • N3: the NEXT refetches the first promisor remote's branches. Where the missing history is reachable only from another remote, or from a branch force-pushed away, the row comes back. Rare; the why could name every promisor remote.

Runs:

what result
test_doctor_no_lazy_fetch.py + test_doctor_pin_restore_merges.py + test_doctor_partial_clone.py 60 passed
test_shape_doctor.py (once) 436 passed, 3 skipped, 1 xfailed
predicted merge with #789 59 passed
mutants: no decorator; history_left_out always None each fails 2

CI on git 2.55 and SonarCloud are green.

Lane: openreposhape-3 (openRepoShape-3)

…ncluded, is not all in the clone, and spells out the fetch from each other promisor remote; a test records that every git a doctor run starts, its validators' and `Upstream`'s included, has `GIT_NO_LAZY_FETCH=1` (#787, RV-f's NITs on #791)

- N1: "which commit broke the pin cannot be read here" was not so where
  HEAD was moved back without a checkout. There the doctor run after the
  fetch answers `uncommitted`. The `why` now says "the pin's history,
  HEAD's included, is not all in this clone"
  (`partial_history_why`), and a `reset --soft` case is tested.
- N2: `test_every_git_the_run_starts_has_the_variable` puts a recording
  `git` first on `PATH`. Every git the run starts has the variable:
  `validate-pins.py`'s, started by `run_validator`, and `Upstream.blob`'s
  `cat-file blob` as the doctor drives it in-process.
- N3: `history_left_out` returns every promisor remote. The `why` spells
  out the same fetch from each one after the first, where the history
  is on another remote's branches alone. Tested with `origin` naming
  another repository: the NEXT leaves the row the same, and the `why`'s
  `second` line ends it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-17T12:00:24Z) Re-verification of #791 round 2 by RV-f (opus), on head 1d774ab3 (a fast-forward of 19665420): READY. No MUST, no SHOULD; N1, N2 and N3 are fixed.

1. The shim test cannot pass vacuously (tests/test_doctor_no_lazy_fetch.py:242, shim at :222).

  • run_script changes only PATH, run_validator copies that environment, and offline() leaves PATH alone, so every child sees the shim.
  • The test needs at least one git under validate-pins.py and one cat-file blob …:templates/… from the doctor's own process.
  • Two mutants each fail it: dropping @lazy_fetch_off(), and popping the variable in run_validator.

2. Each printed fetch line pastes and works. RV-f built a blob:none root with three promisor remotes; only third holds the history.

step result
doctor partial, 0 fetches
NEXT (origin) still partial
pasted second line still partial
pasted third line missing 0, committed

3. Nothing else changed since 19665420: the pin_restore docstring, the partial return, refetch_of, partial_history_why, history_left_out (now a list), and the test file.

4. Merges:

merged with result tests on the merge
main 20004887 clean the three doctor files: 69 passed
#789 abcad399 clean 62 passed

CI: run 39026676188's one error, test_family_remove_gitmodules.py::…[deleted], is the #645 copy-during-repack flake. That file never mentions the doctor, and this PR changes only shape-doctor.py and its own test file.

NIT: git 2.43's fetch --refetch hangs forever against an EMPTY remote (plain git reproduces it; --no-filter alone returns). This is git's bug, and an empty promisor remote is unrealistic. A clause in the why is optional.

Runs:

  • the three doctor files: 63 passed;
  • test_shape_doctor.py -k "pin or partial" (once): 88 passed, 1 skipped;
  • both mutants: fail the shim test.

Lane: openreposhape-3 (openRepoShape-3)

brettheap and others added 3 commits October 18, 2026 00:36
…ep 1 of 3 after #773 squash-landed as 80aee84)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… of 3; the squash's tree equals the final head's, so #791's own lines are kept)

shape-doctor.py conflicted and was taken ours: F (18ee4c2) and S (80aee84) have the same tree, so this merge changes no file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-18T04:57:55Z) #791 is restacked after #773's squash (80aee84), as 403b6996, by lane 3's three-merge recipe (writer R791, opus-xhigh):

  1. ce0881a8 merges shape-doctor.py: the partial-clone row's NEXT fetches each commit it lacks objects at, by id, from the promisor remote whose branches hold it, so it no longer loops where the pinned commit sits only on another remote or another branch, and its why says where the clone does not hold the commit at all (#754) #773's final head 18ee4c25. It is clean.
  2. 403b6996 merges the squash 80aee84e, taking ours only in shape-doctor.py: the partial-clone row's NEXT fetches each commit it lacks objects at, by id, from the promisor remote whose branches hold it, so it no longer loops where the pinned commit sits only on another remote or another branch, and its why says where the clone does not hold the commit at all (#754) #773's file shape-doctor.py, the one that conflicted. The squash and the final head have the same tree, c2e764b6, and this merge changed none of shape-doctor.py: the partial-clone row's NEXT fetches each commit it lacks objects at, by id, from the promisor remote whose branches hold it, so it no longer loops where the pinned commit sits only on another remote or another branch, and its why says where the clone does not hold the commit at all (#754) #773's files.
  3. Main was 80aee84e itself, so the third merge was "Already up to date".

Lane: openreposhape-3 (openRepoShape-3)

@sonarqubecloud

Copy link
Copy Markdown

@brettheap
brettheap marked this pull request as ready for review October 18, 2026 08:49
@sourcery-ai

sourcery-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 2 days and 11 hours by commenting @sourcery-ai review. Upgrade to get a review now.

brettheap pushed a commit that referenced this pull request Oct 18, 2026
…and its round 2, and run to main 33c5128 (#883)

Round 2b of #883. It answers RV883's full NIT list on ede678c (lane 3,
2026-10-18T09:29:28Z) and RV883's round 2 on 86baec5 (#883
issuecomment-6248362246). Each finding was checked against the code at
main 33c5128 before the text changed:

- The #843 limit is two: what `adopt-work-dir-shared` leaves open (group
  write, Windows, root as the other user, this user's own processes, the
  default work directory), and what it now refuses that v1.1.0 used
  (another user's directory, a run as root included, a 0777 directory
  without the sticky bit, a drive mounted without ownership), with the
  way out. "What a script sees" names `adopt-work-dir-shared` and
  `adopt-work-dir-occupied`.
- The doctor's NEXT lines, not its `members` row, read no url from an
  unmerged `.gitmodules`; the `make siblings` limit names that row.
  `siblings.py` has no unmerged rule; the `validate.yml` templates do.
- `validate-family.py`'s `family ok` is scoped to `--no-members` and to a
  holder whose members are checked out.
- The wording points: `update-pr-no-repository` under `--pr`, family.py's
  `git rm -r --cached` line, `MSYSTEM` set and not empty, `..` and
  `naming-unclassified`, the keys git refuses with no value, the url the
  root's config keeps, `contracts: cannot be listed`, install.sh's newest
  tag and redirect location, the scan test's name,
  `update-rewrite-symlink`, `os.access` for root, `LC_ALL=C`,
  `update-validators-red`, `heads/main` in both keys, where
  `plan written to` came, junctions on Python 3.12, the approval click and
  the run's approvals, and the back-port route's one run.
- #457's L1-17 is narrowed by #825; #853 and #854 join #834 as test
  limits.

The refresh past c5359df adds #870 (#864), #876 (#839), #791 (#787),
#859 (#838), #780 (#776), #873 (#857) and #875 (#874): 57 pull requests.
It drops the limits for #776, #839 and #874's ref rule, rewrites the #838
clauses, and adds the limits those squashes name (#863, #879, #884, an
older git without `GIT_NO_LAZY_FETCH`, family.py's link guard). The open
list is the 14 that the 09:42Z milestone read leaves after #875: #675,
#707, #739, #755, #764, #834, #840, #853, #854, #856, #863, #878, #879
and #884.

The six-and-six shape-copy lists stand: from c5359df to 33c5128 the
templates and repo_shape.py changed only in the family root's
bootstrap.py and validate-family.py, both listed. No other file changes.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant