Skip to content

docs: ignore the Version line when checking generated llms files - #449

Merged
loonghao merged 1 commit into
mainfrom
fix/llms-check-ignore-version-line
Oct 13, 2026
Merged

loonghao merged 1 commit into
mainfrom
fix/llms-check-ignore-version-line

Conversation

@loonghao

Copy link
Copy Markdown
Owner

Summary

Two things: fix a build-breaking interaction introduced by #445, and correct
three comment counts that were measured wrong.

The bug. #445 wired scripts/generate_llms.py --check into the Python package workflow. The Version: line in the generated files is the newest git
tag, so it changes as soon as a release lands while the committed file still
carries the previous release. Every release therefore turned main and all
pull requests red until the bot's docs/update-llms-txt pull request was
merged by hand. Reproduced by adding a local v0.25.1 tag, which flipped the
check from exit 0 to exit 1.

Comparison now drops the Version: line on both sides. Everything else —
README, examples, module index — still fails, which is the gate's actual job.
Version lag keeps converging the way it already did, through the llms-txt
job's pull request; this only stops it from being a build failure.

Behaviour, verified

Change Before After
Only Version: differs (new tag) exit 1 exit 0
Hand-edited Version: line exit 1 exit 0
Any other line differs exit 1 exit 1

The first row was verified with a real local v0.25.1 tag (since removed), not
by reasoning about the code. The last row was verified in CI: PR #448 added content drift
to llms.txt on a scratch branch, and run 38477580289
shows ruff check / ruff format passing ("All checks passed!") followed by
##[error]llms.txt are out of date; run scripts/generate_llms.py → exit 1 →
build red. That scratch PR and its branch were closed and deleted.

The bot pull-request mechanism is unchanged: the generator still writes the new
version and the llms-txt job still opens a pull request for it. Only the gate
stopped caring.

Documented in the module docstring and at the workflow step, so the next reader
does not mistake a passing check with a mismatched version for a bug.

Comment counts corrected

Measured against this tree, all three with ruff 0.16.9:

  • 864 is the "no convention" count, not pep257. Measured: 864 with no
    convention set, 804 under convention = "pep257", 665 under the committed
    convention = "google". The comment said pep257.
  • The baseline is 507 / 158 / 0, not "507 inside the CI lint scope":
    507 in photoshop/, 158 in test/, 0 in scripts/. The lint scope was
    photoshop alone when the baseline was written, so 507 was the whole CI
    scope at the time; it is photoshop test scripts now, which is all 665.
  • Per-tier numbers are upper bounds, not counts. The tiers overlap, so the
    eight labels add up to 883 against 665 actual findings. The test tiers
    alone are labelled 118 + 45 = 163 against 158 measured. Stated explicitly,
    because otherwise the numbers will not balance when clearing a tier.

Ruff 0.17 does not break the baseline

PR #440 (renovate,
ruff ^0.17.0) is open. Measured by installing ruff 0.17.0 and running it
against this tree:

  • ruff check photoshop test scripts under the project config: passes,
    exit 0. The baseline holds.
  • D-rule set: 72 rules in both 0.16.9 and 0.17.0, none added, none removed.
  • D findings without the baseline: 665 in both.
  • ruff format --check photoshop test scripts: 81 files already formatted in
    both.

One difference exists but does not fire here. 0.17 adds UP042
(replace-str-enum), which reports 2 hits (FontSize in
photoshop/api/enumerations.py:391, EventID in photoshop/api/event_id.py:5)
— under --target-version py311. This project sets target-version = "py38",
because the package still supports Python 3.8, and the suggested fix needs
enum.StrEnum (3.11+). Under py38, UP042 reports nothing, which is why the
check passes.

So no upper bound is needed to merge #440. If the project later raises
target-version to py311+, UP042 would start reporting and CI would go red —
at which point the two classes should be converted or UP042 added to the
ignore list, as a deliberate decision rather than a surprise.

Note that three places pin ruff independently: pyproject.toml (^0.16.9),
.github/workflows/pythonpackage.yml (pip install ruff==0.16.9), and
.pre-commit-config.yaml (rev: v0.16.9). #440 only bumps the first two. The
pre-commit pin will drift to whatever renovate bumps it to separately; all
three agree today, so this is a heads-up rather than a defect.

Validation

  • ruff check photoshop test scripts: passes.
  • ruff format --check photoshop test scripts: 81 files already formatted.
  • python scripts/generate_llms.py --check: up to date; exits 1 on content
    drift (verified in CI, above), exits 0 on version-only drift.
  • pytest test/test_collections.py test/test_imports.py: 18 passed.
  • pyproject.toml still parses as valid TOML with all 8 baseline entries.

Configuration, comments, and CI only. No packaging or lockfile changes, and the
generated llms.txt / llms-full.txt are untouched — the version line still
converges through the bot pull request.

Tests under test/manual_test/ drive a real Photoshop COM instance and cannot
run in this environment; they were not executed.

The --check gate added in #445 compares the committed llms files against
freshly generated ones. The stamped version comes from the newest git tag,
so it changes the moment a release lands while the committed file still
carries the previous release. A single release therefore turned main and
every pull request red until the bot's docs pull request was merged by hand:
reproduced by adding a local v0.25.1 tag, which flipped --check from 0 to 1.

Comparison now drops the `Version:` line on both sides. Anything else —
README, examples, module index — still fails, which is what the gate is
for. Version lag keeps converging the way it already did, through the
llms-txt job's pull request; this only stops it from being a build failure.
Both the generator docstring and the workflow step say so, so the next
reader does not mistake a passing check for a bug.

Also correct three comment counts measured against this tree:

- 864 is the "no convention" count, not the pep257 count. Measured: 864
  with no convention, 804 under pep257, 665 under the committed google one.
- The baseline is 507 in photoshop/, 158 in test/, 0 in scripts/. The
  lint scope was `photoshop` alone when it was written, so 507 was the
  whole CI scope then; it is `photoshop test scripts` now.
- Per-tier numbers are upper bounds, not counts: overlapping tiers make
  the eight labels add to 883 against 665 actual findings.
@loonghao
loonghao merged commit 3c0b4d1 into main Oct 13, 2026
10 checks passed
@loonghao
loonghao deleted the fix/llms-check-ignore-version-line branch October 13, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant