Skip to content

docs: correct docstring baseline comments and put test and scripts under CI lint - #445

Merged
loonghao merged 1 commit into
mainfrom
docs/llms-and-baseline-followup
Oct 13, 2026
Merged

loonghao merged 1 commit into
mainfrom
docs/llms-and-baseline-followup

Conversation

@loonghao

Copy link
Copy Markdown
Owner

Summary

Follow-up to #443, addressing three of the four P2 findings from its review:

  1. The docstring baseline comments described behaviour that does not hold.
    per-file-ignores is a union of every matching pattern, not a
    most-specific-match lookup. Two claims in the previous PR were wrong and are
    now corrected.
  2. Two baseline tiers were unreachable from CI. The test/** and
    test/manual_test/** tiers (163 findings) could never fire, because the lint
    job only ran on photoshop. The lint scope is now photoshop test scripts.
  3. scripts/generate_llms.py sat outside every gate. It is now linted and
    format-checked, and its --check mode runs in CI so drifted llms.txt files
    fail the build.

Also corrects two stale references: the generator's module docstring pointed at
a workflow file that does not exist, and a comment claimed it stamps the version
about to be released.

Configuration, comments, and CI wiring only. No packaging or lockfile changes,
and the generated llms.txt / llms-full.txt are untouched.

1. Corrected baseline semantics

Both of these were verified against the current tree, not inferred.

per-file-ignores unions every matching pattern. The old comment said "any
NEW docstring debt is caught immediately". It is not: a rule listed for a file is
ignored for that file, so new violations of that same rule are also ignored.
Adding an undocumented public class to photoshop/api/solid_color.py (D101 is
baselined for that path) leaves ruff check green; adding an undocumented
public function to the same file (D103 is not baselined) fails. The baseline
stops the tree going red — it does not stop new debt in already-baselined rules.

"Delete a rule from a tier to tighten it" only works on the widest tier. The
old comment's ratchet was inoperative for four of the eight tiers, because every
file under photoshop/ also matches the photoshop/** entry. Removing D104
from photoshop/api/save_options/** changes nothing; removing it from
photoshop/** does. The comments now say to tighten by editing the widest tier
covering the files you fixed.

The count was wrong in a way that reversed the explanation. 864 is the count
under ruff's default pep257 convention. Under the committed google
convention it is 665 tree-wide and 507 inside the CI lint scope. The
previous description credited the google convention with producing the 864; it
in fact reduces 864 to 665. Relatedly, the google convention is what resolves
the D203/D211 and D212/D213 pairs, so the committed config emits no warnings —
the comment claiming startup warnings are expected was also wrong.

These are comment-only corrections. No rule, tier, or finding count changes;
CI behaviour is unchanged by this item.

2. Lint scope extended to test/ and scripts/

ruff check photoshop test scripts and ruff format --check photoshop test scripts now run in the Python package workflow.

The scope stays an explicit list rather than . so the docstring baseline covers
exactly the linted paths and a generated or vendored tree cannot silently widen
what CI checks.

scripts/ needed no baseline tier — it reports zero docstring findings, so it
gets full enforcement, which is what new code should get.

Six files under test/ were reformatted because ruff format reports them. This
is not new policy: the repo's own pre-commit ruff-format hook already applies
to the whole tree with no path restriction — CI just never enforced it, so they
had drifted. The changes are cosmetic (joining wrapped lines, removing a leading
blank line, one comment space). Verified AST-identical before and after for all
six, so no behaviour changes.

3. Generated-file drift now fails the build

python scripts/generate_llms.py --check runs in the same job. Previously a
stale llms.txt would only ever produce a bot pull request from the Release
workflow; it never turned a build red. Verified that --check exits non-zero and
emits a ::error:: annotation when the committed file is drifted, and exits zero
when it is current.

4. Stale references corrected

  • The module docstring pointed at .github/workflows/llms-txt.yml, which does
    not exist. The job lives in .github/workflows/release.yml.
  • The workflow comment claimed the job "stamps the version about to be
    released". It does not: version() reads the newest tag, and the job runs
    before release-please creates one, so the stamped version is the last
    released one. Each release is followed by one docs commit carrying the new
    version, which is harmless because preflight treats docs: as
    release-neutral. Both the comment and the version() docstring now describe
    the actual behaviour.

Not done

Switching the baseline to per-line or per-file # noqa is deliberately out of
scope. It is a rewrite at the 500+-finding scale that would touch source files
across the package and supersede the per-directory design; it should be
evaluated on its own rather than folded into a follow-up.

Validation

  • ruff check photoshop test scripts: passes.
  • ruff format --check photoshop test scripts: 81 files already formatted.
  • ruff check --target-version py38 scripts/: passes (the script uses no
    3.9+ syntax, so it is safe on every matrix version).
  • python scripts/generate_llms.py --check: up to date; verified it fails on a
    deliberately drifted file.
  • pytest test/test_collections.py test/test_imports.py: 18 passed.
  • pytest --collect-only: 169 tests collected.
  • Release workflow still orders llms-txt before release-please
    (needs: [preflight, llms-txt] with if: always()), so this does not change
    release behaviour.

Tests under test/manual_test/ drive a real Photoshop COM instance and cannot
run in this environment; they were not executed. The six reformatted files in
that directory were verified AST-identical rather than run.

…der CI lint

The docstring baseline comments claimed two things that are not true, both
corrected against measured behaviour:

- per-file-ignores is a union of every matching pattern, not a most-specific
  match. A rule listed for a file is ignored for that file, so new violations
  of an already-baselined rule are also ignored. Removing a rule from a narrow
  tier is a no-op while a wider tier still lists it, which made the ratchet
  inoperative for four of the eight tiers.
- The baseline count is 665 findings under the committed google convention
  (507 inside the CI lint scope), not 864; 864 is the default pep257 count.
  The google convention is also what suppresses the D203/D211 and D212/D213
  warnings, so the config emits none.

Extend the lint scope from `photoshop` to `photoshop test scripts`. The
baseline declares tiers for test/ that no CI job could reach, so they were
dead configuration; scripts/ was outside every gate. Reformat the six test
files that ruff format reports, which is what the repo's own pre-commit hook
already applies. All six are AST-identical before and after.

Wire `generate_llms.py --check` into the same job so drifted llms.txt files
fail the build instead of only producing a bot pull request.

Also correct the generator docstring, which pointed at a workflow file that
does not exist, and the version comment: the stamped version is the last
released one, because the job runs before release-please creates the tag.
@loonghao
loonghao merged commit 04b80f3 into main Oct 13, 2026
10 checks passed
@loonghao
loonghao deleted the docs/llms-and-baseline-followup branch October 13, 2026 22:12
loonghao added a commit that referenced this pull request Oct 13, 2026
Follow-up to #445, fixing a build failure the new gate introduced and
correcting three comment counts that were measured wrong.

The `--check` gate added in #445 compares the committed llms files against
freshly generated ones. The stamped version comes from the newest git tag, so
it changes the moment a release lands while the committed file still carries
the previous release. One release therefore turned main and every pull request
red until the bot's docs pull request was merged by hand: reproduced by adding
a local v0.25.1 tag, which flipped the check from exit 0 to exit 1.

Comparison now drops the `Version:` line on both sides. Everything else —
README, examples, module index — still fails, which is the gate's job. Version
lag keeps converging the way it already did, through the llms-txt job's pull
request. Both the generator docstring and the workflow step say so.

Three comment counts corrected against this tree: 864 is the "no convention"
count rather than the pep257 one (pep257 is 804, the committed google
convention gives 665); the baseline splits 507 / 158 / 0 across photoshop/,
test/ and scripts/, and 507 was the whole CI scope only because the scope was
`photoshop` alone when it was written; and the per-tier labels are upper
bounds, not counts, since overlapping tiers make the eight labels add to 883
against 665 actual findings.

Configuration, comments and CI wiring only. No packaging or lockfile changes,
and the generated llms.txt / llms-full.txt are untouched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant