feat(xtask): guard docs/architecture.md path citations in CI - #432
jussielo-amd wants to merge 11 commits into
Conversation
79e4f26 to
aa7b8aa
Compare
Found by an independent code review of ROCm#432 and verified with targeted repros against the real doc before fixing: - is_hyphenated_bare_word() couldn't tell a crate name from ordinary hyphenated prose (`read-only`); a bare hyphenated word is now only treated as a citation when it's declared in a heading or precedes 's. - A bare scoped citation (agent.rs) repeated later under the same heading without a connector lost its possessive narrowing and fell back to every crate the heading lists; a per-heading map now reuses the earlier narrowing for unconnected repeats. - is_directory_shaped() only excluded BARE_FILE_EXTENSIONS from the directory check, so a full path with any other extension (e.g. ci.yml) was misread as a directory, making anything scoped to it unmatchable; the exclusion is now general (any dotted last segment). - tracked_files() didn't disable core.quotePath, so a non-ASCII tracked filename would come back git-escaped and never match a plain citation. Also collapsed a duplicated SCOPED_BARE_EXTENSIONS check in citation_exists into one computation. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
An independent code review surfaced 4 correctness bugs in the checker itself (each verified with a targeted repro against the real doc before fixing, in f27c73e):
Also collapsed a duplicated |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Citation extraction has unresolved Unicode, heading-state, and indented-code-block correctness issues.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 3
Open (3)
Resolved since last review (1)
Found by review of ROCm#432 and verified with repros before fixing: - is_path_safe rejected every non-ASCII character, so a citation like café.rs was discarded at extraction time — before tracked_files' core.quotePath=false handling (added last round) ever ran, leaving that fix unreachable. Widened to accept any Unicode alphanumeric. - Only fenced code blocks were skipped; a 4+ space indented CommonMark code block was still parsed for citations, so a removed-path example shown as indented code could false-fail CI. Indented blocks are now skipped, including the "can't interrupt a paragraph" rule so a line indented only by mid-paragraph wrapping still gets parsed. - narrowed_owners was cleared after a heading's own citations were processed instead of before, so a new heading bare-citing a filename a prior section had narrowed inherited that stale owner — and, via the BTreeSet's dedup, silently merged into the old entry, leaving a missing file under the new heading undetected. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
Follow-up review found 3 more real bugs (verified with repros before fixing, in 5d62cef):
All 3 threads resolved. 213 workspace tests pass (63 in this module, up from 59), clippy/fmt clean. |
|
🔴 Automated review · pr-review-watcher · 5d62cef This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer. SummaryAdds 🚫 Blocking (must fix before merge)
Non-blocking
|
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 5d62cef
Blocking: 1. This is the first round of review from this account on this pull request.
xtask/src/architecture_doc.rs:732 — the gate's own failure branch has no test, so a future change can disarm the gate with CI still green.
Replacing the filter-and-bail with a no-op leaves the entire suite passing. The only test that drives the entry point end to end asserts the success direction, and it would also pass vacuously against an empty citation set, so nothing in the suite would notice if the check stopped being able to fail. That is the exact outcome this gate exists to prevent, and a permanent build gate is the worst place to leave it, because nobody reopens a landed change to strengthen one.
To be clear about what is NOT wrong here: the checker itself is sound and unusually well tested. Planting a bad citation four different ways — a path that does not exist, and three real files removed from tracking — was caught each time, named precisely, and exited non-zero, while the clean tree passed. Of 38 single-branch mutations across the extraction, fence and code-span parsing, scoping and existence-matching logic, 36 were killed by the suite. The extractor checks 53 citations, with the single skipped path deliberate and documented. The all-green checks are not contradicted: this is a missing regression test, not a live defect.
The fix is about ten lines: lift the filter-and-bail composition out of the entry point into a pure function over the markdown and the tracked-path list, then assert it returns an error naming a path absent from that list, alongside the existing success case.
Five non-blocking items, including a prose-citation trap that fails the build on ordinary wording containing a slash, are in the review comment posted alongside this one.
Superseded: verified @siloteemu's automated review is correct that run()'s failure branch (bail!() on stale citations) is untested, which is a real blocking gap. Replacing this approval with a request-changes review.
Found by review of ROCm#432 and verified with repros before fixing: - is_path_safe rejected every non-ASCII character, so a citation like café.rs was discarded at extraction time — before tracked_files' core.quotePath=false handling (added last round) ever ran, leaving that fix unreachable. Widened to accept any Unicode alphanumeric. - Only fenced code blocks were skipped; a 4+ space indented CommonMark code block was still parsed for citations, so a removed-path example shown as indented code could false-fail CI. Indented blocks are now skipped, including the "can't interrupt a paragraph" rule so a line indented only by mid-paragraph wrapping still gets parsed. - narrowed_owners was cleared after a heading's own citations were processed instead of before, so a new heading bare-citing a filename a prior section had narrowed inherited that stale owner — and, via the BTreeSet's dedup, silently merged into the old entry, leaving a missing file under the new heading undetected. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
run() inlined the stale-citation filter and bail!(), so the only end-to-end test exercised just the Ok path — a refactor could disarm the check entirely with the suite still green. Split the composition into a pure check_citations(markdown, tracked) and added a test asserting it fails and names an absent path. Also dedupe stale_message so a file cited both bare and under a section heading is reported once instead of twice, and note that possessive_owner_for's is_scoped_extension guard is diagnostic-only. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
run()'s own one-line delegation to check_citations was still unproven: discarding its result would leave the suite green. Split out check_doc_at(root) and added a test that drives the full read-doc -> tracked-files -> check pipeline over a throwaway git repo, closing the gap without touching the real doc. Also fix a real bug the dedupe surfaced: format_stale_citation hinted a directory for .md/.toml citations even though citation_exists never scopes those by section, and reworded the possessive_owner_for comment to stop implying that guard alone keeps every stale-message hint honest. Document the new check in CONTRIBUTING.md and docs/architecture.md's own caution note, and bump its CI job's timeout to match its siblings. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The combined `engines/lemonade`, `engines/vllm` heading held the prose describing engines/vllm's split (added by ROCm#436) to a possessive/slash-chain the arch-doc checker's connector heuristic can narrow — otherwise a bare `.rs` citation under a two-crate heading is checked against both crates, and engines/lemonade doesn't have these files. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
1e84003 to
feb99ed
Compare
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · feb99ed
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Summary
This PR adds a CI gate (cargo xtask check-architecture-doc, plus its own always-run workflow job) that extracts backtick-quoted path citations from docs/architecture.md and fails naming any that no longer exist in the tracked tree — needs work: the gate also fails on citations whose paths do exist, and all three pieces of new user-facing text say otherwise. Verified: ran the new checker's own test module at this head (71 passed), then thirteen single-line mutations in a throwaway copy — twelve were each killed by a specifically named test (the fence scanner, the heading/indent rules, the code-span splitter, the root-level and scoped-extension branches, the multi-directory rule, the dedupe, and the hint condition are all genuinely covered, so the suite is strong), and the one survivor is run's single delegation expression, which can still be replaced with let _ = …; Ok(()) leaving every test green — the prior round's disarm finding is unchanged at this head. I also confirmed by reverting docs/architecture.md:58 to its base wording that the gate then reports four files as no longer existing when all four are present in the tree, which is what the blocking item below rests on; and I re-read the extractor's termination and alternation invariants by hand. The full test suite, the lint pass and the end-to-end suites were not run here. Working from 28 successful and 1 pending check at this head, with no failures. Blocking: 1 · Non-blocking: 5.
🚫 Blocking (must fix before merge)
xtask/src/architecture_doc.rs:771,CONTRIBUTING.md:74,docs/architecture.md:11— the gate fails on citations whose paths exist, but its failure message and both new contributor-facing sentences state the opposite. A bare.rscitation under a heading that names two directories must exist in every one of them (a deliberate rule, locked bymulti_directory_citation_requires_every_directory_to_have_the_file), so accurate prose under a two-crate heading fails the gate. This is not hypothetical: I reverteddocs/architecture.md:58to the wording at the base tip — factually correct prose — and the checker reporteddocs/architecture.md cites 4 path(s) that no longer exist in the treefor four files that are all present underengines/vllm/src/. The last commit in this PR is the author working around exactly that, rewriting true prose into a possessive slash-chain so the narrowing heuristic fires; the commit message explains the checker's requirement but nothing user-facing does. So a contributor who writes correct prose gets a merge-blocking failure asserting their files are gone, with no documented way to know why. The fix is small and entirely in text: makestale_messagesay the cited path was not found where the citation is scoped to rather than that it "no longer exist[s] in the tree" (the per-citation(expected under …)hint already carries the scope, so the summary line just needs to stop contradicting it), and change bothCONTRIBUTING.md:74anddocs/architecture.md:11to say the gate checks citations in the shapes it recognises, against the directories its nearest heading names — not "a backtick-quoted path … no longer exists in the tracked tree". Docs that overstate a guard are the defect shape this very PR exists to prevent, and it is in the direction that makes the change look safer than it is.
Non-blocking
xtask/src/architecture_doc.rs:722-730—run's wiring of the workspace root intocheck_doc_atis still unproven: discarding its result leaves all 71 tests green, so the CI gate can be silently disarmed. This is the same irreducible single line the prior round flagged and is fairly a stopping point, but a clause onrun's doc comment naming what stays unproven would stop the next reader re-deriving it.xtask/src/architecture_doc.rs:61— the comment asserts "the doc doesn't cite any other extension bare today", which is false now:docs/architecture.md:54cites`report.json`bare, so it is silently never checked. Reword to name the exception rather than deny it exists.xtask/src/architecture_doc.rs:595,:687,:813—is_scoped_extensionis still computed identically in three places; the bug fixed in this PR was precisely two of them drifting apart, and a shared predicate would prevent the next divergence.xtask/src/architecture_doc.rs:520—prev_line_blankis never updated for a fence delimiter or for lines inside a fence (both pathscontinuefirst), so it stays stale across a whole fenced block; an indented citation on the line right after a closing fence that followed a blank line is dropped rather than extracted. No such construct exists in the doc today, but it undercuts the "matches real markdown rules" claim.xtask/src/architecture_doc.rs:942,:1836— therun_gittest closure is still duplicated verbatim between the two temp-repo tests; a shared helper would be better than a second copy.
…heck The failure message, CONTRIBUTING.md, and docs/architecture.md all claimed the gate fails only on a citation that "no longer exists in the tracked tree" -- false for a scoped .rs citation, where accurate prose under a multi-directory heading can still fail the gate. Reword all three to describe the actual scoped-existence check. Also: extract the duplicated is_scoped_extension check into one function, fix prev_line_blank going stale across a fenced block (it was never updated on the fence-delimiter/inside-fence continue paths), dedupe the run_git test helper, correct a stale comment about bare extensions, and note the one remaining unproven line in run(). Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
🔴 Automated review · pr-review-watcher · 15cf21a This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer. SummaryThis PR adds a CI gate ( 🚫 Blocking (must fix before merge)
Non-blocking
|
Superseded by a new round at 15cf21a. Part of this objection is discharged; the remainder, plus a new inaccuracy introduced by the rewording, is restated in a fresh change request so there is only one live objection to answer.
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 15cf21a
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
The previous round's objection is only partly discharged, and the wording that replaced it introduces a new inaccuracy. Full detail is in the round comment on this PR; the blocking part is:
Two of the five surfaces still carry the original overclaim. xtask/src/main.rs:104 still reads "Fail if any path cited (in backticks) in docs/architecture.md no longer exists in the tracked tree" — the exact sentence flagged last round, untouched, and the most user-facing of the three since it is what the command's --help prints. .github/workflows/ci.yml:1195 still says the job "verifies every backtick-quoted path citation in docs/architecture.md still exists in the tracked tree". Both overstate on two counts: most backtick spans are never candidates, and existence is not what is checked.
The new wording is false in the permissive direction. CONTRIBUTING.md:74 and docs/architecture.md:11 now end with "or anywhere in the tracked tree otherwise". For a bare .md/.toml citation, citation_exists requires p == Path::new(text) — the repository root only. Confirmed by construction: adding a `testing.md` citation makes the gate fail even though docs/testing.md is tracked and the prose is accurate, and the failure prints no (expected under …) hint, so the contributor is told the path was "not found where they're cited" with nothing saying where that is.
Suggested fix. Use the module doc comment's wording — which is correct — at xtask/src/main.rs:104 and in the comment at .github/workflows/ci.yml:1195. In CONTRIBUTING.md:74 and docs/architecture.md:11, replace "anywhere in the tracked tree otherwise" with something true: a bare .md/.toml citation is checked against the repository root, and only a bare directory name or an unscoped citation is checked tree-wide.
Separately, and worth folding into the same change: format_stale_citation at xtask/src/architecture_doc.rs:852 prints a bare comma list for the multi-directory case, so "(expected under a, b)" reads as a disjunction while the rule is a conjunction. A reader checks one of the two named directories, finds the file there, and concludes the gate is broken.
The parser fix itself is sound and its regression test genuinely pins it: reverting the fence/blank-line handling to its pre-fix form makes that test the sole failure. The extracted is_scoped_extension helper is well covered — ten named tests kill a mutation of it — so the drift that caused the earlier bug cannot recur silently. This PR has no twin workflow or aggregator job to keep in step; that was checked directly.
…ng its scope
The CLI help text, the CI job comment, CONTRIBUTING.md, and
docs/architecture.md each described check-architecture-doc's citation
matching differently from how citation_exists actually behaves: the
first two claimed a universal "still exists in the tracked tree" rule,
while the latter two claimed unscoped citations are checked tree-wide
even though a bare .md/.toml citation is root-only. Reworded all four
to the accurate scoped/root/tree-wide split the module doc comment
already uses.
Also fixed format_stale_citation's multi-directory hint, which printed
a bare comma list ("expected under `a`, `b`") that reads as either-or
even though citation_exists requires the file under every listed
directory — now says "expected under all of ..." for that case, with a
regression test. Reworded run()'s doc comment to say precisely what's
untested (the Err direction of its one delegation line) instead of the
now-inaccurate "no test drives run end-to-end".
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
Confirmed and fixed in c481e82:
Left as-is: the suggestion to add a
|
Superseded: this objection was filed against an earlier commit and is replaced by a fresh round published at the current head. Only the newest change request is operative, so this one is withdrawn to leave a single live objection.
|
🔴 Automated review · pr-review-watcher · c481e82 This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer. SummaryAdds 🚫 Blocking (must fix before merge)
All four of This blocks because it is the defect class the whole change exists to prevent, and the accurate sentence already lives two files away. Concrete fix. Copy the three-clause sentence already in Non-blocking
|
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · c481e82
Change request filed by the automated review round at this commit. The full round, including non-blocking notes, is in the comment posted alongside it.
🚫 Blocking (must fix before merge)
xtask/src/main.rs:105-107 (and its three twins) — this one is our fault, and it is still a defect. Last round we wrote: "Use the module doc comment's wording — which is correct — at xtask/src/main.rs:104 and in the comment at .github/workflows/ci.yml:1195." That was wrong. The module doc comment is not correct: it carries the same permissive overclaim we blocked on in the two prose files. The author did exactly as asked, so the falsehood now sits on four surfaces instead of two.
All four of xtask/src/architecture_doc.rs:13-14 (module doc), xtask/src/architecture_doc.rs:745-746 (run's doc comment), xtask/src/main.rs:105-107 (the --help text) and .github/workflows/ci.yml:1196-1198 (the job comment) state a two-way rule: "scoped to its nearest heading's directories, or anywhere in the tracked tree for an unscoped citation". A bare .md/.toml citation is unscoped — is_scoped_extension accepts only .rs — and citation_exists (xtask/src/architecture_doc.rs:733-738) matches it with p == Path::new(text), the repository root and nowhere else. So the stated rule is false in the permissive direction for a whole citation shape, and --help is the most user-facing of the four. The failure it produces says "not found where they're cited" and, by design (format_stale_citation's !is_scoped guard), prints no "(expected under …)" hint — so a contributor who adds a bare testing.md citation gets a red gate, a message that doesn't say where the file was expected, and help text that told them any tracked location would do.
This blocks because it is the defect class the whole change exists to prevent, and the accurate sentence already lives two files away.
Concrete fix. Copy the three-clause sentence already in CONTRIBUTING.md:74 / docs/architecture.md:11 — "scoped to its nearest heading's directories for a subsystem-specific .rs citation, at the repository root for a bare .md/.toml citation, or anywhere in the tracked tree for a bare directory name or an otherwise-unscoped citation" — onto all four surfaces, so the six texts agree and agree with the code. I traced each of those three clauses against citation_exists independently before proposing it: clause 1 covers both the bare and the partial-suffix .rs branches, clause 2 covers the root-level branch exactly, and clause 3 covers the leading whole-path match, the unconstrained suffix fallback and the path-component fallback. Adding a one-line test that asserts the --help string contains the .md/.toml clause would stop the six copies drifting apart again — the drift is what produced both this round's and last round's finding.
Two of the four surfaces flagged as fixed last round still stated a
two-way rule ("scoped to its nearest heading's directories, or
anywhere in the tracked tree for an unscoped citation"), which is
false for a bare .md/.toml citation: citation_exists checks those at
the repository root only. Copy the accurate three-clause wording
already used in CONTRIBUTING.md and docs/architecture.md onto the
module doc comment, run()'s doc comment, the CLI --help text, and the
CI job comment so all six surfaces agree with citation_exists.
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
Confirmed and fixed in c8067da: the module doc comment and Verified Left as-is: the suggested |
|
🔴 Automated review · pr-review-watcher · c8067da This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer. SummaryThis PR adds a CI gate ( 🚫 Blocking (must fix before merge)None. Non-blocking
|


Summary
cargo xtask check-architecture-doc: extracts every backtick-quotedpath citation from
docs/architecture.mdand fails, naming each stalepath, if it no longer exists in the tracked tree.
rustpath filter — the checker can validate a citation to any trackedfile, not just the ones the doc happens to cite today, so gating it on a
fixed filter list could skip the check on the very PR that goes stale.
xtask/src/crate_edges.rsandxtask/src/workflow_contract.rsstructural guards.rules for fenced code blocks, ATX heading indentation, and code-span
delimiter matching (rather than simplified approximations of each), and
to scope possessive/partial-suffix citations precisely instead of
falling back to an overly broad match.
run's filter-and-bail into a purecheck_citations(markdown, tracked)so the gate's own failure branch is directly testable, anddeduplicated the failure message so a file cited both bare and under a
section heading is reported once, not twice.
run's own delegation tocheck_citationswas itself unproven, so split outcheck_doc_at(root)and tested it end to end over a throwaway git repo. Also fixed a real bug
the dedupe surfaced — the failure message hinted a directory for
.md/.tomlcitations even though existence-checking never scopes thoseby section — and documented the new check in
CONTRIBUTING.mdanddocs/architecture.md's own caution note.Why
docs/architecture.md(added in #423) tells contributors to verify itscited paths manually because nothing currently checks that they still
exist — a renamed or removed file rots silently in the doc until a reader
notices. Filed as #430, a follow-up to #423 (which stayed documentation-only
per its stated scope).
Non-obvious decisions
(
`crate::`,`pub(crate) fn`, bare type names like`ActionReport`) and cites some files by bare name only(
`main.rs`,`lib.rs`), trusting surrounding prose/headings forwhich subsystem they belong to. The extraction heuristic and its known,
deliberate blind spots (documented in the module's doc comments) were
built and tested directly against the real doc's content rather than
designed in the abstract.
.rscitation (or a partial slash-path suffix like`app/mod.rs`)is scoped to its nearest
### \`heading's directories, or to a single crate narrowed out of a multi-crate heading by possessive prose (``rocm-dash-tui'sagent.rs``), rather than matched anywhere in the repo — the workspace has ~17 files literally namedmain.rs/lib.rs`, soan unscoped check would be a near no-op for exactly the citations it most
needs to catch.
not prose accuracy (e.g. which extraction pattern a module follows).
Risk: low
Dev-tooling only (
xtask+ CI wiring); no change to shipped CLI/daemonbehavior.
docs/architecture.mditself is explicitly out of scope forxtaskper its own module-map scoping note.Test plan
cargo test -p xtask— 68 unit tests in this module (including severalregression tests pinned to real doc content, review-driven CommonMark
edge cases, and direct tests of
check_citations's andcheck_doc_at'sfailure branches), plus a real-tree test that runs the actual check
against
docs/architecture.mdand asserts it passes.cargo xtask check-architecture-docrun manually against the doc (passes).(
apps/rocmd/src/lib.rs, thenengines/vllm/src/lib.rs) while leaving asame-named sibling file untouched, confirming the check fails and names
the specific stale path — each time reverted before committing.
cargo test --workspace,cargo clippy --workspace --all-targets -- -D warnings, andcargo fmt --checkall clean.PR checklist
expectations.tomlxfail rows involved.xtaskisexplicitly out of scope for
docs/architecture.md's own module map.Closes #430