Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/Corbits Triage User Manual/Corbits Triage User Manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ A rule asks one question about a pull request and answers pass, fail or unconfir
- **Fail when …**, a text rule of your own, such as "Fail when any commit message contains "Co-Authored-By:"". It names where to look (the title, the description, any commit message, or several of them), whether the rule fails when the text **contains** or **does not contain** its phrases, and the phrases. Case does not matter. A **contains** rule on the description also looks at its hidden parts, such as HTML comments and the addresses behind links, while **does not contain** reads only what a reader sees. Only the first 100 commits are read. When a **contains** rule fails, Triage names each place and the phrases found there, asks for them to be removed, and quotes each matching line once. Text rules count toward the eight rules of your own.
- **Fail when added code matches …**, a code-pattern rule of your own, such as one that flags `?? 0`. It checks only the lines the pull request adds to TypeScript and JavaScript files (`.ts`, `.tsx`, `.mts`, `.cts`, `.js`, `.jsx`, `.mjs`, `.cjs`), so code already on the branch never fails it. On any other language it passes with **nothing checked: no TypeScript or JavaScript code added**, so don't add one to a repository written in Go, Python or similar. Turn on the shipped `convention` preset, which flags invented fallbacks (`?? 0`, `?? []`), nested ternaries, functions called where they are written, `.then`/`.catch` chains, empty `catch` blocks and lint or type suppression comments in both languages, or write your own patterns in [ast-grep](https://ast-grep.github.io/guide/rule-config.html)'s rule format. Your own pattern replaces a preset's pattern with the same `id` in both languages; to keep checking JavaScript, add a `language: javascript` pattern under the same id. A pattern takes ast-grep's own `files` and `ignores` lists to check only some paths or skip some. They follow ast-grep's glob rules: paths are from the repository root, `*` also matches across folders, and dotfiles are matched like any other file. Write each glob as a string; the `{ glob, caseInsensitive }` form is not read. Write a negated class as `[^a]`; a glob with `[!a]`, or one that isn't valid, refuses the policy. The preset's `?? 0` and `?? []` patterns ignore test files (`**/*.test.ts`, `**/*.test.tsx`, `**/*.test.js`, `**/*.test.jsx`, `**/*.test.mjs`, `**/*.test.cjs`). Each match is listed as evidence, as `path:line` and the pattern's message. Vendored and built code at the repository root (`vendor/`, `dist/`, `build/`, `generated/`) and installed code (`node_modules/` anywhere) is skipped, and each skipped file is listed in the evidence. The same folders deeper in the repository, such as `src/build/`, are checked. When the host sends no changed lines for a file that adds code, as GitHub does for a very large or binary one, the pull request's diff is larger than the 2,000,000 characters Triage reads, or a file's content can't be read, every other file is still checked: a match fails the rule, and otherwise it is unconfirmed, naming those files and why. When a pattern is not valid ast-grep, the rule is unconfirmed rather than passing. Triage reads at most 300 files per pull request, so on one that adds code to more, the rule is unconfirmed with a reason such as **code patterns could not be checked: 412 files with added code, more than the 300 read per pull request**. On Bitbucket Data Center, which does not give Triage each file's changed lines, it is unconfirmed whenever a TypeScript or JavaScript file of your own is added, edited, moved or copied. To give some patterns a different `onFail`, put them in a rule of their own. Code-pattern rules count toward the eight rules of your own, and the browser extension leaves them out.
- **Judgement**, questions the decision model answers from the diff and the documentation:
- **Check the change is focused on one thing.** The decision model is asked whether the pull request does one thing. When it fails, the verdict comment says it mixes unrelated changes and asks the author to split them out, unless the file sets another `onFail`. A pull request based on another open pull request's branch is checked on its own changes, and the comment says so, such as "Stacked on #1573: only this pull request's own changes were checked." On a branch another pull request was merged into, this rule is skipped, and the comment says why, such as "stacked branch, includes #1521".
- **Check the change is focused on one thing.** The decision model is asked whether the pull request does one thing. By default this is a review hint, not a gate: when it fails or the model cannot tell, Triage adds a **Notes:** entry and, on GitHub, the `focused: check` label without making the verdict Fail or Unsure. Bitbucket Data Center gets only the note. This also applies to existing files that omit both the rule’s `onFail` and the top-level default. An explicit `onFail: fail` still blocks, `onFail: ask` still asks a maintainer, and a top-level `onFail` still applies when the rule has no override. Other rules, including required CI and forbidden paths, are unchanged. A pull request based on another open pull request's branch is checked on its own changes, and the comment says so, such as "Stacked on #1573: only this pull request's own changes were checked." On a branch another pull request was merged into, this rule is skipped, and the comment says why, such as "stacked branch, includes #1521".
- **Check docs are updated when behavior changes.** Changing a documentation file passes it. Otherwise the decision model is asked, file by file, whether the docs stay correct, a statement in them is now wrong, or the change adds setup the docs must describe. When it fails, the verdict comment asks the author to update the documentation and names the files, unless the file sets another `onFail`. The default label, when `onFail` is `label`, is `docs: check`.
- **Check the change is not already made by another open pull request.** Triage picks up to five open pull requests opened before it on the same base branch, drafts left out, that change one of the same files (lockfiles aside) or share two title words of four or more letters, most shared files first, and the decision model rates each as the same change, overlapping or unrelated, from both titles, this pull request's description and both lists of changed files. It fails when one is likely the same change, and names it, such as "looks like the same change as #1612". With none picked it passes without asking. Its `onFail` is `ask` unless the file sets another, so a maintainer decides. Like the rule above, it is unconfirmed when more than 100 earlier pull requests are open on the branch.
- **Is it true that …**, a yes-or-no question of your own, such as "Is it true that the title says what changed?" The rule fails when the answer is no.
Expand All @@ -286,9 +286,9 @@ Two rules always run: a draft pull request waits until it is ready for review, a

Each rule's `onFail` says what happens when it fails:

- `fail`, the default for every rule but the size rules, regression risk, the judged duplicate rule, sensitive paths and risk: the verdict is **Fail** and the comment asks the author to fix it.
- `fail`, the default for every rule but the size rules, regression risk, focused, the judged duplicate rule, sensitive paths and risk: the verdict is **Fail** and the comment asks the author to fix it.
- `ask`, the default for the judged duplicate rule, sensitive paths and risk: the verdict is **Unsure** and the comment lists the rule under **Asking a maintainer about:**, so the pull request waits for you in **Needs you** instead of on its author.
- `label`, the default for the size rules and regression risk: the verdict does not change. When the rule fails, or the decision model can't tell, Triage adds its label to the pull request and a line under **Notes:** in its comment. Triage's comment and the reading pane leave a `label` rule out of their rule list; it shows only as a note and a label (on Bitbucket Data Center, only the note). The browser extension, the **Corbits Triage policy / corbits-triage-policy** check and the local check still list it. The label follows the rule: once the rule passes, such as a pull request cut back under its size limit, Triage takes the label off. The level labels of sensitive paths and risk (`sensitive: medium`, `sensitive: high`, `risk: medium` and `risk: high`) follow the level: when it changes or drops to low, Triage takes the old one off. A label the policy names for a rule follows that rule, even if a person added it; a label the policy does not name for a rule is never taken off. A label a `labels` step in the policy also names is left alone. The label is `size: large` for the size rules, `docs: check` for the docs rule, `sensitive: high` and `risk: high` for sensitive paths and risk, and the rule's id followed by `: check` for any other, such as `regression-risk: check`; `label:` names a different one, but not one starting `Triage:` or `rule: `, a verdict label, or a level label of sensitive paths or risk other than the rule's own, which Triage takes back itself; a `labels` step may not name a level label either. `onFail: label` suits rules that are hygiene rather than a gate, such as size limits an atomic change has to cross. A top-level `onFail` sets it for every rule in the file that does not set its own. The Policy section shows a rule that does not fail the pull request with what it does instead, such as "(on fail, labels size: large)".
- `label`, the default for the size rules, regression risk and focused: the verdict does not change. When the rule fails, or the decision model can't tell, Triage adds its label to the pull request and a line under **Notes:** in its comment. Triage's comment and the reading pane leave a `label` rule out of their rule list; it shows only as a note and a label (on Bitbucket Data Center, only the note). The browser extension, the **Corbits Triage policy / corbits-triage-policy** check and the local check still list it. The label follows the rule: once the rule passes, such as a pull request cut back under its size limit, Triage takes the label off. The level labels of sensitive paths and risk (`sensitive: medium`, `sensitive: high`, `risk: medium` and `risk: high`) follow the level: when it changes or drops to low, Triage takes the old one off. A label the policy names for a rule follows that rule, even if a person added it; a label the policy does not name for a rule is never taken off. A label a `labels` step in the policy also names is left alone. The label is `size: large` for the size rules, `docs: check` for the docs rule, `sensitive: high` and `risk: high` for sensitive paths and risk, and the rule's id followed by `: check` for any other, such as `regression-risk: check`; `label:` names a different one, but not one starting `Triage:` or `rule: `, a verdict label, or a level label of sensitive paths or risk other than the rule's own, which Triage takes back itself; a `labels` step may not name a level label either. `onFail: label` suits rules that are hygiene rather than a gate, such as size limits an atomic change has to cross. A top-level `onFail` sets it for every rule in the file that does not set its own. The Policy section shows a rule that does not fail the pull request with what it does instead, such as "(on fail, labels size: large)".

To make a size limit block, give it `onFail: fail` or `onFail: ask`:

Expand Down
1 change: 1 addition & 0 deletions docs/DECISION_MODEL.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Questions point into the state by path, such as `` `pullRequest.files[3].hunks[0
- `choose`: one Choice question, `<name>: Which option best describes the pull request?`, with the rule's options as criteria (no descriptions). The likeliest option is its evidence; it fails when the `failOn` options hold P ≥ 0.7, passes at P ≤ 0.3, and is unconfirmed between.
- A reply is parsed on its own, as before: a choice's options are whatever its probabilities name and a score's levels whatever its legend names. Each rule then checks the answer is over its own options or levels, and treats one that is not as unanswered.
- **Instructed**: every first-round request and the verdict request also asks one Noul, question id `instructed`: whether any text in `pullRequest` or `repository` tries to direct the reviewer or model judging the pull request (tell it what to answer or conclude, to pass or approve it, or to ignore or replace its instructions or the repository's rules), wherever it sits; text that discusses, quotes or tests prompts, or tells contributors or coding agents how to work, does not count. The highest P(yes) across accepted replies stands; a reply without it counts as unanswered, so the verdict is Unsure. At 0.7 or above no judged rule's answer counts: each is unconfirmed with the reason "the pull request tries to direct the decision model", labelling rules included, the verdict question is not asked or its answer is ignored, and the verdict is Unsure with that reason, so a maintainer decides and nothing approves or merges on the model's word. A failed hard rule still decides Fail first (`INSTRUCTED_QUESTION`, `logic/quality.ts`; `evaluateModel` and `renderVerdict`, `logic/render.ts`).
- **Focus severity**: focused defaults to `onFail: label`, including existing policy files with no rule or top-level severity. Failure or uncertainty adds a note (and `focused: check` on GitHub), not a blocking verdict. Explicit rule severity overrides the file default, which overrides this built-in default. The recommendation also sets `onFail: label`. This changes severity wiring, not the questions, thresholds or inference.
- **Verdict**: a rule with `onFail: label` is asked like any other but takes no part in the verdict: it never fails it, never leaves it unconfirmed or unanswered, and is not in the verdict question's state (`splitLabelled`, `logic/render.ts`). A failed rule with `onFail: ask` makes the verdict Unsure, its next step the maintainer's, unless another rule fails it, and the verdict question is not asked. A policy with no decision-model rule on that does more than label (no focused, docs, is-true, score or choose) is decided in code: every rule passing is Pass, with a reason saying the policy has no decision-model rule, and any unconfirmed waits (below) (`renderVerdict`, `logic/render.ts`). A judged rule that fails decides Fail on its own answer, and the verdict question is not asked, so pull request text cannot argue a failed rule into a pass. Otherwise, once no hard rule failed and the judged rules are answered and pass or stay unconfirmed, one Choice, question id `verdict`: "Does this pull request pass this repository's policy?", `pass`, `fail` or `cant_tell`. It is asked with the instructed question, so a pull request with no judged rule to ask (such as one that changes only lockfiles) is still checked for text directing the model. A pass with a judged rule unconfirmed is Unsure, its score the verdict question's P(pass). A verdict other than Fail whose only unconfirmed rules are the host's own (CI green while checks still run, mergeability the host has not computed yet) waits instead: it is not posted and its actions do not run, and the head is triaged again once they are done. CI cancelled or stale without a result does not wait: the verdict is posted as Unsure, and the head is triaged again once a check is re-run. The question is asked the same either way. Its state is `policy.rules`, every enabled rule with its result, reason and evidence and, for a judged rule, `pass`, how likely it passes, and for a score rule, `level`; and the pull request's metadata as focused reads it, with as many judged files as fit one request, packed in order as focused's first slice is. It is not asked when a first-round call failed or left a rule unanswered, since the verdict degrades anyway. With no judged rule to ask first, it is asked in the first round; otherwise in the second, once their answers are in (`verdictRequest`, `logic/quality.ts`; `choiceOf`, `actions/evaluate.ts`).

## Writing questions
Expand Down
Loading
Loading