Repository navigation
Conversation
Self-Review (
|
TheGreatAxios
left a comment
There was a problem hiding this comment.
Review · Request changes
Folds one trailing plural "s" on name tokens so a singular query can match a plural tool name and the reverse, with an exact spelling still scoring higher.
Findings
src/agent/lexical-rank.ts:24—FOLDED_NAME_TOKEN_SCOREis 2.5 so a folded name hit stays under the exact token score of 3. That clearsTOOL_SEARCH_SCORE_RATIO(0.75) for a two-word query whose top is 6, which is what the comment next to the constant describes. It does not clear the same cut for a one-word plural query.name.includes(rawQuery)adds 1 on top of the exact token score, solist_issuesscores 4 for"issues"and the floor is 3.get_issueandsave_issueonly fold, so they score 2.5 andcreateToolIndex(...).search("issues", 20)returns just["mcp__linear__list_issues"]. The other direction still works:search("issue", 20)keepslist_issuesat 3.5. No weight strictly under 3 can clear a floor of 3, and 3 would tiesearch("issue")(both names also take the substring point), which drops "exact spelling outranks folded." One way out is to stop the one-token raw-substring bonus from setting that floor — a one-word query is already paid for by the token score — so the top stays 3, 2.5 still clears0.75 * 3, and the exact token still ranks first. The new tests never callsearch("issues").
Checked with bun -e against a Linear-shaped catalog (list_issues, get_issue, save_issue) in the PR worktree: scores 4 / 2.5 / 2.5 for "issues", and only list_issues returned. bun test src/agent/lexical-rank.test.ts src/agent/tool-search.test.ts was 99 pass, 0 fail; those tests do not cover this query.
1e1ff30 to
eef2bd2
Compare
f96a9c2 to
1ed4a78
Compare
|
Thanks, good catch. Reproduced: "issues" alone returned only Fixed in 1ed4a78 ( I didn't drop the bonus for every one-word query as suggested: |
TheGreatAxios
left a comment
There was a problem hiding this comment.
Review · Approve
Matches name tokens across a trailing plural s without double-counting spelling variants or a one-word whole-query bonus.
No findings.
Reviewed 1ed4a78af29424e93ec6f1ba9193c3c7858f47aa against this PR's declared #1573 base. The prior whole-query-bonus finding is closed by the regression at src/agent/tool-search.test.ts:531–541. The documented trailing-s collisions and five-result replay miss remain accepted limitations, not new findings.
Verification: focused lexical-rank, tool-search, replay, and scorer-parity suites: 107 passed; three temporary probes passed. bun run check rerun exited 0; the full-suite review run reported 9,535 passed, 0 failed. Temporary tests were removed.
|
Triage: In progress Triage is waiting for CI to finish before it decides. This comment shows the verdict once it has. |
1ed4a78 to
5fd61fe
Compare
Summary
mcp__linear__list_issuesand "issues" findsget_issueandsave_issue. An exact-spelling name match outranks a folded one, so "list issues" and "linear issues" ranklist_issuesfirst in any catalog order. Description and parameter tokens keep exact matching.list_issuesto 4, set the cutoff at 3, and cut every singular-named tool at 2.5. A one-word query found only inside a longer name token ("granola" inmygranolahoard) keeps the bonus.list_issues(it ranks 10th). It passed on the base branch only because "issue" and "issues" counted as two words. The query names both releases and issues, and the release tools match more of its words. The replay test keeps it astest.failing, so a later scorer change that recovers it shows up.mainby hand.Verification
bun run check(lint, typecheck, dead exports, build, test) exits 0 on this branch: 9535 pass, 0 fail.list_issues, against three on the base branch. "Linear retrieve pipeline release issue list and issue details" and "issue list" now pass; "Linear read pipeline release issues and issue details" is the known miss above.search-scorer-parity.test.ts, pass with no change to their expected order; the diff adds tests and changes no existing assertion.test.failingreplay case, which passes there.Fixes CL-11435