Repository navigation
Conversation
|
Known limitations, deliberately out of scope:
|
TheGreatAxios
left a comment
There was a problem hiding this comment.
Review · Request changes
Ranks tool_search hits by name, description, and parameter text, counting each query word once. A parameter hit only raises a tool that already matches on name or description.
Findings
src/agent/lexical-rank.ts:40— parameter names go throughtokenizeLexical, which only splits on non-alphanumerics (lexical-rank.ts:11).includeArchivedbecomes the one tokenincludearchived, so the query wordarchivedadds nothing.include_archivedtokenizes toarchivedand scores 0.5.paramTokenScore(lexical-rank.ts:59) then requires that exact token. Tool names still match inside a larger token vianame.includes(lexical-rank.ts:53); parameter names do not, and nothing splits a camelCase boundary. Against two tools both described as "List records", withincludeClosedlisted beforeincludeArchivedand neither description containing "archived",search("list records archived")returns theincludeClosedtool first. The same catalog withinclude_closed/include_archivedreturns the archived tool first (scores 6 and 6.5). A barereleaseIdlikewise adds nothing for the query wordrelease; the parameter namedreleasedoes (6.5 against 6). The new fixtures useteamIdandincludeReleaseNotes, but those tests pass because the descriptions repeat the word ("Owning team", "release notes"). A boolean flag with no description, a normal JSON Schema shape, stays tied with catalog order. Splitting camelCase boundaries on parameter names before tokenizing (includeArchived→include/archived,releaseId→release/id) would close it. Substring-matching the raw lowercased name would not: the query tokenidwould then hitvalidandwidth.
Checked with bun -e in the PR worktree on those two catalogs. bun test of the lexical-rank, tool-search, tools-mcp-disconnect, skill-search, agent-search, and search-scorer-parity suites was 141 pass, 0 fail. None of those tests use a camelCase parameter name whose description does not already contain the query word.
1e1ff30 to
eef2bd2
Compare
|
Fixed in eef2bd2. Parameter keys now split on camelCase boundaries before tokenizing, so Your scenario is a regression test in |
Summary
tool_searchalso scores each tool's parameter names and descriptions, so a tool whose matching capability is a filter, such as Linear'slist_issuesand itsreleaseparameter, reaches the shortlist. camelCase keys split into words first, so an undescribedincludeArchivedflag matches "archived". A parameter match weighs less than a name substring and only boosts a tool that already matches by name or description.Verification
bun run checkpasses (9520 tests).list_issues, and a tool named for the query outranks a parameter-only match.includeClosedlisted beforeincludeArchived) ranks the archived tool first for "list records archived".Fixes CL-11366