fix(memory): reduce hybrid noise with cosine floor and tighter RRF

[codeaholicguy]

Summary

• require cosine similarity >= 0.50 only for semantic-only search entrants
• keep lexical matches eligible at any semantic similarity
• tighten reciprocal-rank fusion from k=60 to k=10
• add deterministic unit coverage and lifecycle documentation for the measured decision

Semantic search improved expanded-100 hit@1/3/5 from 81/91/96 to 88/97/98, but judged-irrelevant top-three results increased from 2.9% to 4.7%. Explained-query diagnosis found two weak semantic-only fillers and four lexical distractors reinforced by semantic rank. The selected combination addresses both mechanisms without adding configuration, inference, schema, or API surface.

Evidence

Configuration	hit@1	hit@3	hit@5	Known bad	Judged irrelevant	Bad / returned slots	Coverage	Zero
Dev semantic baseline	88%	97%	98%	6	4.7%	2.00%	42.3%	0%
Equal RRF k=10	88%	97%	98%	5	4.0%	1.67%	42.0%	0%
Asymmetric RRF 60/90	86%	97%	98%	5	4.0%	1.67%	41.7%	0%
Semantic-only cosine >=0.50	88%	97%	98%	4	3.3%	1.82%	55.5%	1%
Combined selected policy	88%	97%	98%	3	2.5%	1.36%	55.0%	1%

The go/no-go gate allowed at most a 1 percentage-point hit@3 regression. The selected policy has no hit@1/3/5 regression and halves the raw known-bad count from 6 to 3.

The 1% zero-result rate is deliberate: the lexical-zero 169.254.169.254 query previously received five semantic results with cosine 0.14-0.18 and no relevant top-five answer.

Judgment coverage caveat

The irrelevant metric is conditional on explicitly judged returned slots. Lexical search had 3 irrelevant results among 104 judged and 207 returned top-three slots. The semantic baseline had 6 among 127 judged and 300 returned slots. Coverage therefore fell from 50.2% to 42.3% despite hybrid producing 23 more judged slots in absolute terms.

Across the pooled lexical/hybrid top-three results, only 127 of 360 unique query/result pairs are judged; 233 remain unjudged. The table reports raw known-bad counts, conditional judged rate, returned-slot rate, and coverage together so unjudged results are not silently treated as relevant or irrelevant.

Benchmark provenance

Candidate verification used the expanded-100 fixture with AI_DEVKIT_BIN pointed at this worktree's built CLI in dev mode:

AI_DEVKIT_BIN=/home/ubuntu/code/ai-devkit/.worktrees/feature-memory-noise/packages/cli/dist/cli.js \
  npm run bench -- --label <candidate> --semantic --output /tmp/<candidate>.json

No experiment row was added to the benchmark README. A released-version leaderboard row will follow the next ai-devkit npm release.

Validation

• npm run build - 6/6 projects built
• npm test - 174 files, 2,165 tests passed across 6 projects
• npm run lint - 6/6 projects passed, 0 errors; 3 pre-existing CLI warnings
• npx vitest run --config e2e/vitest.config.ts - 1 file, 41 tests passed
• regression proof - both new unit tests fail when baseline fusion behavior is restored and pass with this change

Risk and rollback

The fixed 0.50 threshold is calibrated on the current fixture, whose pooled judgments are incomplete. The guard applies only to semantic-only candidates; lexical matches are never filtered by cosine. Reverting the two final ranking commits restores the prior fusion behavior without data migration.