[kamronis]

Cache completed partition row IDs and quantized distances for repeated opt-in IVF_RQ refinement queries in the existing session index cache. Hits skip candidate discovery and Accurate scoring; exact refinement and projections still execute independently.

Entries are memory-only, own their result columns, and share the existing budget, admission policy and load coalescing. Keys bind dataset/store, manifest generation/branch, physical index identity, original vector bits, resolved controls, budgets, factors and bounds. The manifest binds coverage and FRI state; ranked results never cross snapshots. Vector bits and invariant controls are fingerprinted once per segment; partition keys add the partition ID and centroid-distance bits.

Eligibility is limited to fully indexed single-vector flat IVF_RQ queries with a proven snapshot identity. Caller filters, explicit fragment/segment selections, external masks, and stale vector overlays bypass the result cache. Entries hold coarse candidates only: cross-column refinement (#9127) still reranks them with its own column, key, metric and final-space bounds, so distinct refinement inputs may share one coarse entry. Immutable index-data caches remain independent and reusable across queries and snapshots. No public signatures, options, dependencies, or persistent formats change.

Part of #8634. Builds on merged #8741 and preserves #10642's immutable-cache boundaries. SQ8 and shared batch execution remain separate PRs.

Validation after merging main at e602c45026ecace05081c880906615e6341ab713: result-cache key/value/lifetime/admission/concurrency cases, the affected scanner ANN tests and the io::exec::knn tests pass, together with cross-column refinement's paired single/batch and quantized-coarse cases. One added case warms the coarse entry and then reranks with a different refinement key: the hit reuses the cached coarse candidates and still returns the uncached final rows, distances and final-space bounds. Coverage also includes row/distance-bit parity and recall, both row-ID schemes, multi-fragment updates/deletes/compactions and FRI changes, snapshots/clones/index replacement, normalization/key separation, pressure/zero capacity, concurrent loads/cancellation/failures, and buffer ownership. Tests distinguish fully indexed queries with original vector files removed from exact refinement and unindexed fallback reads.

cargo fmt --all -- --check and full workspace cargo clippy --all --tests --benches --profile ci --locked -- -D warnings passed (one Cargo job; tests use ci).

Local performance measured before rebase (00c563668c7c33c0c66e7cc4cfd164f0d1fde8ee): reuse the frozen 16,384 × 128 float32 dataset, IVF_RQ8/Fast rotation model, eight partitions/probes, k=10 and quantized factor=32. Test 0/1/5/10/20/100% evenly distributed hot-query occurrences; other vectors are distinct. Index budgets are 32/3/2 MiB, metadata budget 8 MiB, resident columns Auto and Off. Two rounds in both arm orders, 400 requests per arm, give 1,600 observations per arm/case and 115,200 timed queries overall. All matched independent uncached row IDs and exact distance bits; minimum recall@10 was 0.9.

Warm traffic with default resident columns (Auto), 32 MiB index cache:

Repeats	p50 µs, uncached → cached	p95 µs	p99 µs	Timed QPS change
0%	901 → 938	944 → 994	998 → 1043	-3.9%
1%	907 → 935	954 → 982	999 → 1028	-2.3%
5%	909 → 933	954 → 999	1002 → 1039	+0.3%
10%	908 → 932	965 → 981	1007 → 1018	+4.2%
20%	906 → 928	961 → 997	1014 → 1239	+9.1%
100%	877 → 284	929 → 304	976 → 330	+208.3%

Distinct-query median overhead remains about 4.0%. At 5% repetition, paired QPS changes span −0.8% to +0.7%; at 10%, +2.8% to +5.0%. The 20% throughput gain accompanies a 22% worse p99. Full repeats reach 3.1× throughput and reduce the index-comparison counter from 18,944/query to zero; IVF ranking/planning still execute. Origin reads are zero in both arms.

Cache pressure with default resident columns:

Budget	Repeats	Origin KiB/query, uncached → cached	p95 µs	Timed QPS change
3 MiB	0%	0.0 → 0.0	953 → 979	-2.4%
3 MiB	20%	0.0 → 0.0	957 → 996	+7.7%
3 MiB	100%	0.0 → 0.0	926 → 305	+208.7%
2 MiB	0%	2125.5 → 2104.6	2161 → 2166	-0.4%
2 MiB	20%	2135.2 → 2148.5	2120 → 2190	-2.9%
2 MiB	100%	1725.7 → 1749.2	1747 → 1749	-0.9%

At 3 MiB, native occupancy stays at 2,569,802 bytes in both arms; results reach 575,616 bytes with turnover and no extra reads. With resident columns Off, native data also stays resident at 3 MiB: QPS changes −3.1% at 0% repeats and +12.1% at 20%.

At 2 MiB, native data does not fit in either arm and result hits are zero at every repeat rate. Results occupy <2.7 KiB. Auto origin bytes change −3.2% to +1.4%; Off bytes/requests are identical between arms. Across both modes, QPS changes −2.9% to +0.3%, with p95 up to 3.9% worse. Size removals include rejected candidates and evicted residents. Keep the existing admission policy based on this evidence; other workload/cache mixes still need evaluation.

Uncached parity oracles are prepared in an independent session, then dropped before timing; validation performs no searches on timed sessions. Recall uses exhaustive original-vector L2 references. Timing covers planning/execution, excluding preparation, verification, diagnostics and logging. QPS uses summed timed durations. Sessions are prewarmed; OS cache stays warm. Per-round/order observations are retained; no confidence intervals are claimed.

Lifetime VmHWM was 81.7 MiB including oracle preparation; maximum accounted cache occupancy was 3.64 MiB. Dataset size is ~11 MiB. Build peaks were 6.65 GiB RSS/11.76 GiB artifacts under 9/24 GiB guards. Optimized artifacts were removed after recording the binary hash.

Reproduction: base cf47711bda260edaad14c34afd780742e0452f0a, measured revision above, complete patch SHA-256 a7017027c2b214042ec01dd7b412a8332df946513013216a0ded64b2730f207b. Rust 1.98.1, Linux/i7-12700, release-with-debug/Haswell, two build jobs, affinity 2/4, two CPU/Rayon/Tokio workers, two allocator arenas, 2 GiB runtime address-space limit. Harness cloud/geo features are disabled; workspace checks use normal features. Baseline: the same executable’s uncached scorer selected by explicit full-fragment scope.

Harness, raw CSV, per-order comparisons, hashes, environment and reproduction instructions: /tmp/quantized-refinement-pressure/. Frozen fixture and original warm/fresh-session observations: /tmp/quantized-refinement-bench/. Benchmark files are excluded from the PR.

Benchmarks were not rerun against the rebased upstream cache implementation. Real remote-storage and production-workload performance remain unverified. The measured optimization is conditional on repeat traffic and a suitable cache budget.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The benefit still depends on repeat traffic and cache capacity. Reported pre-rebase measurements show about 4% median overhead for distinct queries, worse p99 at 20% repeats despite higher throughput, and no result hits with the tested 2 MiB budget.

Completed results compete with immutable index data and row-ID lookups for the shared budget. These timings predate the current base and its cache-resizing changes. Evaluate representative repeat rates, cache pressure and tail latency before broad adoption; current-head performance, remote-storage gains and production-workload behavior remain unverified.