Skip to content

refactor(memory): optimize exact retrieval and batch writes - #153

Merged
Grivn merged 9 commits into
masterfrom
codex/memory-algorithm-performance
Oct 3, 2026
Merged

Grivn merged 9 commits into
masterfrom
codex/memory-algorithm-performance

Conversation

@Grivn

@Grivn Grivn commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

Problem and change

Bounded Memory reads still scan/sort corpus-sized candidate sets, local BFS loads the whole store, and exact recall retains token/vector intermediates and repeats cosine work. This change uses indexes that match active ordered reads, indexed local BFS, bounded semantic/diff selection, scalar recall signals, a reusable vector decode buffer, and transaction-local prepared inserts.

Synced with master at 5c98ff9c (including #148). Preserve its covering edge indexes, caller-loaded snapshots, intent-ranked transition cache and rank-before-visit-budget rule. Cached transitions consume the same precomputed cosine scores as vector anchors and reranking. Scores, thresholds, per-anchor budgets, supersession demotion, causal ordering and commit boundaries retain their current-master contracts; no approximate search is introduced. Independent traversal parity is checked against the upstream implementation.

Performance against current master

Same benchmark sources, N=1/10/100/1,000/10,000, five alternating runs per version: 1,080 final samples. Apple M4, Go 1.25.4, one P, real temporary SQLite WAL, local 128-dimensional vectors. Values below are median process CPU time; raw wall time and allocations are also committed.

Case Current master Final PR
Hybrid recall, N=100 3.038 ms 2.555 ms
Hybrid recall, N=10,000 93.062 ms 66.531 ms
Latest by source, N=10,000 0.602 ms 0.0146 ms
Bounded two-hop neighborhood, N=10,000 40.306 ms 0.0595 ms
Atomic 10,000-row insert 524.331 ms 431.604 ms
Dense recall, N=100 18.324 ms 16.866 ms

Hybrid recall allocation at N=10,000 drops from 78.85 MB to 49.10 MB per call (37.7%). An isolated comparison with the merge-only version attributes about 10.24 MB of that reduction to decode-buffer reuse.

The earlier 6.51× dense-recall gain belongs to the historical baseline; current master already includes most of that graph-cache improvement. Small cases retain tradeoffs: N=10 BFS is 50.09 → 59.34 µs; single-record semantic paths add roughly 2 µs. Known-entity scanning at N=10,000 is about 4% slower; retention is near neutral. Exact vector scanning remains O(Nd), dense neighborhood ranking and negative substring/entity scans remain explicit costs. Shared-host timing is not a production latency promise.

Validation

  • go build -o mnemon .
  • make test (including vet and architecture gates)
  • go test -race ./internal/memory/... ./cmd/memory -count=1
  • MNEMON_EMBED_ENDPOINT=http://127.0.0.1:1 bash scripts/e2e_test.sh: 258 passed
  • npm test --prefix npm/cli: 13 passed
  • Local OpenCode runtime and ZCode shell suites passed; the Windows PowerShell-specific probe was skipped on macOS.
  • Upstream traversal parity across four intents, three budgets, ties, repeated anchors, deleted nodes, zero/negative vectors and dimension mismatch; caller snapshot ordering; decode buffer resizing and special float bits; existing index migration, BFS parity, rollback/panic and compaction tests.

CI is green on 30d61deb: Linux deterministic/npm checks and Windows Memory checks passed. The first Linux attempt failed in the unchanged Agency test TestStoreConcurrentPutHasOneFinalEffectAcrossInstances with promotion did not settle. That case passed 100 local repetitions on both the pure master baseline and this branch; the failed Linux job then passed on rerun. The failure was not reproduced locally, and its root cause remains unconfirmed.

Paid live providers and the Agency/Docker integration umbrella were not run. Benchmarks exclude CLI startup, provider latency and cold filesystem-cache timing.

Reports and complete samples: English analysis, 中文报告. The original measurements remain explicitly labeled as historical evidence.

GRIVN added 9 commits October 3, 2026 18:37
Measure local lookup, keyword and hybrid recall, diff, semantic candidates, bounded graph traversal, and atomic insert batches at 1, 10, 100, 1000, and 10000 insights. Fixtures keep vector dimensions, content length, limits, and graph degree fixed and exclude providers and setup from timings.
Extend the corpus benchmarks with dense graphs, metadata scans, retention refresh, entity-provided writes, and unsuccessful filtered lookups. Report process CPU time on Darwin and Linux alongside wall time and allocation metrics so scheduler contention does not masquerade as an algorithm change.
Replace three single-column insight indexes with active ordering indexes, avoiding full sorts for ranked and source reads without adding redundant write indexes. Read bounded BFS neighborhoods through endpoint indexes while preserving insertion order, filtering and limits; keep bulk loading for unlimited traversals. Skip known-entity scans when entities are explicitly provided.

Validated query plans, index upgrades and repeated opens, bounded/bulk traversal parity, make test, Memory race tests, and all 258 CLI E2E assertions.
Keep query-local scalar keyword and cosine scores, reuse active insight hydration and adjacency reads across anchors, and stream embeddings without retaining every vector. Reuse ordered time anchors after checking absolute timestamps. Select semantic and embedding-only diff candidates with one bounded typed heap and tokenize repeated query text only once.

Preserve scoring weights, thresholds, traversal budgets, supersession demotion and provider fallbacks. Characterization tests compare cached traversal against the prior algorithm across all intents and budgets, check query refresh and timestamp offsets, and enforce the top-K comparison bound. make test, Memory race tests and all 258 CLI E2E assertions passed.
Prepare insight and edge insert statements lazily inside their owning transaction, so import batches and graph writes do not compile identical SQL for every row. Commit and rollback close the statements; callback panics now roll back and release the connection before transaction-local pointers are cleared.

Validated commit, rollback, replacement, subsequent transactions, panic cleanup, make test, Memory race tests, all 258 CLI E2E assertions, and insert scaling benchmarks.
Document the read, recall, graph and storage complexity audit in English and Chinese, with N=1/10/100/1000/10000 comparisons, dense graph results, operation-count bounds and reproducible benchmark commands. Include all 1080 final samples with wall time, process CPU and allocation metrics, including unchanged controls and small-workload regressions.

Record successful product build, deterministic CI, Memory race tests and 258 CLI E2E assertions, and distinguish the remaining exact-scan and durability costs from the optimized paths.
Merge master at 5c98ff9 and preserve intent-ranked visit budgets, covering edge indexes, caller-loaded snapshots, embedding keep-alive and compaction. Reuse per-query scalar scores when caching the ranked transitions, and compare traversal against an independent copy of the upstream implementation.

Validated with the product build, make test, Memory race tests, all 258 CLI E2E cases, npm tests and local OpenCode/ZCode hook suites. Windows-only PowerShell coverage remains in integration CI.
Decode each stored vector into a request-local reusable buffer before retaining its scalar cosine score. Keep one float32 decoder and preserve the existing arithmetic, invalid-input behavior and query lifetime.

Validated buffer growth, shrinkage and special float encodings, upstream traversal parity, the product build, make test, Memory race tests and all 258 CLI E2E assertions. A paired 10,000-record probe reduced hybrid recall allocation by about 10.24 MB per operation.
Compare the merged implementation with master at 5c98ff9 using 1,080 fresh alternating benchmark samples. Attribute upstream transition caching separately, document the decode-buffer allocation reduction, and retain small-input regressions and exact-search complexity limits.

Update the English and Chinese reports with current traversal parity and local validation results. Preserve the original experiment as historical evidence and include the separate decode-buffer probe.
@Grivn
Grivn merged commit 55ecd3e into master Oct 3, 2026
3 of 4 checks passed
@Grivn
Grivn deleted the codex/memory-algorithm-performance branch October 11, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant