fix(memory): escape scope and id values in LanceDB filters

[stepchanges]

Related issue

Fixes #5728

Summary

LanceDBStorage passes its filters to LanceDB where(), which takes a raw SQL (DataFusion) expression with no parameter binding. Several filters were built by interpolating caller-supplied values without escaping:

• scope prefixes in search(), _scan_rows() (used by list_records, get_scope_info, count, list_categories, and category/metadata deletes), delete(scope_prefix=...) and reset(scope_prefix=...)
• record ids in delete(record_ids=...) and in the id list built by category/metadata deletes

Memory.recall(), forget(), list_records(), info() and reset() forward an explicit scope= argument to these methods after only joining it under root_scope. A scope value containing a quote could therefore change the predicate and read or delete records outside the configured root_scope, and % or _ in a scope acted as LIKE wildcards. Legitimate scope names or ids containing an apostrophe failed with a SQL tokenizer error.

Fix (one source file, lancedb_storage.py):

• _sql_str() doubles single quotes. Every value interpolated into a filter now goes through it; the three call sites that already escaped ids inline now use the shared helper.
• _id_in_filter() builds id IN (...) from escaped ids (both delete paths and touch_records).
• _scope_prefix_filter() escapes \, % and _ and emits scope LIKE '<escaped>%' ESCAPE '\', so only the trailing wildcard is a pattern character. Prefix-match semantics are otherwise unchanged.
• reset() escapes quotes in its range predicate.

Compatibility. Lance accepts only backslash as the LIKE escape character (any other character is rejected at query time). LIKE ... ESCAPE '\' was verified on lancedb 0.29.2 and 0.30.0, which is the full range allowed by lancedb>=0.29.2,<0.30.1, with and without the BTREE scope index, for scan queries, vector queries (prefilter and postfilter) and deletes. The only behavior change for existing callers: % or _ inside a scope prefix now matches itself instead of acting as a wildcard, and a prefix ending in a backslash now works as a prefix.

Input validation on explicit scopes: considered, not added. Scope strings come from users, from LLM-inferred scopes and from data already stored, and can legitimately contain characters outside a strict allowlist. With escaping at the sink every character is matched literally, so an allowlist would add breakage risk without closing any additional path.

Not changed here (pre-existing, separate behaviors):

• Scope matching is a plain string prefix, so /app also matches /apple ([BUG] [BUG] LanceDBStorage scope-prefix filters cross path boundaries: /app matches /apple #7543, fix(memory): keep lancedb scope filters inside path boundaries #7544).
• delete(scope_prefix=...) also removes root-scope (/) records, and its OR is not parenthesized when combined with older_than ([BUG] LanceDBStorage.delete() ignores scope/older_than filters and causes accidental mass deletion when combining record_ids with categories #7419, fix(memory): honor combined LanceDB deletion filters #7472).

Verification

• Tests added or updated for the changed behavior
• Relevant tests and quality checks pass locally

New lib/crewai/tests/memory/test_lancedb_filter_escaping.py (26 cases) runs against a real temporary LanceDB table holding two tenants' records:

• control: a tenant A prefix returns only tenant A
• crafted scope prefixes (quote breakout, predicate injection, bare %, bare _, _ inside a tenant name) cannot read tenant B through search, list_records or count, and cannot remove tenant B through delete, category-filtered delete or reset
• Memory(root_scope="/tenant-a"): crafted explicit scope= values passed to recall() and forget() stay inside tenant A
• a crafted record id deletes nothing; ids containing an apostrophe delete exactly the matching record
• legitimate scopes with -, _ and / still match; %, _ and \ in scope names match literally

On current main 22 of the 26 cases fail. The other four are the control case and reset with wildcard payloads, whose range predicate never treated % or _ as wildcards.

Local checks (Python 3.13, macOS):

• uv run pytest tests/memory in lib/crewai: 179 passed (153 existing + 26 new); also 179 passed with lancedb 0.29.2
• uv run ruff check lib/ and uv run ruff format --check lib/: clean
• uv run mypy lib/: no issues in 930 source files
• pre-commit hooks on the changed files: pass

Additional context

Reported by @shaysakazi. A security advisory will be published with the release that includes this fix.

Related open PRs, neither reviewed yet:

• fix(memory): keep lancedb scope filters inside path boundaries #7544 (@coderdailyone) and fix(memory): honor combined LanceDB deletion filters #7472 (@subPTcon) also escape quotes, but match descendants with starts_with(scope, '<prefix>/'). On lancedb 0.29.2, which the current pin still allows, lance rewrites starts_with into an unescaped LIKE '<prefix>%': starts_with(scope, '/a_b') also matches /axb, /a%b and /a\b, so % and _ in the prefix still act as wildcards. 0.30.0 handles it correctly. For that reason this PR uses an explicit LIKE ... ESCAPE and does not take code from either PR. Their path-boundary and delete-filter fixes address separate behaviors and can build on _scope_prefix_filter() (for example scope = '<p>' OR <_scope_prefix_filter('<p>/')>), or raise the lancedb floor to 0.30.0. Thanks to both authors for the analysis.
• The earlier fix(memory): escape user input in LanceDBStorage SQL filters (#5728) #5729 (closed, unmerged) took a similar escaping approach.

🤖 Generated with Claude Code

---

Note

High Risk
Security fix for memory isolation: unescaped filters could allow cross-tenant read/delete via crafted scope or id strings passed through recall/forget and storage APIs.

Overview
Fixes SQL-style filter injection in LanceDBStorage, where caller-supplied scope prefixes and record IDs were interpolated into LanceDB where() expressions without escaping. Crafted scopes could broaden reads/deletes across tenants; % and _ in prefixes acted as LIKE wildcards; apostrophes in legitimate ids/scopes could break queries.

The change centralizes escaping in _sql_str, _id_in_filter, and _scope_prefix_filter (using LIKE ... ESCAPE '\' so only the trailing % is a wildcard), and wires them through save/touch, search, scan, delete, and reset paths. reset() also escapes quotes in its range predicate.

New integration tests in test_lancedb_filter_escaping.py cover tenant isolation, crafted scopes/ids, Memory with root_scope, and literal matching for %, _, \, and quotes. Behavior change: % and _ inside a scope prefix now match themselves, not wildcard semantics.

^{Reviewed by Cursor Bugbot for commit 2692913. Bugbot is set up for automated code reviews on this repo. Configure here.}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 6dce950d-a803-4222-9135-d7988d5c685a

📥 Commits

Reviewing files that changed from the base of the PR and between f414bbf and 2692913.

📒 Files selected for processing (1)

• lib/crewai/tests/memory/test_lancedb_filter_escaping.py

🚧 Files skipped from review as they are similar to previous changes (1)

• lib/crewai/tests/memory/test_lancedb_filter_escaping.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

LanceDB storage now escapes record IDs and scope prefixes when it builds SQL filters. Integration tests check literal matching and tenant isolation across searches, listings, counts, deletions, resets, and Memory operations.

Changes

LanceDB filter handling

Layer / File(s)	Summary
SQL predicates and ID operations lib/crewai/src/crewai/memory/storage/lancedb_storage.py, lib/crewai/tests/memory/test_lancedb_filter_escaping.py	Shared helpers escape SQL string values and build literal ID-list predicates. ID-based update, lookup, and deletion paths use these helpers. Tests cover crafted and quoted IDs.
Scope-prefix reads and mutations lib/crewai/src/crewai/memory/storage/lancedb_storage.py, lib/crewai/tests/memory/test_lancedb_filter_escaping.py	Search, scan, deletion, and reset filters escape scope-prefix values and wildcard characters. Tests cover tenant isolation, literal punctuation, and scoped Memory operations.

Priority: ⬆️ High

Severity of issue fixed: High

Merge Risk: ⚪ Minimal · up to 26929

The change hardens LanceDB filter construction and adds tests for tenant isolation and special characters. No merge-blocking risk was found in the supplied context.

Security Architecture Review

Security architecture risk: 🔵 Low · up to f414b

The change narrows the ability of crafted scopes or IDs to select unintended records. No new exposure was identified, but existing deletion paths do not consistently enforce a supplied scope, and database behavior has not been verified in this review.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — Where multiple scopes share a LanceDB table, predicate selection can affect records outside a caller’s intended scope. The changed escaping narrows that route for crafted scope strings and IDs; it does not establish authorization for every deletion branch.

Security Findings and Attack Paths

• observed — An ID-only forget call can pass a root-joined scope to storage, yet storage deletes solely by supplied IDs. This scope-independent behavior existed before the PR; the new ID escaping does not expand it.

Trust Boundaries and Controls

• observed — Scoped deletion still adds a root-record OR condition independently of the escaped prefix. That condition and the absence of parentheses when combined with an age predicate predate this PR.

Hardening Proposals

• proposed — If root_scope is intended to constrain ID-based deletion, include that scope in the ID-only predicate and define explicitly whether scoped deletion should include root records.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: escaping scope and ID values in LanceDB filters.
Description check	✅ Passed	The description includes the related issue, detailed summary, verification results, test coverage, compatibility notes, and additional context. It satisfies the repository template.
Linked Issues check	✅ Passed	No active directly linked issue targets remain. Issue #5728 is closed and provides historical context only. Therefore, no linked-issue coding requirements apply.
Out of Scope Changes check	✅ Passed	The production changes implement the PR objective by escaping LanceDB filter values and by building literal ID and scope-prefix predicates. The tests cover affected read, count, delete, reset, and ten…

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

🛠️ Fix failing CI checks 💡

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[linear]

SECT-245