Skip to content

fix: harden memory tool input validation - #5701

Open
MatthiasHowellYopp wants to merge 14 commits into
crewAIInc:mainfrom
MatthiasHowellYopp:feat/valkey-2-memory-tools-hardening
Open

MatthiasHowellYopp wants to merge 14 commits into
crewAIInc:mainfrom
MatthiasHowellYopp:feat/valkey-2-memory-tools-hardening

Conversation

@MatthiasHowellYopp

@MatthiasHowellYopp MatthiasHowellYopp commented May 4, 2026 •

Copy link
Copy Markdown
Contributor

Description:

Part 2/4 of adding Valkey as a storage backend for CrewAI. This PR hardens the memory tools against malformed inputs that surface more frequently with the new storage paths.

What changed:

  • memory_tools.py — RecallMemoryTool and RememberTool now handle None, empty lists, and lists of empty strings gracefully, returning descriptive error messages instead of crashing. String inputs are normalized to lists. The queries and contents fields are now optional with min_length=1 validation, and both schemas use extra="forbid" to reject unexpected parameters.

  • en.json — Updated tool descriptions for recall_memory and save_to_memory to include explicit parameter format examples (e.g. {"queries": ["search term"]}), reducing LLM misuse of the tool interface.

Testing:

test_memory_tools.py (15 tests) — Covers None input, empty lists, empty strings, string-to-list conversion, deduplication across queries, and single vs. batch remember paths.

Summary by CodeRabbit

  • New Features

    • Memory tools accept optional inputs and normalize single strings or lists; non-string items are ignored.
  • Bug Fixes

    • Improved handling of missing/empty/whitespace inputs with clearer error messages; previous recall/remember flows preserved.
  • Documentation

    • Instruction text updated to require explicit JSON parameters for recall and save operations.
  • Tests

    • Added tests covering non-string item handling and related edge cases.
  • Chores

    • Linting config adjusted to exclude JSON files.

Review Change Stack


Note

Medium Risk
Changes distributed A2A cancellation and upload caching when VALKEY_URL is set; misconfiguration or missing crewai[valkey] can break cancellable tasks, though agent-card pickle deserialization is deliberately isolated from remote caches.

Overview
Adds optional Valkey (crewai[valkey], VALKEY_URL) as a JSON-backed cache layer alongside existing aiocache/Redis, with shared URL parsing in cache_config.py and a new ValkeyCache (Glide client, lazy init, event-loop rebind for sync asyncio.run() callers).

Upload cache is refactored behind a CacheBackend protocol: memory/redis unchanged via aiocache; cache_type="valkey" uses ValkeyCacheBackend with CachedUpload dict serialization, per-key clear, and a minimum 1s TTL when expires_at is past/zero so entries do not live forever. Valkey/crewai imports stay lazy so crewai-files still loads without crewai on the default path.

A2A routes task cancellation flags through Valkey when VALKEY_URL is set (polling; no pub/sub yet); otherwise aiocache/Redis behavior is preserved. Agent card fetch caching is pinned to in-process SimpleMemoryCache so pickle never reads from operator-controlled Redis/Valkey.

Memory tools (RecallMemoryTool / RememberTool) now tolerate missing/empty/malformed queries/contents, normalize strings to lists, filter junk, use extra="forbid", and return actionable errors; en.json documents required JSON parameter shapes.

Reviewed by Cursor Bugbot for commit fc188ce. Bugbot is set up for automated code reviews on this repo. Configure here.

@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 3 times, most recently from 32ed03e to 0f0823f Compare May 7, 2026 19:59
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 0f0823f to c35cd7e Compare May 11, 2026 19:53
@coderabbitai

coderabbitai Bot commented May 11, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Memory tools' input schemas now permit optional queries and contents and forbid extra fields; tools normalize string inputs to lists, trim and filter entries (including non-strings), and return explicit errors when no usable items remain; translations and tests updated accordingly.

Changes

Memory Tools Input Validation

Layer / File(s) Summary
Input Schemas
lib/crewai/src/crewai/tools/memory_tools.py
RecallMemorySchema.queries and RememberSchema.contents change from required list[str] to optional list[str] | None with min-length validation and model_config = {"extra": "forbid"}.
Recall Tool Input Processing
lib/crewai/src/crewai/tools/memory_tools.py
RecallMemoryTool._run signature expands queries to accept list[str] | str | None. Adds validation for missing/empty inputs, string-to-list normalization, trimming/filtering blank and non-string queries, and targeted errors when no valid queries remain.
Remember Tool Input Processing
lib/crewai/src/crewai/tools/memory_tools.py
RememberTool._run signature expands contents to accept list[str] | str | None. Adds validation for missing/empty inputs, string-to-list normalization, trimming/filtering blank and non-string contents, and targeted errors when no valid contents remain.
Tool Instruction Translations
lib/crewai/src/crewai/translations/en.json
recall_memory and save_to_memory tool instructions now explicitly require queries and contents parameters respectively, with clearer JSON examples.
Test Module & Fixtures
lib/crewai/tests/tools/test_memory_tools.py
New pytest fixtures instantiate a mocked memory object and both tool instances; tests added/expanded to cover validation and behavior.
Recall Tool Tests
lib/crewai/tests/tools/test_memory_tools.py
TestRecallMemoryToolValidation covers None/empty/whitespace inputs, string-to-list normalization, per-query memory.recall invocation, no-match messaging, match formatting, and deduplication.
Remember Tool Tests
lib/crewai/tests/tools/test_memory_tools.py
TestRememberToolValidation covers None/empty/whitespace inputs, string-to-list normalization, routing to memory.remember vs memory.remember_many, whitespace filtering, and success messaging.
Non-String Item Tests
lib/crewai/tests/tools/test_memory_tools.py
TestNonStringItemHandling asserts filtering of non-string elements and errors when no valid strings remain for both recall and remember tools.
Ruff Exclude JSON
pyproject.toml
Added **/*.json to tool.ruff.extend-exclude.

Sequence Diagram

sequenceDiagram
  participant User
  participant RecallMemoryTool
  participant Memory
  User->>RecallMemoryTool: sends queries (None|string|list)
  RecallMemoryTool->>RecallMemoryTool: normalize, trim, filter, dedupe
  RecallMemoryTool->>Memory: memory.recall(valid_queries)
  Memory-->>RecallMemoryTool: returns matches
  RecallMemoryTool-->>User: formatted matches or error
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the primary change: hardening memory tool input validation to handle malformed inputs gracefully.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
lib/crewai/tests/tools/test_memory_tools.py (1)

50-55: ⚡ Quick win

Add malformed mixed-type input tests to lock in hardening behavior.

Given the hardening objective, consider adding coverage for cases like queries=["ok", 123] and contents=["fact", None] to ensure the tools never crash and always return stable validation/success responses.

Also applies to: 109-113

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/tools/test_memory_tools.py` around lines 50 - 55, Add tests
that exercise malformed mixed-type inputs so the memory tools never crash and
always return stable validation or error responses: extend
test_list_of_empty_strings_returns_error (and the analogous test around lines
109-113) to include cases like queries=["ok", 123] and contents=["fact", None],
calling RecallMemoryTool._run (and any corresponding tool methods used in the
other test) and asserting the call does not raise and returns a predictable
error/validation result (e.g., contains "Error" or a stable validation payload)
instead of crashing.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/src/crewai/tools/memory_tools.py`:
- Around line 54-63: The current guard assumes every item in queries (and
similarly in contents at the other block) is a string and calls .strip(), which
will crash for non-string items like integers or booleans; update the
normalization to first handle non-list inputs, then coerce/filter items to
strings (or filter with isinstance(item, str)) before calling .strip() so
queries = [q for q in queries if isinstance(q, str) and q.strip()] (and apply
the same change to the contents handling around the 115-123 region) ensuring the
function that processes queries/contents rejects or converts non-string entries
rather than raising AttributeError.

---

Nitpick comments:
In `@lib/crewai/tests/tools/test_memory_tools.py`:
- Around line 50-55: Add tests that exercise malformed mixed-type inputs so the
memory tools never crash and always return stable validation or error responses:
extend test_list_of_empty_strings_returns_error (and the analogous test around
lines 109-113) to include cases like queries=["ok", 123] and contents=["fact",
None], calling RecallMemoryTool._run (and any corresponding tool methods used in
the other test) and asserting the call does not raise and returns a predictable
error/validation result (e.g., contains "Error" or a stable validation payload)
instead of crashing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a1b25a8-04e6-464a-bfa4-5abfa951e4df

📥 Commits

Reviewing files that changed from the base of the PR and between 63a9e7e and c35cd7e.

📒 Files selected for processing (3)
  • lib/crewai/src/crewai/tools/memory_tools.py
  • lib/crewai/src/crewai/translations/en.json
  • lib/crewai/tests/tools/test_memory_tools.py

Comment thread lib/crewai/src/crewai/tools/memory_tools.py
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from c35cd7e to d9d3416 Compare May 11, 2026 20:32
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 3 times, most recently from bf40d62 to b8cdaf3 Compare May 14, 2026 13:35
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from b8cdaf3 to 2cdb2e5 Compare May 19, 2026 18:17
@MatthiasHowellYopp

Copy link
Copy Markdown
Contributor Author

@greysonlalonde hoping I can get a review on this.

@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 8 times, most recently from c88fd5c to 8c2d634 Compare May 26, 2026 13:58
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 4 times, most recently from b3d2927 to 3bf8442 Compare June 2, 2026 15:17
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch 4 times, most recently from f7e81cb to e499e9b Compare June 12, 2026 13:16

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/a2a/utils/task.py
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 7edcd28 to d2084dc Compare September 29, 2026 15:27

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/a2a/utils/agent_card.py Outdated
Comment thread lib/crewai-files/src/crewai_files/cache/upload_cache.py
Matthias Howell and others added 4 commits September 29, 2026 13:05
Extract duplicated Redis URL parsing into a shared cache_config utility.
Introduce ValkeyCache as a lightweight async key/value cache using
valkey-glide. Wire it into A2A task handling, agent card caching, and
file upload caching.

Part 1/4 of Valkey storage implementation.
Sets CLIENT SETNAME to 'crewai_valkey' so connections are identifiable
in CLIENT LIST and monitoring tools (Valkey Admin, CloudWatch).

Signed-off-by: Matthias Howell <matthias.howell@improving.com>
Set client_info_tag="crewai" on the Glide client so the connection reports
lib-name GlidePy(crewai) via CLIENT INFO, letting operators attribute Valkey
usage to CrewAI. Bumps the valkey-glide floor to >=2.5.2, where client_info_tag
was introduced. client_name (CLIENT SETNAME) is unchanged; the two are
independent fields. Metadata only, no behavioural change.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- crewai-files no longer hard-imports crewai at module load. parse_cache_url
  moves to a lazy import inside the Valkey backend path, so importing
  crewai_files without crewai installed works again (the default in-memory
  cache never needs it). Preserves the crewai[file-processing] -> crewai-files
  direction. Adds regression tests.
- agent_card: pin the AgentCard PickleSerializer @cached to an in-process
  SimpleMemoryCache. Previously it used the default alias, which A2A wires to
  VALKEY_URL/REDIS_URL — pickling cache values to a network Redis/Valkey is a
  code-execution surface on cache read if that store is writable. Pickle now
  never deserializes from an operator-controlled backend.
- ValkeyCache/ValkeyCacheBackend/parse_cache_url now carry use_tls (from
  rediss/valkeys schemes) so cache connections honor TLS instead of silently
  opening a plaintext connection to a managed endpoint.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from d2084dc to 86591c7 Compare September 29, 2026 17:05

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/a2a/utils/task.py
The agent-card pickle cache is pinned to its own SimpleMemoryCache via the
@cached decorator, so it no longer needs the process-wide aiocache default
alias configured. The lazy _ensure_cache_configured() helper still called
caches.set_config(get_aiocache_config()) on every fetch, which replaced the
default alias that the in-memory A2A cancel path (poll_for_cancel_aiocache ->
caches.get('default')) depends on. A running poller and a later cancel() could
then hold different SimpleMemoryCache instances and never see the cancel flag.

Remove the helper, its call, and the now-unused imports. Add regression tests
asserting the helper is gone and the default alias is stable.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 86591c7 to cd7d494 Compare September 29, 2026 19:07
_ensure_task_cache built ValkeyCache without use_tls, so a VALKEY_URL with a
TLS scheme (valkeys:// or rediss://) opened a plaintext connection and A2A
cancel/cancel-watch failed against a TLS-only Valkey. parse_cache_url already
extracts use_tls from the scheme; thread it through. Lives on the base branch
where the A2A task cache is introduced so it applies to the whole stack.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from cd7d494 to 58fcc71 Compare September 29, 2026 19:14
Setting VALKEY_URL routes A2A task cancellation through ValkeyCache, which
imports glide at module load. Without the optional valkey extra installed, that
raised a bare ImportError from every cancellable A2A task. Catch it in
_ensure_task_cache and re-raise with an actionable message pointing at
'pip install crewai[valkey]' (or unsetting VALKEY_URL).

This is the fail-loud option. A graceful fallback to the aiocache Redis path is
deliberately left for later, pending the broader third-party provider/config
decision. Adds a regression test.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 58fcc71 to 1882a85 Compare September 29, 2026 19:20

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py Outdated
Comment thread lib/crewai-files/src/crewai_files/cache/upload_cache.py
Comment thread lib/crewai/src/crewai/utilities/cache_config.py
ValkeyCache cached a single GlideClient and asyncio.Lock on the instance.
UploadCache's sync methods each call asyncio.run(), which creates then closes a
fresh loop, so the second sync get/set reused a client and lock bound to a
closed loop and failed. Track the loop the client was created on and drop the
stale client/lock when _get_client runs on a different loop. Adds a regression
test that drives two asyncio.run() calls.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 1882a85 to a07f6d0 Compare September 29, 2026 19:36

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/utilities/cache_config.py Outdated
- ValkeyCache._get_client dropped the stale GlideClient on an event-loop change
  without closing it, leaking a connection and reader task per asyncio.run()
  cycle (UploadCache sync path). Best-effort close the old client before
  rebinding.
- get_aiocache_config never forwarded use_tls to aiocache.RedisCache, so a
  rediss:// / valkeys:// REDIS_URL opened a plaintext connection. Pass ssl=True
  when the URL scheme is TLS.
Adds regression tests for both.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from a07f6d0 to 85d2e45 Compare September 29, 2026 20:01
Reorder the aiocache import in agent_card.py per isort and remove an unused
'# noqa: BLE001' directive in valkey_cache.py (BLE001 is not enabled in this
project's ruff config, so the directive was flagged as unused by RUF100). No
behavior change.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 85d2e45 to e4f4487 Compare September 30, 2026 13:04
- parse_cache_url now reads the URL username; ValkeyCache, ValkeyCacheBackend,
  the a2a task cache, and get_aiocache_config forward it, and ValkeyCache builds
  ServerCredentials with username+password. ACL users in VALKEY_URL/REDIS_URL
  (common on managed Valkey) previously failed auth because only the password
  was used.
- UploadCache computed ttl = max(0, ...), so an already-expired or sub-second
  expires_at became ttl=0, which ValkeyCache.set treats as never-expire — the
  key then lived forever. Floor an explicit expiry to 1s so it expires promptly.
Adds regression tests.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from e4f4487 to 869ea69 Compare September 30, 2026 13:48

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 869ea69. Configure here.

Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py
MatthiasHowellYopp and others added 3 commits September 30, 2026 10:24
- get_aiocache_config: pass an ACL username via connection_pool_kwargs so it
  reaches the redis ConnectionPool. Pinned aiocache 0.12.3 forwards unknown
  top-level keys to BaseCache.__init__, which raised TypeError on 'username'
  and broke the A2A REDIS_URL cancel path.
- parse_cache_url: normalize an empty username ('' from redis://:pw@host) to
  None so GLIDE ServerCredentials does password-only auth instead of
  authenticating as a blank ACL user; also normalized at the credential build.
- _get_client: guard the loop-change rebind and lazy lock creation with a
  threading.Lock so concurrent callers on a new loop can't each reset the
  asyncio lock and create duplicate clients.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
Add None/empty input handling to RecallMemoryTool and RememberTool.
Filter empty strings, convert string inputs to lists, and return
descriptive error messages. Update tool descriptions in en.json to
include explicit parameter examples.

Part 2/4 of Valkey storage implementation.
Drop min_length=1 and widen queries/contents to list[str] | str | None on
RecallMemorySchema/RememberSchema. min_length=1 rejected empty lists at schema
validation, raising a Pydantic error before _run could return its guidance
string — so ToolUsage retried the same bad payload. The schema now mirrors what
_run accepts (empty lists, bare strings), letting _run own the validation and
return a useful message. Adds schema-level tests.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-2-memory-tools-hardening branch from 869ea69 to fc188ce Compare September 30, 2026 14:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant