Skip to content

feat(valkey): shared cache config + ValkeyCache for A2A and file uploads - #5700

Open
MatthiasHowellYopp wants to merge 12 commits into
crewAIInc:mainfrom
MatthiasHowellYopp:feat/valkey-1-cache-config
Open

MatthiasHowellYopp wants to merge 12 commits into
crewAIInc:mainfrom
MatthiasHowellYopp:feat/valkey-1-cache-config

Conversation

@MatthiasHowellYopp

@MatthiasHowellYopp MatthiasHowellYopp commented May 4, 2026 •

Copy link
Copy Markdown
Contributor

Description:

Part 1/4 of adding Valkey as a storage backend for CrewAI. This PR lays the caching foundation that the vector storage implementation will build on.

What changed:

  • cache_config.py (new) — Three small helpers: parse_cache_url() reads VALKEY_URL or REDIS_URL from the environment, get_aiocache_config() builds an aiocache config dict from it, and use_valkey_cache() returns whether the Valkey path is active. Replaces the inline _parse_redis_url() that lived in task.py.

  • valkey_cache.py (new) — A thin async get/set/delete/exists wrapper around valkey-glide. JSON serialization, optional TTL, lazy client initialization. This is the simple key/value cache; vector storage comes in part 4.

  • task.py — Replaced inline URL parsing and module-level caches.set_config() with the shared cache_config helpers. Task cancellation polling now uses ValkeyCache when VALKEY_URL is set, falling back to aiocache otherwise.

  • agent_card.py — Added lazy aiocache configuration via get_aiocache_config() so the cache backend is configured before first use rather than at import time.

  • upload_cache.py — Refactored to optionally use ValkeyCache as its backend when VALKEY_URL is set, with JSON-based serialization instead of pickle.

  • pyproject.toml — Added valkey-glide>=1.3.0 as an optional [valkey] extra.

  • Testing:

test_cache_config.py (11 tests) — URL parsing, priority, defaults, aiocache config generation
test_valkey_cache.py (27 tests) — get/set/delete/exists, TTL, JSON serialization, connection management, edge cases
Existing test_agent_card.py, test_task.py, and test_upload_cache.py cover the refactored paths

Summary by CodeRabbit

  • New Features

    • Optional Valkey cache integration and a backend abstraction for interchangeable cache storage.
    • Unified cache configuration with lazy initialization and consistent expiry/eviction behavior across backends.
    • Task cancellation now supports Valkey-backed cancellation flags with resilient fallback behavior.
  • Tests

    • Added comprehensive tests for Valkey cache (CRUD, TTL, JSON serialization, lifecycle, edge cases) and cache configuration.

Review Change Stack


Note

Medium Risk
Changes distributed A2A cancellation and upload caching when Valkey is enabled; misconfiguration or missing crewai[valkey] can break cancellable tasks, though defaults remain in-memory.

Overview
Adds optional Valkey (valkey-glide) as a shared distributed cache layer, driven by VALKEY_URL (with REDIS_URL still used for aiocache/Redis wiring).

New pieces: cache_config.py centralizes URL parsing (TLS, ACL username, DB) and aiocache setup; ValkeyCache is a JSON get/set/delete wrapper with lazy GLIDE clients and event-loop rebinding for sync asyncio.run() callers.

Wiring: A2A task cancellation uses ValkeyCache when VALKEY_URL is set (polling until pub/sub exists); otherwise behavior stays on aiocache/Redis pub/sub. UploadCache gains a CacheBackend abstraction (memory / redis / valkey), JSON-serialized CachedUpload for Valkey, a ≥1s TTL floor for past expires_at, and per-key clear instead of namespace-wide clear. Agent-card fetch caching is pinned to in-process SimpleMemoryCache so pickle is never read from operator-controlled Redis/Valkey.

Packaging: optional crewai[valkey] extra (valkey-glide>=2.5.2 for client_info_tag). Tests cover config parsing, ValkeyCache, lazy crewai imports in crewai-files, and clearer errors when Valkey is configured but not installed.

Reviewed by Cursor Bugbot for commit 5f38d18. Bugbot is set up for automated code reviews on this repo. Configure here.


Valkey library-name tag

Sets the Valkey client's library-name tag to crewai (via valkey-glide client_info_tag), so the connection reports lib-name=GlidePy(crewai) in CLIENT INFO. This lets operators attribute Valkey usage to CrewAI. Metadata only — no behavioural change. Requires valkey-glide>=2.5.2, where client_info_tag was introduced.

client_name is unchanged; CLIENT SETNAME and CLIENT SETINFO LIB-NAME are independent fields.

@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch 3 times, most recently from 79047cb to c459701 Compare May 7, 2026 19:58
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch from c459701 to a4ba47b Compare May 11, 2026 19:53
@coderabbitai

coderabbitai Bot commented May 11, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Adds Valkey-GLIDE support and a CacheBackend abstraction, centralizes cache config, implements ValkeyCache, serializes CachedUpload, refactors UploadCache and task cancellation to use pluggable backends, updates agent cache init, and adds tests and an optional valkey dependency.

Changes

Valkey Cache Integration

Layer / File(s) Summary
Shared Cache Configuration
lib/crewai/src/crewai/utilities/cache_config.py
Exports parse_cache_url(), get_aiocache_config(), and use_valkey_cache() to centralize environment-based cache selection and aiocache config.
ValkeyCache Implementation
lib/crewai/src/crewai/memory/storage/valkey_cache.py
New async ValkeyCache with lazy Glide client creation, JSON (de)serialization, TTL support, delete/exists/close APIs, and connection error handling.
CachedUpload Serialization
lib/crewai-files/src/crewai_files/cache/upload_cache.py (lines 55-97)
Adds CachedUpload.to_dict() and CachedUpload.from_dict() to enable JSON-compatible persistence of cached entries and preserves the streaming hash helper.
Upload Cache Backend Abstraction
lib/crewai-files/src/crewai_files/cache/upload_cache.py (lines 107-236)
Introduces CacheBackend Protocol, AiocacheBackend, and ValkeyCacheBackend; UploadCache creates _backend via _create_backend(cache_type) (adds "valkey" option).
Upload Cache Async Operations
lib/crewai-files/src/crewai_files/cache/upload_cache.py (lines 262-478)
All async CRUD/eviction/clear methods (aget*, aset*, aremove*, aclear_expired, aclear, aget_all_for_provider) route through _backend and use CachedUpload.is_expired() for expiry handling and key untracking.
A2A Task Cancellation Refactor
lib/crewai/src/crewai/a2a/utils/task.py
Adds thread-safe lazy initialization for a module-level task cache (ValkeyCache when enabled); cancellation watching branches on backend (Valkey polling vs aiocache pub/sub or polling) and cleanup deletes cancel flags via the active backend.
Agent Card Lazy Configuration
lib/crewai/src/crewai/a2a/utils/agent_card.py
Adds _ensure_cache_configured() to lazily call caches.set_config(get_aiocache_config()) once before aiocache use in cached agent card fetching.
Optional Dependency
lib/crewai/pyproject.toml
Adds valkey optional dependency group with valkey-glide>=2.0.0,<3.
Tests
lib/crewai/tests/utilities/test_cache_config.py, lib/crewai/tests/memory/storage/test_valkey_cache.py
Adds tests for cache URL parsing, aiocache config selection, and comprehensive ValkeyCache behavior (CRUD, TTL, serialization, lifecycle, edge cases).

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 74.51% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description explains the implementation, affected files, testing scope, and additional context. However, it omits the required Related issue section and linked issue reference, and it does not inc… Add the required Related issue section with an open issue reference, such as "Fixes #123". Add the Verification section with the required checklist and mark the applicable tests and quality checks based on actual results.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: shared Valkey cache configuration and ValkeyCache support for A2A and file uploads.
Full details: Description check

Explanation

The description explains the implementation, affected files, testing scope, and additional context. However, it omits the required Related issue section and linked issue reference, and it does not include the required verification checklist with explicit local test and quality-check results.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@MatthiasHowellYopp

Copy link
Copy Markdown
Contributor Author

@greysonlalonde Hi, hoping someone can look at the 4 valkey PR's this week, tks.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
lib/crewai-files/src/crewai_files/cache/upload_cache.py (2)

156-160: 💤 Low value

Use lazy % formatting for logging.

Minor: Consider using lazy formatting for the warning log to avoid string interpolation when the log level is disabled.

♻️ Suggested change
         try:
             return CachedUpload.from_dict(data)
         except (KeyError, ValueError) as e:
-            logger.warning(f"Failed to deserialize cached upload: {e}")
+            logger.warning("Failed to deserialize cached upload: %s", e)
             return None
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py` around lines 156 -
160, The logger.warning call in upload_cache.py uses f-string interpolation
(logger.warning(f"Failed to deserialize cached upload: {e}")) which eagerly
formats the message; change it to use lazy logging formatting
(logger.warning("Failed to deserialize cached upload: %s", e)) in the except
block that handles KeyError/ValueError around CachedUpload.from_dict so the
string interpolation is deferred when the warning level is disabled.

238-253: ⚖️ Poor tradeoff

Provider key tracking is instance-local, limiting distributed cleanup.

The _provider_keys dictionary tracks keys locally within each UploadCache instance. In a distributed Valkey setup with multiple application instances, each instance only knows about keys it created. This means aclear(), aclear_expired(), and aget_all_for_provider() will only operate on keys known to that specific instance.

This is an inherent limitation of the current design. Consider documenting this behavior or, for future enhancement, implementing key discovery via Valkey's SCAN command with the namespace prefix.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py` around lines 238 -
253, The provider-key tracking via _provider_keys (managed by _track_key and
_untrack_key) is instance-local, so aclear, aclear_expired and
aget_all_for_provider only affect keys known to that UploadCache instance;
update the UploadCache class and the docstrings for _track_key, _untrack_key,
aclear, aclear_expired, and aget_all_for_provider to explicitly document this
limitation and mention a future enhancement: using Valkey's SCAN with the
namespace prefix to discover keys across distributed instances for global
cleanup.
lib/crewai/src/crewai/memory/storage/valkey_cache.py (1)

126-130: 💤 Low value

Use lazy % formatting for logging.

The f-string in the warning log is evaluated even if the log level is disabled. Consider using lazy formatting for better performance.

♻️ Suggested change
         try:
             return json.loads(value)
         except json.JSONDecodeError:
-            _logger.warning(f"Failed to deserialize cached value for key: {key}")
+            _logger.warning("Failed to deserialize cached value for key: %s", key)
             return None
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py` around lines 126 - 130,
The warning log in valkey_cache.py currently uses an f-string which is evaluated
eagerly; update the except block that handles json.JSONDecodeError (the one
returning None after attempting json.loads(value)) to use lazy logging with
_logger.warning("Failed to deserialize cached value for key: %s", key) so the
interpolated key is only formatted when the warning level is enabled; leave the
rest of the try/except logic unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 145-160: json.dumps(value) can raise TypeError for
non-JSON-serializable objects; wrap the serialization in a try/except around the
json.dumps(value) call (inside the method using _get_client, ttl_to_use and
_default_ttl) and handle failures by either (a) falling back to a safe
serializer like json.dumps(value, default=str) or (b) raising a clearer error
including context (e.g., key and value type) so callers don't get an unhelpful
stacktrace; ensure the rest of the logic still uses the serialized variable for
client.set and that the TTL logic (ttl_to_use) remains unchanged.

---

Nitpick comments:
In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py`:
- Around line 156-160: The logger.warning call in upload_cache.py uses f-string
interpolation (logger.warning(f"Failed to deserialize cached upload: {e}"))
which eagerly formats the message; change it to use lazy logging formatting
(logger.warning("Failed to deserialize cached upload: %s", e)) in the except
block that handles KeyError/ValueError around CachedUpload.from_dict so the
string interpolation is deferred when the warning level is disabled.
- Around line 238-253: The provider-key tracking via _provider_keys (managed by
_track_key and _untrack_key) is instance-local, so aclear, aclear_expired and
aget_all_for_provider only affect keys known to that UploadCache instance;
update the UploadCache class and the docstrings for _track_key, _untrack_key,
aclear, aclear_expired, and aget_all_for_provider to explicitly document this
limitation and mention a future enhancement: using Valkey's SCAN with the
namespace prefix to discover keys across distributed instances for global
cleanup.

In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 126-130: The warning log in valkey_cache.py currently uses an
f-string which is evaluated eagerly; update the except block that handles
json.JSONDecodeError (the one returning None after attempting json.loads(value))
to use lazy logging with _logger.warning("Failed to deserialize cached value for
key: %s", key) so the interpolated key is only formatted when the warning level
is enabled; leave the rest of the try/except logic unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 217f16f8-f68f-4fc9-a12b-b58fc31423ac

📥 Commits

Reviewing files that changed from the base of the PR and between 63a9e7e and a4ba47b.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • lib/crewai-files/src/crewai_files/cache/upload_cache.py
  • lib/crewai/pyproject.toml
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/src/crewai/a2a/utils/task.py
  • lib/crewai/src/crewai/memory/storage/valkey_cache.py
  • lib/crewai/src/crewai/utilities/cache_config.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py
  • lib/crewai/tests/utilities/test_cache_config.py

Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch from a4ba47b to d745a45 Compare May 11, 2026 20:15
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch 2 times, most recently from 540e9ba to fba2c17 Compare May 13, 2026 14:28

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/crewai/src/crewai/memory/storage/valkey_cache.py (1)

1-199: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix formatting to unblock CI.

uv run ruff format --check lib/ is failing in pipeline; please run formatter on this file (and any other changed files) before merge.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py` around lines 1 - 199,
CI is failing due to formatting issues; run the project's formatter (ruff format
as used in CI) on this file and any other modified files to satisfy "uv run ruff
format --check lib/". Reformat the module containing the ValkeyCache class (and
its methods _get_client, get, set, delete, exists, close) so import ordering,
spacing, and line breaks conform to the formatter, then re-run the ruff format
check and commit the formatted changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 159-169: The current logic in valkey_cache.py computes ttl_to_use
from ttl or self._default_ttl and treats negative values (e.g., ttl == -1) as
"no-expiration", which can hide caller bugs; update the code around ttl_to_use
to explicitly reject negative TTLs by raising a ValueError (or similar) when
ttl_to_use < 0, and only call client.set(..., expiry=ExpirySet(ExpiryType.SEC,
ttl_to_use)) when ttl_to_use > 0 otherwise call client.set(key, serialized) for
None/zero; adjust references: ttl_to_use, _default_ttl, client.set, ExpirySet,
ExpiryType.SEC.
- Around line 57-109: Concurrent callers can race creating multiple GlideClient
instances because _get_client checks self._client without synchronization; add
an asyncio.Lock (e.g., self._client_lock created in __init__) and wrap the lazy
initialization in _get_client with "async with self._client_lock:" then recheck
"if self._client is None" inside the lock before calling GlideClient.create (and
keep the existing timeout/error handling), so only one coroutine performs
initialization and others await the same client.

---

Outside diff comments:
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 1-199: CI is failing due to formatting issues; run the project's
formatter (ruff format as used in CI) on this file and any other modified files
to satisfy "uv run ruff format --check lib/". Reformat the module containing the
ValkeyCache class (and its methods _get_client, get, set, delete, exists, close)
so import ordering, spacing, and line breaks conform to the formatter, then
re-run the ruff format check and commit the formatted changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1f6f8c3c-37fa-4aa1-835e-7a4d2eab1b56

📥 Commits

Reviewing files that changed from the base of the PR and between d745a45 and 540e9ba.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • lib/crewai-files/src/crewai_files/cache/upload_cache.py
  • lib/crewai/pyproject.toml
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/src/crewai/a2a/utils/task.py
  • lib/crewai/src/crewai/memory/storage/valkey_cache.py
  • lib/crewai/src/crewai/utilities/cache_config.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py
  • lib/crewai/tests/utilities/test_cache_config.py
🚧 Files skipped from review as they are similar to previous changes (7)
  • lib/crewai/pyproject.toml
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/tests/utilities/test_cache_config.py
  • lib/crewai/src/crewai/utilities/cache_config.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py
  • lib/crewai-files/src/crewai_files/cache/upload_cache.py
  • lib/crewai/src/crewai/a2a/utils/task.py

Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py
Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/pyproject.toml`:
- Around line 113-115: The dependency entry for valkey currently uses an
open-ended constraint ("valkey-glide>=1.3.0"); update the valkey list to pin to
the maintained v2.x series by changing the requirement to a bounded range such
as "valkey-glide>=2.0.0,<3.0.0" so future v3.x breaking changes are avoided
(edit the valkey array entry in pyproject.toml and replace the existing
"valkey-glide>=1.3.0" line).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1dba85f4-bcbc-4fd7-84f0-7cb1b19bd2ee

📥 Commits

Reviewing files that changed from the base of the PR and between 540e9ba and fba2c17.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • lib/crewai-files/src/crewai_files/cache/upload_cache.py
  • lib/crewai/pyproject.toml
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/src/crewai/a2a/utils/task.py
  • lib/crewai/src/crewai/memory/storage/valkey_cache.py
  • lib/crewai/src/crewai/utilities/cache_config.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py
  • lib/crewai/tests/utilities/test_cache_config.py
🚧 Files skipped from review as they are similar to previous changes (7)
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py
  • lib/crewai/src/crewai/a2a/utils/task.py
  • lib/crewai/tests/utilities/test_cache_config.py
  • lib/crewai/src/crewai/memory/storage/valkey_cache.py
  • lib/crewai-files/src/crewai_files/cache/upload_cache.py
  • lib/crewai/src/crewai/utilities/cache_config.py

Comment thread lib/crewai/pyproject.toml
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch from fba2c17 to 81f403b Compare May 13, 2026 14:45
@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py`:
- Around line 165-167: The delete wrapper always returns True which incorrectly
signals success for non-existent keys; change upload_cache.delete to detect
whether the key actually existed before/after deletion and return that boolean.
Specifically, in delete(self, key: str) call the underlying cache in a way that
determines existence (e.g. check await self._cache.exists(key) or await
self._cache.contains(key) before deleting, or use the boolean return value of
await self._cache.delete(key) if present) and return that real result instead of
unconditionally True; update the code in upload_cache.delete to handle both
ValkeyCache (void delete) and caches that return bool by using existence check
when delete returns None.
- Around line 208-224: The Valkey branch in _create_backend drops the namespace,
causing cross-namespace key collisions; update the ValkeyCacheBackend
construction in _create_backend to pass the namespace through (e.g., include
namespace=namespace or the appropriate key_prefix/namespace parameter accepted
by ValkeyCacheBackend) alongside host/port/db/password/default_ttl so each
UploadCache instance is isolated by namespace.

In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 163-174: In the set() method of valkey_cache.py, validate ttl and
JSON-serialize the value before calling self._get_client() so invalid input
raises the documented TypeError/ValueError without opening a Valkey connection;
specifically, move the json.dumps(value) try/except block and the ttl_to_use/ttl
< 0 check to run prior to the call to client = await self._get_client(),
preserving the same exception types and messages raised by the existing
TypeError/ValueError paths.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 01a6c7d7-b80b-4e7a-a7d1-a92d539f95b0

📥 Commits

Reviewing files that changed from the base of the PR and between fba2c17 and 81f403b.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • lib/crewai-files/src/crewai_files/cache/upload_cache.py
  • lib/crewai/pyproject.toml
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/src/crewai/a2a/utils/task.py
  • lib/crewai/src/crewai/memory/storage/valkey_cache.py
  • lib/crewai/src/crewai/utilities/cache_config.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py
  • lib/crewai/tests/utilities/test_cache_config.py
🚧 Files skipped from review as they are similar to previous changes (5)
  • lib/crewai/tests/utilities/test_cache_config.py
  • lib/crewai/src/crewai/utilities/cache_config.py
  • lib/crewai/src/crewai/a2a/utils/agent_card.py
  • lib/crewai/src/crewai/a2a/utils/task.py
  • lib/crewai/tests/memory/storage/test_valkey_cache.py

Comment thread lib/crewai-files/src/crewai_files/cache/upload_cache.py Outdated
Comment thread lib/crewai-files/src/crewai_files/cache/upload_cache.py
Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py Outdated
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch from 81f403b to c4804a1 Compare May 13, 2026 15:14
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch 2 times, most recently from 0cbd87a to 8901835 Compare May 19, 2026 18:19
@MatthiasHowellYopp

Copy link
Copy Markdown
Contributor Author

@greysonlalonde hoping I can get a review on this.

@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch from 8901835 to 1a2bdf0 Compare May 20, 2026 13:32
@greysonlalonde

Copy link
Copy Markdown
Contributor

@greysonlalonde hoping I can get a review on this.

Reply appreciate the patience here, thanks! Will start getting to these today

@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch 6 times, most recently from 752be6f to 20f7ee0 Compare May 25, 2026 13:23
Comment thread lib/crewai/src/crewai/a2a/utils/agent_card.py Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai-files/src/crewai_files/cache/upload_cache.py Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/utilities/cache_config.py

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/a2a/utils/task.py
Comment thread lib/crewai/src/crewai/a2a/utils/task.py

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/utilities/cache_config.py Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/memory/storage/valkey_cache.py
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch from 8e241e6 to 75b35ae Compare October 9, 2026 18:46

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 75b35ae. Configure here.

Comment thread lib/crewai/src/crewai/a2a/utils/task.py
@MatthiasHowellYopp
MatthiasHowellYopp force-pushed the feat/valkey-1-cache-config branch 2 times, most recently from 5b7a761 to 5f38d18 Compare October 14, 2026 18:58
Matthias Howell and others added 12 commits October 15, 2026 10:25
Extract duplicated Redis URL parsing into a shared cache_config utility.
Introduce ValkeyCache as a lightweight async key/value cache using
valkey-glide. Wire it into A2A task handling, agent card caching, and
file upload caching.

Part 1/4 of Valkey storage implementation.
Sets CLIENT SETNAME to 'crewai_valkey' so connections are identifiable
in CLIENT LIST and monitoring tools (Valkey Admin, CloudWatch).

Signed-off-by: Matthias Howell <matthias.howell@improving.com>
Set client_info_tag="crewai" on the Glide client so the connection reports
lib-name GlidePy(crewai) via CLIENT INFO, letting operators attribute Valkey
usage to CrewAI. Bumps the valkey-glide floor to >=2.5.2, where client_info_tag
was introduced. client_name (CLIENT SETNAME) is unchanged; the two are
independent fields. Metadata only, no behavioural change.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- crewai-files no longer hard-imports crewai at module load. parse_cache_url
  moves to a lazy import inside the Valkey backend path, so importing
  crewai_files without crewai installed works again (the default in-memory
  cache never needs it). Preserves the crewai[file-processing] -> crewai-files
  direction. Adds regression tests.
- agent_card: pin the AgentCard PickleSerializer @cached to an in-process
  SimpleMemoryCache. Previously it used the default alias, which A2A wires to
  VALKEY_URL/REDIS_URL — pickling cache values to a network Redis/Valkey is a
  code-execution surface on cache read if that store is writable. Pickle now
  never deserializes from an operator-controlled backend.
- ValkeyCache/ValkeyCacheBackend/parse_cache_url now carry use_tls (from
  rediss/valkeys schemes) so cache connections honor TLS instead of silently
  opening a plaintext connection to a managed endpoint.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
The agent-card pickle cache is pinned to its own SimpleMemoryCache via the
@cached decorator, so it no longer needs the process-wide aiocache default
alias configured. The lazy _ensure_cache_configured() helper still called
caches.set_config(get_aiocache_config()) on every fetch, which replaced the
default alias that the in-memory A2A cancel path (poll_for_cancel_aiocache ->
caches.get('default')) depends on. A running poller and a later cancel() could
then hold different SimpleMemoryCache instances and never see the cancel flag.

Remove the helper, its call, and the now-unused imports. Add regression tests
asserting the helper is gone and the default alias is stable.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
_ensure_task_cache built ValkeyCache without use_tls, so a VALKEY_URL with a
TLS scheme (valkeys:// or rediss://) opened a plaintext connection and A2A
cancel/cancel-watch failed against a TLS-only Valkey. parse_cache_url already
extracts use_tls from the scheme; thread it through. Lives on the base branch
where the A2A task cache is introduced so it applies to the whole stack.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
Setting VALKEY_URL routes A2A task cancellation through ValkeyCache, which
imports glide at module load. Without the optional valkey extra installed, that
raised a bare ImportError from every cancellable A2A task. Catch it in
_ensure_task_cache and re-raise with an actionable message pointing at
'pip install crewai[valkey]' (or unsetting VALKEY_URL).

This is the fail-loud option. A graceful fallback to the aiocache Redis path is
deliberately left for later, pending the broader third-party provider/config
decision. Adds a regression test.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
ValkeyCache cached a single GlideClient and asyncio.Lock on the instance.
UploadCache's sync methods each call asyncio.run(), which creates then closes a
fresh loop, so the second sync get/set reused a client and lock bound to a
closed loop and failed. Track the loop the client was created on and drop the
stale client/lock when _get_client runs on a different loop. Adds a regression
test that drives two asyncio.run() calls.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- ValkeyCache._get_client dropped the stale GlideClient on an event-loop change
  without closing it, leaking a connection and reader task per asyncio.run()
  cycle (UploadCache sync path). Best-effort close the old client before
  rebinding.
- get_aiocache_config never forwarded use_tls to aiocache.RedisCache, so a
  rediss:// / valkeys:// REDIS_URL opened a plaintext connection. Pass ssl=True
  when the URL scheme is TLS.
Adds regression tests for both.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
Reorder the aiocache import in agent_card.py per isort and remove an unused
'# noqa: BLE001' directive in valkey_cache.py (BLE001 is not enabled in this
project's ruff config, so the directive was flagged as unused by RUF100). No
behavior change.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- parse_cache_url now reads the URL username; ValkeyCache, ValkeyCacheBackend,
  the a2a task cache, and get_aiocache_config forward it, and ValkeyCache builds
  ServerCredentials with username+password. ACL users in VALKEY_URL/REDIS_URL
  (common on managed Valkey) previously failed auth because only the password
  was used.
- UploadCache computed ttl = max(0, ...), so an already-expired or sub-second
  expires_at became ttl=0, which ValkeyCache.set treats as never-expire — the
  key then lived forever. Floor an explicit expiry to 1s so it expires promptly.
Adds regression tests.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- get_aiocache_config: pass an ACL username via connection_pool_kwargs so it
  reaches the redis ConnectionPool. Pinned aiocache 0.12.3 forwards unknown
  top-level keys to BaseCache.__init__, which raised TypeError on 'username'
  and broke the A2A REDIS_URL cancel path.
- parse_cache_url: normalize an empty username ('' from redis://:pw@host) to
  None so GLIDE ServerCredentials does password-only auth instead of
  authenticating as a blank ACL user; also normalized at the credential build.
- _get_client: guard the loop-change rebind and lazy lock creation with a
  threading.Lock so concurrent callers on a new loop can't each reset the
  asyncio lock and create duplicate clients.

Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants