Repository navigation
feat(valkey): shared cache config + ValkeyCache for A2A and file uploads - #5700
MatthiasHowellYopp wants to merge 12 commits into
Conversation
79047cb to
c459701
Compare
c459701 to
a4ba47b
Compare
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 Walkthrough
🚥 Pre-merge checks | ✅ 3 | ❌ 2
✨ Finishing Touches
Warning Review ran into problems🔥 Problems
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@greysonlalonde Hi, hoping someone can look at the 4 valkey PR's this week, tks. |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
lib/crewai-files/src/crewai_files/cache/upload_cache.py (2)
156-160: 💤 Low valueUse lazy % formatting for logging.
Minor: Consider using lazy formatting for the warning log to avoid string interpolation when the log level is disabled.
♻️ Suggested change
try: return CachedUpload.from_dict(data) except (KeyError, ValueError) as e: - logger.warning(f"Failed to deserialize cached upload: {e}") + logger.warning("Failed to deserialize cached upload: %s", e) return None🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py` around lines 156 - 160, The logger.warning call in upload_cache.py uses f-string interpolation (logger.warning(f"Failed to deserialize cached upload: {e}")) which eagerly formats the message; change it to use lazy logging formatting (logger.warning("Failed to deserialize cached upload: %s", e)) in the except block that handles KeyError/ValueError around CachedUpload.from_dict so the string interpolation is deferred when the warning level is disabled.
238-253: ⚖️ Poor tradeoffProvider key tracking is instance-local, limiting distributed cleanup.
The
_provider_keysdictionary tracks keys locally within eachUploadCacheinstance. In a distributed Valkey setup with multiple application instances, each instance only knows about keys it created. This meansaclear(),aclear_expired(), andaget_all_for_provider()will only operate on keys known to that specific instance.This is an inherent limitation of the current design. Consider documenting this behavior or, for future enhancement, implementing key discovery via Valkey's
SCANcommand with the namespace prefix.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py` around lines 238 - 253, The provider-key tracking via _provider_keys (managed by _track_key and _untrack_key) is instance-local, so aclear, aclear_expired and aget_all_for_provider only affect keys known to that UploadCache instance; update the UploadCache class and the docstrings for _track_key, _untrack_key, aclear, aclear_expired, and aget_all_for_provider to explicitly document this limitation and mention a future enhancement: using Valkey's SCAN with the namespace prefix to discover keys across distributed instances for global cleanup.lib/crewai/src/crewai/memory/storage/valkey_cache.py (1)
126-130: 💤 Low valueUse lazy % formatting for logging.
The f-string in the warning log is evaluated even if the log level is disabled. Consider using lazy formatting for better performance.
♻️ Suggested change
try: return json.loads(value) except json.JSONDecodeError: - _logger.warning(f"Failed to deserialize cached value for key: {key}") + _logger.warning("Failed to deserialize cached value for key: %s", key) return None🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py` around lines 126 - 130, The warning log in valkey_cache.py currently uses an f-string which is evaluated eagerly; update the except block that handles json.JSONDecodeError (the one returning None after attempting json.loads(value)) to use lazy logging with _logger.warning("Failed to deserialize cached value for key: %s", key) so the interpolated key is only formatted when the warning level is enabled; leave the rest of the try/except logic unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 145-160: json.dumps(value) can raise TypeError for
non-JSON-serializable objects; wrap the serialization in a try/except around the
json.dumps(value) call (inside the method using _get_client, ttl_to_use and
_default_ttl) and handle failures by either (a) falling back to a safe
serializer like json.dumps(value, default=str) or (b) raising a clearer error
including context (e.g., key and value type) so callers don't get an unhelpful
stacktrace; ensure the rest of the logic still uses the serialized variable for
client.set and that the TTL logic (ttl_to_use) remains unchanged.
---
Nitpick comments:
In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py`:
- Around line 156-160: The logger.warning call in upload_cache.py uses f-string
interpolation (logger.warning(f"Failed to deserialize cached upload: {e}"))
which eagerly formats the message; change it to use lazy logging formatting
(logger.warning("Failed to deserialize cached upload: %s", e)) in the except
block that handles KeyError/ValueError around CachedUpload.from_dict so the
string interpolation is deferred when the warning level is disabled.
- Around line 238-253: The provider-key tracking via _provider_keys (managed by
_track_key and _untrack_key) is instance-local, so aclear, aclear_expired and
aget_all_for_provider only affect keys known to that UploadCache instance;
update the UploadCache class and the docstrings for _track_key, _untrack_key,
aclear, aclear_expired, and aget_all_for_provider to explicitly document this
limitation and mention a future enhancement: using Valkey's SCAN with the
namespace prefix to discover keys across distributed instances for global
cleanup.
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 126-130: The warning log in valkey_cache.py currently uses an
f-string which is evaluated eagerly; update the except block that handles
json.JSONDecodeError (the one returning None after attempting json.loads(value))
to use lazy logging with _logger.warning("Failed to deserialize cached value for
key: %s", key) so the interpolated key is only formatted when the warning level
is enabled; leave the rest of the try/except logic unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 217f16f8-f68f-4fc9-a12b-b58fc31423ac
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
lib/crewai-files/src/crewai_files/cache/upload_cache.pylib/crewai/pyproject.tomllib/crewai/src/crewai/a2a/utils/agent_card.pylib/crewai/src/crewai/a2a/utils/task.pylib/crewai/src/crewai/memory/storage/valkey_cache.pylib/crewai/src/crewai/utilities/cache_config.pylib/crewai/tests/memory/storage/test_valkey_cache.pylib/crewai/tests/utilities/test_cache_config.py
a4ba47b to
d745a45
Compare
540e9ba to
fba2c17
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
lib/crewai/src/crewai/memory/storage/valkey_cache.py (1)
1-199:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFix formatting to unblock CI.
uv run ruff format --check lib/is failing in pipeline; please run formatter on this file (and any other changed files) before merge.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py` around lines 1 - 199, CI is failing due to formatting issues; run the project's formatter (ruff format as used in CI) on this file and any other modified files to satisfy "uv run ruff format --check lib/". Reformat the module containing the ValkeyCache class (and its methods _get_client, get, set, delete, exists, close) so import ordering, spacing, and line breaks conform to the formatter, then re-run the ruff format check and commit the formatted changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 159-169: The current logic in valkey_cache.py computes ttl_to_use
from ttl or self._default_ttl and treats negative values (e.g., ttl == -1) as
"no-expiration", which can hide caller bugs; update the code around ttl_to_use
to explicitly reject negative TTLs by raising a ValueError (or similar) when
ttl_to_use < 0, and only call client.set(..., expiry=ExpirySet(ExpiryType.SEC,
ttl_to_use)) when ttl_to_use > 0 otherwise call client.set(key, serialized) for
None/zero; adjust references: ttl_to_use, _default_ttl, client.set, ExpirySet,
ExpiryType.SEC.
- Around line 57-109: Concurrent callers can race creating multiple GlideClient
instances because _get_client checks self._client without synchronization; add
an asyncio.Lock (e.g., self._client_lock created in __init__) and wrap the lazy
initialization in _get_client with "async with self._client_lock:" then recheck
"if self._client is None" inside the lock before calling GlideClient.create (and
keep the existing timeout/error handling), so only one coroutine performs
initialization and others await the same client.
---
Outside diff comments:
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 1-199: CI is failing due to formatting issues; run the project's
formatter (ruff format as used in CI) on this file and any other modified files
to satisfy "uv run ruff format --check lib/". Reformat the module containing the
ValkeyCache class (and its methods _get_client, get, set, delete, exists, close)
so import ordering, spacing, and line breaks conform to the formatter, then
re-run the ruff format check and commit the formatted changes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1f6f8c3c-37fa-4aa1-835e-7a4d2eab1b56
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
lib/crewai-files/src/crewai_files/cache/upload_cache.pylib/crewai/pyproject.tomllib/crewai/src/crewai/a2a/utils/agent_card.pylib/crewai/src/crewai/a2a/utils/task.pylib/crewai/src/crewai/memory/storage/valkey_cache.pylib/crewai/src/crewai/utilities/cache_config.pylib/crewai/tests/memory/storage/test_valkey_cache.pylib/crewai/tests/utilities/test_cache_config.py
🚧 Files skipped from review as they are similar to previous changes (7)
- lib/crewai/pyproject.toml
- lib/crewai/src/crewai/a2a/utils/agent_card.py
- lib/crewai/tests/utilities/test_cache_config.py
- lib/crewai/src/crewai/utilities/cache_config.py
- lib/crewai/tests/memory/storage/test_valkey_cache.py
- lib/crewai-files/src/crewai_files/cache/upload_cache.py
- lib/crewai/src/crewai/a2a/utils/task.py
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai/pyproject.toml`:
- Around line 113-115: The dependency entry for valkey currently uses an
open-ended constraint ("valkey-glide>=1.3.0"); update the valkey list to pin to
the maintained v2.x series by changing the requirement to a bounded range such
as "valkey-glide>=2.0.0,<3.0.0" so future v3.x breaking changes are avoided
(edit the valkey array entry in pyproject.toml and replace the existing
"valkey-glide>=1.3.0" line).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1dba85f4-bcbc-4fd7-84f0-7cb1b19bd2ee
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
lib/crewai-files/src/crewai_files/cache/upload_cache.pylib/crewai/pyproject.tomllib/crewai/src/crewai/a2a/utils/agent_card.pylib/crewai/src/crewai/a2a/utils/task.pylib/crewai/src/crewai/memory/storage/valkey_cache.pylib/crewai/src/crewai/utilities/cache_config.pylib/crewai/tests/memory/storage/test_valkey_cache.pylib/crewai/tests/utilities/test_cache_config.py
🚧 Files skipped from review as they are similar to previous changes (7)
- lib/crewai/src/crewai/a2a/utils/agent_card.py
- lib/crewai/tests/memory/storage/test_valkey_cache.py
- lib/crewai/src/crewai/a2a/utils/task.py
- lib/crewai/tests/utilities/test_cache_config.py
- lib/crewai/src/crewai/memory/storage/valkey_cache.py
- lib/crewai-files/src/crewai_files/cache/upload_cache.py
- lib/crewai/src/crewai/utilities/cache_config.py
fba2c17 to
81f403b
Compare
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai-files/src/crewai_files/cache/upload_cache.py`:
- Around line 165-167: The delete wrapper always returns True which incorrectly
signals success for non-existent keys; change upload_cache.delete to detect
whether the key actually existed before/after deletion and return that boolean.
Specifically, in delete(self, key: str) call the underlying cache in a way that
determines existence (e.g. check await self._cache.exists(key) or await
self._cache.contains(key) before deleting, or use the boolean return value of
await self._cache.delete(key) if present) and return that real result instead of
unconditionally True; update the code in upload_cache.delete to handle both
ValkeyCache (void delete) and caches that return bool by using existence check
when delete returns None.
- Around line 208-224: The Valkey branch in _create_backend drops the namespace,
causing cross-namespace key collisions; update the ValkeyCacheBackend
construction in _create_backend to pass the namespace through (e.g., include
namespace=namespace or the appropriate key_prefix/namespace parameter accepted
by ValkeyCacheBackend) alongside host/port/db/password/default_ttl so each
UploadCache instance is isolated by namespace.
In `@lib/crewai/src/crewai/memory/storage/valkey_cache.py`:
- Around line 163-174: In the set() method of valkey_cache.py, validate ttl and
JSON-serialize the value before calling self._get_client() so invalid input
raises the documented TypeError/ValueError without opening a Valkey connection;
specifically, move the json.dumps(value) try/except block and the ttl_to_use/ttl
< 0 check to run prior to the call to client = await self._get_client(),
preserving the same exception types and messages raised by the existing
TypeError/ValueError paths.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 01a6c7d7-b80b-4e7a-a7d1-a92d539f95b0
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
lib/crewai-files/src/crewai_files/cache/upload_cache.pylib/crewai/pyproject.tomllib/crewai/src/crewai/a2a/utils/agent_card.pylib/crewai/src/crewai/a2a/utils/task.pylib/crewai/src/crewai/memory/storage/valkey_cache.pylib/crewai/src/crewai/utilities/cache_config.pylib/crewai/tests/memory/storage/test_valkey_cache.pylib/crewai/tests/utilities/test_cache_config.py
🚧 Files skipped from review as they are similar to previous changes (5)
- lib/crewai/tests/utilities/test_cache_config.py
- lib/crewai/src/crewai/utilities/cache_config.py
- lib/crewai/src/crewai/a2a/utils/agent_card.py
- lib/crewai/src/crewai/a2a/utils/task.py
- lib/crewai/tests/memory/storage/test_valkey_cache.py
81f403b to
c4804a1
Compare
0cbd87a to
8901835
Compare
|
@greysonlalonde hoping I can get a review on this. |
8901835 to
1a2bdf0
Compare
Reply appreciate the patience here, thanks! Will start getting to these today |
752be6f to
20f7ee0
Compare
7c66013 to
9579385
Compare
9579385 to
854cf0f
Compare
8e241e6 to
75b35ae
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 75b35ae. Configure here.
5b7a761 to
5f38d18
Compare
Extract duplicated Redis URL parsing into a shared cache_config utility. Introduce ValkeyCache as a lightweight async key/value cache using valkey-glide. Wire it into A2A task handling, agent card caching, and file upload caching. Part 1/4 of Valkey storage implementation.
Sets CLIENT SETNAME to 'crewai_valkey' so connections are identifiable in CLIENT LIST and monitoring tools (Valkey Admin, CloudWatch). Signed-off-by: Matthias Howell <matthias.howell@improving.com>
Set client_info_tag="crewai" on the Glide client so the connection reports lib-name GlidePy(crewai) via CLIENT INFO, letting operators attribute Valkey usage to CrewAI. Bumps the valkey-glide floor to >=2.5.2, where client_info_tag was introduced. client_name (CLIENT SETNAME) is unchanged; the two are independent fields. Metadata only, no behavioural change. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- crewai-files no longer hard-imports crewai at module load. parse_cache_url moves to a lazy import inside the Valkey backend path, so importing crewai_files without crewai installed works again (the default in-memory cache never needs it). Preserves the crewai[file-processing] -> crewai-files direction. Adds regression tests. - agent_card: pin the AgentCard PickleSerializer @cached to an in-process SimpleMemoryCache. Previously it used the default alias, which A2A wires to VALKEY_URL/REDIS_URL — pickling cache values to a network Redis/Valkey is a code-execution surface on cache read if that store is writable. Pickle now never deserializes from an operator-controlled backend. - ValkeyCache/ValkeyCacheBackend/parse_cache_url now carry use_tls (from rediss/valkeys schemes) so cache connections honor TLS instead of silently opening a plaintext connection to a managed endpoint. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
The agent-card pickle cache is pinned to its own SimpleMemoryCache via the @cached decorator, so it no longer needs the process-wide aiocache default alias configured. The lazy _ensure_cache_configured() helper still called caches.set_config(get_aiocache_config()) on every fetch, which replaced the default alias that the in-memory A2A cancel path (poll_for_cancel_aiocache -> caches.get('default')) depends on. A running poller and a later cancel() could then hold different SimpleMemoryCache instances and never see the cancel flag. Remove the helper, its call, and the now-unused imports. Add regression tests asserting the helper is gone and the default alias is stable. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
_ensure_task_cache built ValkeyCache without use_tls, so a VALKEY_URL with a TLS scheme (valkeys:// or rediss://) opened a plaintext connection and A2A cancel/cancel-watch failed against a TLS-only Valkey. parse_cache_url already extracts use_tls from the scheme; thread it through. Lives on the base branch where the A2A task cache is introduced so it applies to the whole stack. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
Setting VALKEY_URL routes A2A task cancellation through ValkeyCache, which imports glide at module load. Without the optional valkey extra installed, that raised a bare ImportError from every cancellable A2A task. Catch it in _ensure_task_cache and re-raise with an actionable message pointing at 'pip install crewai[valkey]' (or unsetting VALKEY_URL). This is the fail-loud option. A graceful fallback to the aiocache Redis path is deliberately left for later, pending the broader third-party provider/config decision. Adds a regression test. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
ValkeyCache cached a single GlideClient and asyncio.Lock on the instance. UploadCache's sync methods each call asyncio.run(), which creates then closes a fresh loop, so the second sync get/set reused a client and lock bound to a closed loop and failed. Track the loop the client was created on and drop the stale client/lock when _get_client runs on a different loop. Adds a regression test that drives two asyncio.run() calls. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- ValkeyCache._get_client dropped the stale GlideClient on an event-loop change without closing it, leaking a connection and reader task per asyncio.run() cycle (UploadCache sync path). Best-effort close the old client before rebinding. - get_aiocache_config never forwarded use_tls to aiocache.RedisCache, so a rediss:// / valkeys:// REDIS_URL opened a plaintext connection. Pass ssl=True when the URL scheme is TLS. Adds regression tests for both. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
Reorder the aiocache import in agent_card.py per isort and remove an unused '# noqa: BLE001' directive in valkey_cache.py (BLE001 is not enabled in this project's ruff config, so the directive was flagged as unused by RUF100). No behavior change. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- parse_cache_url now reads the URL username; ValkeyCache, ValkeyCacheBackend, the a2a task cache, and get_aiocache_config forward it, and ValkeyCache builds ServerCredentials with username+password. ACL users in VALKEY_URL/REDIS_URL (common on managed Valkey) previously failed auth because only the password was used. - UploadCache computed ttl = max(0, ...), so an already-expired or sub-second expires_at became ttl=0, which ValkeyCache.set treats as never-expire — the key then lived forever. Floor an explicit expiry to 1s so it expires promptly. Adds regression tests. Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
- get_aiocache_config: pass an ACL username via connection_pool_kwargs so it
reaches the redis ConnectionPool. Pinned aiocache 0.12.3 forwards unknown
top-level keys to BaseCache.__init__, which raised TypeError on 'username'
and broke the A2A REDIS_URL cancel path.
- parse_cache_url: normalize an empty username ('' from redis://:pw@host) to
None so GLIDE ServerCredentials does password-only auth instead of
authenticating as a blank ACL user; also normalized at the credential build.
- _get_client: guard the loop-change rebind and lazy lock creation with a
threading.Lock so concurrent callers on a new loop can't each reset the
asyncio lock and create duplicate clients.
Signed-off-by: MatthiasHowellYopp <matthias.howell@improving.com>
5f38d18 to
8d3831f
Compare

Description:
Part 1/4 of adding Valkey as a storage backend for CrewAI. This PR lays the caching foundation that the vector storage implementation will build on.
What changed:
cache_config.py (new) — Three small helpers: parse_cache_url() reads VALKEY_URL or REDIS_URL from the environment, get_aiocache_config() builds an aiocache config dict from it, and use_valkey_cache() returns whether the Valkey path is active. Replaces the inline _parse_redis_url() that lived in task.py.
valkey_cache.py (new) — A thin async get/set/delete/exists wrapper around valkey-glide. JSON serialization, optional TTL, lazy client initialization. This is the simple key/value cache; vector storage comes in part 4.
task.py — Replaced inline URL parsing and module-level caches.set_config() with the shared cache_config helpers. Task cancellation polling now uses ValkeyCache when VALKEY_URL is set, falling back to aiocache otherwise.
agent_card.py — Added lazy aiocache configuration via get_aiocache_config() so the cache backend is configured before first use rather than at import time.
upload_cache.py — Refactored to optionally use ValkeyCache as its backend when VALKEY_URL is set, with JSON-based serialization instead of pickle.
pyproject.toml — Added valkey-glide>=1.3.0 as an optional [valkey] extra.
Testing:
test_cache_config.py (11 tests) — URL parsing, priority, defaults, aiocache config generation
test_valkey_cache.py (27 tests) — get/set/delete/exists, TTL, JSON serialization, connection management, edge cases
Existing test_agent_card.py, test_task.py, and test_upload_cache.py cover the refactored paths
Summary by CodeRabbit
New Features
Tests
Note
Medium Risk
Changes distributed A2A cancellation and upload caching when Valkey is enabled; misconfiguration or missing
crewai[valkey]can break cancellable tasks, though defaults remain in-memory.Overview
Adds optional Valkey (
valkey-glide) as a shared distributed cache layer, driven byVALKEY_URL(withREDIS_URLstill used for aiocache/Redis wiring).New pieces:
cache_config.pycentralizes URL parsing (TLS, ACL username, DB) and aiocache setup;ValkeyCacheis a JSON get/set/delete wrapper with lazy GLIDE clients and event-loop rebinding for syncasyncio.run()callers.Wiring: A2A task cancellation uses
ValkeyCachewhenVALKEY_URLis set (polling until pub/sub exists); otherwise behavior stays on aiocache/Redis pub/sub.UploadCachegains aCacheBackendabstraction (memory/redis/valkey), JSON-serializedCachedUploadfor Valkey, a ≥1s TTL floor for pastexpires_at, and per-key clear instead of namespace-wide clear. Agent-card fetch caching is pinned to in-processSimpleMemoryCacheso pickle is never read from operator-controlled Redis/Valkey.Packaging: optional
crewai[valkey]extra (valkey-glide>=2.5.2forclient_info_tag). Tests cover config parsing, ValkeyCache, lazy crewai imports in crewai-files, and clearer errors when Valkey is configured but not installed.Reviewed by Cursor Bugbot for commit 5f38d18. Bugbot is set up for automated code reviews on this repo. Configure here.
Valkey library-name tag
Sets the Valkey client's library-name tag to
crewai(via valkey-glideclient_info_tag), so the connection reportslib-name=GlidePy(crewai)inCLIENT INFO. This lets operators attribute Valkey usage to CrewAI. Metadata only — no behavioural change. Requiresvalkey-glide>=2.5.2, whereclient_info_tagwas introduced.client_nameis unchanged;CLIENT SETNAMEandCLIENT SETINFO LIB-NAMEare independent fields.