Repository navigation
Conversation
`zarr.storage` swaps its class for a `VerboseModule` subclass to warn when `default_compressor` is set. Serializers that dispatch on the exact type, such as cloudpickle (Dask's default), had no reduce for the subclass and raised `TypeError: cannot pickle 'VerboseModule' object`. Give `VerboseModule` a `__reduce__` that re-imports the module by name, so `pickle` and `cloudpickle` both round-trip it. The deprecation warning is unchanged; removing it is left for a minor release. Refs zarr-developers#4029. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
marked this pull request as ready for review
October 22, 2026 20:47
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4557 +/- ##
=======================================
Coverage 94.71% 94.71%
=======================================
Files 94 94
Lines 13663 13667 +4
=======================================
+ Hits 12941 12945 +4
Misses 722 722
🚀 New features to boost your workflow:
|
Closed
1 of 7 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR partially reverts the changes in #4505 and solves the underlying bug #4505 aimed to solve more directly by allowing the
VerboseModuleto be pickled (by defining__reduce__).edit: I thought #4505 had been merged, but it hasn't. so this PR just closes #4029 directly without the removal component, which is slated for the 3.5.0 release
🤖 AI text below 🤖
The
zarr.storagemodule can now be serialized withpickleandcloudpickle. Previously, code that referenced the module (for example a Dask task that usedzarr.storage.ObjectStore) failed withTypeError: cannot pickle 'VerboseModule' object.Closes #4029.
zarr.storageswaps its class for aVerboseModulesubclass ofModuleTypeso it can warn whendefault_compressoris set. Serializers that dispatch on the exact type, such ascloudpickle(Dask's default), have no reduce for that subclass and fall through toobject.__reduce_ex__, which raises. This PR givesVerboseModulea__reduce__that re-imports the module by name, so bothpickleandcloudpickleround-trip it to the same module object.The deprecation warning for
zarr.storage.default_compressoris unchanged, which keeps this a pure bugfix suitable for a patch release. #4505 fixes the same bug by removingVerboseModuleand the warning altogether. The versioning policy indocs/contributing.mdexcludes behavior changes from patch releases, so that removal fits a minor release, and it can land there on top of this fix.Testing:
test_storage_module_pickles_by_nameintests/test_store/test_core.pyround-tripszarr.storagethrough the standard library'spickle, so it needs no new test dependency. It fails onmainwith theTypeErrorabove.cloudpickleround-tripszarr.storagewith this change, checked in a throwaway environment.tests/test_store/test_core.py,tests/test_v2.py, andtests/test_api.pypass, including the existingdefault_compressordeprecation-warning test. Ruff, mypy, and the other pre-commit hooks pass.🤖 Generated with Claude Code