Repository navigation
deprecate old creation functions #3976
Description
the create function has a few things against it:
- it uses zarr python 2.x parameters, many of which do nothing (
synchronizer,chunk_store,object_codec) - it has an ambiguous name. create what?
- it is functionally superseded by the
create_arrayfunction, which doesn't suffer the previously stated deficiencies
I recommend deprecating create, and deprecating the use of the create-style signature in functions that wrap it. this would entail:
- adding a deprecation warning to the
createfunction, indicating that it will be removed in the next minor release (3.3.0), and directing people to usecreate_arrayinstead. - adding deprecations as needed to any of the functions that use
create(I thinkzeros,empty, etc all use it). Upon release 3.3.0, these functions will all wrapcreate_arrayand use keyword arguments suitable for it.
Activity
[-]deprecate `create`[/-][+]deprecate old creation functions[/+]d-v-b commentedon May 14, 2026
I didn't realize there were so many different creation routines. In addition to
create, I propose deprecating:copycopy_allcopy_storeloadopen(high priority for deprecation because it was VERY unclear semantics)open_consolidatedsavesave_arraysave_grouparraygroupopen_group@zarr-developers/python-core-devs please let me know if you or someone you know is still using any of these functions! I see no urgency for deprecating them, but I also think we are far enough from the 3.0.0 release that we can relax a bit of our 2.x compatibility layer.
maxrjones commentedon May 15, 2026
There are almost certainly users of those functions, given that we were telling v2→v3 migrators to use them for the past 15 months:
Therefore, we should expect these deprecations to cause user-pain. I'd prefer we make this decision in the context of the overall post-v2-migration API roadmap, so we're confident we won't identify another component that's sub-optimal after this deprecation cycle and impact users with another cycle.
d-v-b commentedon May 15, 2026
given that many of these functions take placeholder zarr-python 2.x kwargs like
synchronizerandchunk_storethat we do not support and do not plan to support, I feel like we can do something to clean this up before setting up a post-v2 migration roadmap?maxrjones commentedon May 15, 2026
Yes, we could deprecate those functions without a longer plan in place. The risk of that path as mentioned in #3976 (comment) is that we deprecate users of those functions to other functions that themselves go through later deprecation cycles if we decide we're not fully satisfied with those portions of the API. To mitigate that risk without a full roadmap, we'd at least need high confidence that the preferred paths (
create_array, etc.) are stable.More broadly, I'm advocating for considering users' long-term experience with zarr-python before starting a deprecation cycle, since each cycle causes some amount of pain. So, are you confident in the API design of
create_array? Which specific functions would you recommend instead of the 12 listed in #3976 (comment)? Are you confident enough in those that we won't subject users to more churn shortly after?The roadmap is one mechanism for ensuring we consider the long-term experience of users. It's my preferred mechanism, but I'm open to alternatives. For instance, if the broken kwargs are the specific motivation, we could raise on those kwargs (
synchronizer,chunk_store,object_codec, etc.) without deprecating the functions themselves.d-v-b commentedon May 15, 2026
That's along the lines of the second of my two proposals (the first being full deprecation):
More broadly, I recommend reading through each of the functions I enumerated. It's a mixed bag, but there are some simple answers: some of those functions literally do nothing except raise
NotImplementedError. We don't need a roadmap in place before we remove this kind of stuff. Others are weirdly storage-backend-specific in a way that's antithetical to the basic design of the library (saveonly targets the local file system). And others are genuinely useful for numpy compatibility (zerosand the like), but they take silly kwargs.maxrjones commentedon May 15, 2026
I worry that going back on an intention to implement a feature, implied by the existing function with a
NotImplementedError, a 1.5 year old issue that has promised users this functionality (#2407), and the "yet" in our own documentation, erodes the trust of less-engaged maintainers, users, and stakeholders if it isn't accompanied by communication about how we'll support that functionality in the future.If we had a roadmap embedded in zarr-python, we could deprecate the functions with a pointer to how we plan to support that functionality, so that we match the pain with a promise and a specific timeline. The lack of a timeline for V2 migration functionality contributed to us being in this position, so I propose we break out of that limitation.
Each of the functions you mentioned falls into one of three categories:
We could remove tier 1 without further planning. All others benefit from a comprehensive planning exercise. The sooner that is done, the sooner we can proceed with cleaning out the cruft in a systematic way. While the kwargs-specific changes could be exempted, I don't find that as urgent as producing a plan that would include prioritizing the most impactful cleanup or features.
d-v-b commentedon May 15, 2026
I think this gives too much credit to timelines. The lack of a timeline had no power to put us in this position, it was simply a lack of execution: we did not write implementations for
copy,copy_all, andcopy_store. Personally I think there are good reasons for not copying these functions verbatim, and thinking a bit more about the API we want here, in part because this functionality is so important. I am 100% in favor of having a plan for adding the functionality thatcopy,copy_all, andcopy_storeserved, but I also don't see why the lack of such a plan in the short term gives us any reason to keep functions that raiseNotImplementedError.maxrjones commentedon May 15, 2026
I'm glad we're aligned on the value of a plan. Timelines matter because they force the plan to be grounded in actual resources, including developer time. A post-V2 migration plan with timelines would have made it visible early that high-importance features like the
copyAPI weren't going to land successfully given the resources we had, which would have prompted a choice to rescope, reprioritize, or seek more help. Without timelines to surface overly optimistic parts of the plan, gaps only become visible at moments like this one, when we're considering deprecating the stubs for functions users have been asking about for 15 months. Accountability is a related benefit, but the usefulness to scoping is the relevant one to this discussion.My argument in #3976 (comment) is that our lack of a plan reduces trust/confidence across users and other stakeholders. They can't see where the library is going or when the gaps they're hitting will be addressed, only that we're active enough to clean up the API surface. Trust and confidence is what gives our library value via people/organizations using and/or investing in zarr-python, so I think it's worth weighing when making decisions like this.
d-v-b commentedon May 15, 2026
These two things can be true:
copytoday is worse than useless, because it takes up space in our public API but only raisesNotImplementedError. Frankly it looks unserious to have dead weight like this in the public API (e.g., confidence-eroding in stakeholders), and as there are no concrete plans to fix this, we should remove it immediately.copy. We should eventually plan out acopyfunction that does useful stuff for users.I don't see how these two actions -- remove dead code, and write a roadmap -- are related. From my POV, they are both things we need to do, that don't conflict.
maxrjones commentedon May 15, 2026
I agree that it was not a good design decision to include an empty stub for
copy. I still want to avoid this release note: "We removed the stubs forcopy", as it would not promote confidence in our execution ability. My preference would be: "We removed the stubs forcopy, see<roadmap>for the implementation plan and timeline". Concretely, I think that the two components are related because deletion and communicating a plan is better than deletion with no communication about the plan. My preference for the former is predicated on our ODD team creating and getting feedback on a roadmap by July 10. That calculus could sway to deletion without a plan being better if we are not able to get the plan merged into zarr-python during that timeframe. It seems worth it to me to do the better option here that includes a plan given that this dead code has already been sitting for 15 months.maxrjones commentedon Oct 17, 2026
the discussion in #4192 inspired me to follow through on this discussion. I think this is the consensus that came out of https://github.com/zarr-developers/zarr-python-planning.
Group 1: dead keyword arguments (remove in 3.5.0, ~Nov 2026)
synchronizer,chunk_store,cache_metadata,cache_attrs,object_codec,read_only,meta_arrayfrom thecreate,group,open_groupfunctions, which do nothing. Today they emit aZarrRuntimeWarningsaying "not yet implemented", which reads as a promise we aren't going to keep.ZarrDeprecationWarningsaying the argument is ignored and will be removed in a release after January 2027docs/user-guide/v3_migration.mdNo working code changes behavior.
Group 2: the
copystubs (3.5.0 for the message, implementation per the plan)copy,copy_all,copy_storeonly raiseNotImplementedError. The plan commits to implementing them in 3.x. Deleting the names now and adding them back later just creates two migrations where we need zero, so I'd rather keep the names and make the stubs honest.raise NotImplementedErrorwith a message pointing at [v3] Implementzarr.copy_store#2407 and the planning doccopy_store,copy,copy_allas additive 3.x features (tracked in [v3] Implementzarr.copy_store#2407, slotted as M2 in the plan)Group 3: working functions with real users
These need replacements first.
create,zeros,ones,empty,fulland the_likevariants. The numpy-style wrappers are worth keeping IMO.createis only a problem because of its signature and may be redemable.zeros/ones/empty/full/arrayand friends to callcreate_array, translating the legacy spellings (compressortocompressors,codecstoserializer/compressors,chunk_shapetochunks,dimension_separatortochunk_key_encoding) with aZarrDeprecationWarningon each legacy spelling onlycreateitself, pointing atcreate_arraycreateand the legacy spellingsopen,open_group,open_array,open_consolidated,group. All of these aremode=under the hood. xarray calls them in five places and icechunk in over twenty, and both run their test suites with warnings as errors, so a deprecation here before a replacement exists breaks downstream CI signaling they should migrate.open_for_read,create_or_overwrite,open_or_create)mode=onopen/open_group/open_array, and deprecateopen,open_consolidatedandgroupin favor of the new constructorssave,save_array,save_group,load. These work and are cheap to keep. Two small fixes:savedocstring that claims it only targets the local file systemloadfor groups (it currently raises) or document that it is array-onlyBefore any Group 3 deprecation lands
zarr.createappears 33 times,zarr.open28,zarr.group17; the suite runs with warnings as errors)Timeline summary
copystub messagescreate_array;createdeprecatedmode=,open,open_consolidated,groupdeprecateddocs: say save/save_array/save_group write to a store, not the local …
docs: say save/save_array/save_group write to a store, not the local …