Repository navigation
fix: validate consolidated metadata listing keys and hierarchy - #4485
barlowa124 wants to merge 5 commits into
Conversation
668c547 to
7b01078
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4485 +/- ##
==========================================
+ Coverage 94.69% 94.72% +0.03%
==========================================
Files 94 94
Lines 13606 13685 +79
==========================================
+ Hits 12884 12963 +79
Misses 722 722
🚀 New features to boost your workflow:
|
d-v-b
left a comment
There was a problem hiding this comment.
- Warn instead of raising when a v3 consolidated listing has a reserved node name (
__-prefixed,zarr.json,.,..). today we can write__a(create_array("__a")plusconsolidate_metadata), and those stores open on main. With this PR,zarr.open_group(store)raisesMetadataValidationErrorfor the whole group, siblings included. Make it a deprecation warning that describes the listing and says to rename the node, and raise only after #4484 has stopped these names being written. - Don't apply the reserved-name rules to v2 listings. The v2 spec doesn't reserve these names, and v2
.zmetadatagoes through the samefrom_dict. - Keep raising for a child listed without its parent and for a node listed below an array. Those listings already failed (
KeyError, unopenable root), so a clear error is an improvement. - Use single backticks in
changes/4485.bugfix.md, and say which cases warn and which raise.
|
we don't want to break stuff that works today, even if it violates a subtle part of the spec. so lets do this via deprecation |
|
fc0e8cf makes the changes.
|
|
Done in fc0e8cf: reserved node names in a v3 consolidated listing now emit a |
Summary
Second item of #4453.
ConsolidatedMetadata.from_dict(andGroupMetadata.from_dict, which delegates to it) accepted listingsthe spec rules out, two of which broke reads.
Listing keys are node paths, so keys with an empty segment (
"",a//b) are now rejected withMetadataValidationError. Keys carryinga reserved segment emit
a
ZarrDeprecationWarningnaming the key and advising a rename,avoiding breakage of stores that work today.
Every non-root key also needs its ancestors present as group entries:
{"a/b": GROUP}used to surface asGroupNotFoundErroron open,making the root unopenable from one entry. It now fails with a clear
MetadataValidationError, as does a listing nested under an array.v2 listings skip the name checks entirely. The remaining #4453 items
are untouched.
For reviewers
The reserved-name path moved from rejection to deprecation after
review, since stores written today with
__-prefixed names still openon main. It becomes a hard error once #4484 stops those names being
written. The empty-segment rule stays strict because consolidated keys
are node paths and node names cannot be empty. If #4484 lands first,
this can reuse its helper for the reserved-name part.
Author attestation
TODO
Done:
docs/user-guide/*.mdOutstanding:
changes/