Repository navigation
Conversation
…ers#4586) * ci(examples): run example scripts once in their own job Each example's PEP 723 header defines its environment, so running tests/test_examples.py in every pytest matrix job repeated the same work. Its header-rewriting helper also depended on the test environment's packaging and tomlkit, which broke the min_deps job on Python 3.12.3. A new `just examples` recipe uses `uv add --script` to point each header at the local checkout, and one CI job runs it. `uv add` resolves the header, and that showed two examples declaring requires-python >=3.11 although zarr requires 3.12. Those headers now say >=3.12. tomlkit was only used by the removed test, so it is dropped from the test dependency group. Assisted-by: ClaudeCode:claude-opus-5-5 * test(examples): run the example scripts through pytest The previous commit ran the examples from a bash loop, which stops at the first failing script, so one broken example hid the others. The examples now run from tests/test_examples.py again, one test per script. The tests are marked `examples` and skipped unless pytest gets --run-examples, as slow_hypothesis tests are with --run-slow-hypothesis, so the matrix jobs skip them. `just examples` passes the option, and the CI job runs it in the test.py3.13-minimal hatch environment. Assisted-by: ClaudeCode:claude-opus-5-5 * chore(just): rename the examples recipe to test-examples `just examples` could mean listing or building the examples. `test-examples` says what the recipe does, and follows the `<group>-<variant>` naming of recipes like `coverage-html` and `docs-build`. Assisted-by: ClaudeCode:claude-opus-5-5
…arr-developers#4575) * feat(zarr-metadata): add Hypothesis strategies for metadata fields Add `zarr_metadata.testing.strategies`, behind a new `testing` extra. It ships Hypothesis strategies that draw valid Zarr v3 metadata fields, one for every codec, data type, chunk grid and chunk key encoding the package defines that holds no other field. - Each kind's strategy takes what its configurations depend on: a data type's fill values take its configuration, a chunk grid takes the array shape, and a codec takes the `Chunk` it is handed. - `fields` draws every envelope spelling the spec allows, or only the canonical one. - Strategies are a mapping keyed by definition, so an extension author can add strategies for their own definitions. The tests read every draw back in `CORE_AND_EXTENSIONS` and require no problems. They also check: - that every definition in scope has a strategy, apart from the three that hold nested fields; - reach, with targets derived from each configuration's TypedDict: each optional key present and absent, each `Literal` value, and a non-canonical spelling where one exists; - fill-value spellings and envelope forms. Strategies follow the spec where the package is looser. Core data types are drawn only as bare strings, and regular-grid chunk lengths are never zero ("Chunk sizes must be greater than zero"), although the package's rules accept both today. Assisted-by: ClaudeCode:claude-opus-5-5 * chore(zarr-metadata): number the strategies changelog fragment after #414 Assisted-by: ClaudeCode:claude-opus-5-5 * chore(zarr-metadata): number the strategies changelog fragment after PR 4575 The fragment was numbered after fork PR 414, so the changelog linked to an unrelated upstream PR. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): draw a fresh empty configuration on every draw `st.just({})` (crc32c) and `st.just(EmptyConfiguration())` (every data type with nothing to configure) handed out one dict for every draw, and `fields` puts the configuration in the field as it is. A test that changed the configuration it drew changed every later draw, which Hypothesis reports as a flaky strategy. Both are now `st.builds(EmptyConfiguration)`. Export `Configurations` and `NOTHING_KNOWN` from `zarr_metadata.testing.strategies`: they appear in the signatures of `DefinitionStrategy.unconstrained` and `codec_configurations`. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): check every drawn field, and draw hex scale_offset values `fields` read a field only when asked for the canonical spelling, so a caller's configurations the definition refuses were drawn silently in the default spelling. Every field is now read, in a scope built once per call rather than once per draw, and a refused one raises in both spellings. A definition no Zarr v3 scope holds, such as a v2 codec, is a `TypeError` when `fields` is called, not a `StopIteration` on a draw. A float `scale_offset` codec's `offset` and `scale` are fill values of its type, so they are now also drawn as the hex string of their bits. The docstring no longer claims a non-zero scale makes decoding defined: a tiny one still overflows. The codec chunk test now draws chunks of an unknown data type and with unknown lengths, which every codec takes. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): draw scale_offset for every number type, and widen Strategies keys `scale_offset` drew nothing for complex numbers, numpy times and data types the package does not define, though its rules refuse only data types whose values are not numbers, and a numpy time "is compatible with any codec that supports arrays of signed 64-bit integers". A numpy time is now drawn as that integer, a complex number as a pair of finite floats, and a type the package does not define takes the identity configuration. None now means only what `CodecStrategy` says it means: no configuration takes the chunk. `Strategies` is keyed by `Any`: a mapping's key type is invariant, so a mapping an extension author builds, keyed by a `CodecDefinition`, was not a `Mapping[Definition[Any], ...]` to pyright. A rectilinear dimension of unknown length may now be drawn with no edges, which its rules accept. Assisted-by: ClaudeCode:claude-opus-5-5 * refactor(zarr-metadata): share the strategies' empty configuration and rank cap `EMPTY_CONFIGURATIONS` and `MAX_RANK` are defined once, in `_strategy`, rather than once in each module that draws them. A numpy time's `scale_offset` values are now drawn finite, never the integer numpy reads as `NaT`, as the docstring says. The `fields` docstring now says that a codec's chunk rules are not checked, since no chunk is known. Assisted-by: ClaudeCode:claude-opus-5-5 * test(zarr-metadata): cover scale_offset over a data type the package does not define The identity configuration `scale_offset` draws for a data type the package does not define had no test, since the chunk test draws only the package's own data types. A test now reads an extension `bfloat16` in a scope of its own and checks the codec takes every draw. Also splits the `_scope_of` error message across two literals and drops a doubled blank line in `_data_type`. Assisted-by: ClaudeCode:claude-opus-5-5 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* chore(deps): bump cast-value.rs to >=0.5.2 * chore(deps): raise the runtime cast-value-rs floor to 0.5.1 The previous commit raised the packaging floor to 0.5.1 but left the runtime check at 0.4.2. cast-value-rs 0.5.0 makes casts exact at integer and float range limits; 0.5.0 was never published to PyPI, so the floor is 0.5.1. Also update the error message, the version-check test cases, and uv.lock. Assisted-by: ClaudeCode:claude-opus-5-5 * test(cast_value): derive version-check cases from the minimum The cases hard-coded versions near the floor, so every floor bump meant rewriting them. Build them from CAST_VALUE_RS_MIN_VERSION instead, and drop cases that repeated the same comparison. A floor bump now touches only pyproject.toml and the constant. Assisted-by: ClaudeCode:claude-opus-5-5
…names `zarr_metadata.v3.data_type`, `.codec`, `.chunk_grid` and `.chunk_key_encoding` export every name their submodules export, so `from zarr_metadata.v3.data_type import BoolDataTypeName` works again, as it did before 0.6.0 (downstream zarrista imports the data type names that way). Each submodule stays the name's home: the packages are aliases, like the top-level namespace and the format front doors, and a new test pins that each re-exports exactly its submodules' names as the same objects. The naming checks skip the alias packages, whose names they already check in the submodules. Assisted-by: ClaudeCode:claude-opus-5-5
…r-developers#4588 Assisted-by: ClaudeCode:claude-opus-5-5
…ONTEXT scope (zarr-developers#4583) * feat(zarr-metadata)!: read what zarr-python writes in a ZARR_PYTHON_CONTEXT scope Add definitions for the extensions zarr-python writes. zarr-extensions registers `fixed_length_utf32` and the `vlen-utf8` and `vlen-bytes` codecs, so they join CORE_AND_EXTENSIONS_CONTEXT. zarr-python's own `null_terminated_bytes` and `variable_length_bytes`, which no specification registers, are only in the new ZARR_PYTHON_CONTEXT, so the default scope stays spec-pure. Rename CORE to CORE_CONTEXT and CORE_AND_EXTENSIONS to CORE_AND_EXTENSIONS_CONTEXT, so that a scope's name says it is a context. Fixes #423 Assisted-by: ClaudeCode:claude-opus-5-5 * docs(zarr-metadata): number the changelog fragments for PR 426 Assisted-by: ClaudeCode:claude-opus-5-5 * docs(zarr-metadata): number the changelog fragments for PR 4583 Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): keep the old scope names and tighten the string rules Keep `CORE` and `CORE_AND_EXTENSIONS` importable from `zarr_metadata.v3` and `zarr_metadata.v3.definition` as the same scopes with a DeprecationWarning, so a consumer such as pydantic-zarr keeps working across the rename. Check that `vlen-utf8` encodes `string` and `vlen-bytes` encodes `bytes` (or zarr-python's `variable_length_bytes`), as zarr-extensions says, instead of accepting any data type whose values vary in size. Do not count trailing null bytes against the size of a `null_terminated_bytes` fill value, since they are padding. Spell a `fixed_length_utf32` fill value without its trailing U+0000 padding, so `"a"` and `"a\u0000"` have one canonical spelling. Assisted-by: ClaudeCode:claude-opus-5-5 * feat(zarr-metadata): read zarr-python's raw_bytes and rename CORE_V2 Add zarr-python's `raw_bytes` data type (NumPy's `V`) to ZARR_PYTHON_CONTEXT, so the scope reads every data type zarr-python writes. Its names start with `ZarrPython`, since `RawBytes` already names the spec's raw bits. A fill value longer than `length_bytes` is a problem, because zarr-python cuts it short, and the canonical spelling pads it with null bytes, as zarr-python writes it. Rename CORE_V2 to CORE_V2_CONTEXT, so every scope's name ends in `_CONTEXT`. The old name keeps working with a DeprecationWarning, through the same `renamed_scope` helper as the v3 names, which moves to `zarr_metadata._renamed`. Share the base64 decoding and spelling of fill values in `_byte.py`, and say that the vlen codecs' `configuration` key is optional, since zarr-python writes an empty object. Assisted-by: ClaudeCode:claude-opus-5-5 * refactor(zarr-metadata): declare the v2 scope's old name once Move CORE_V2_CONTEXT and its table of old names into `v2/_core.py`, as `v3/_core.py` holds the v3 scopes, so `zarr_metadata.v2` and `zarr_metadata.v2.definition` share one table. Say why a `fixed_length_utf32` fill value counts its trailing U+0000 code points, as the spec requires, while `null_terminated_bytes` follows zarr-python and does not, and test a value that only its padding makes too long. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): give main's strategies the renamed scopes and new definitions The Hypothesis strategies merged from main (zarr-developers#4575) imported `CORE`, `CORE_AND_EXTENSIONS` and `CORE_V2` from modules that now hold only the `_CONTEXT` names, and their coverage test requires a strategy for every definition in `CORE_AND_EXTENSIONS_CONTEXT`. Rename the imports, and add strategies for `fixed_length_utf32`, `vlen-utf8` and `vlen-bytes`; each vlen codec draws nothing for a data type it does not read. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): draw zarr-python's own data types in the testing strategies `DEFAULT_STRATEGIES` holds a strategy for each definition the package defines, but `null_terminated_bytes`, `variable_length_bytes` and `raw_bytes` had none, so `fields()` raised `LookupError` for them. Draw each, and check the strategies against `ZARR_PYTHON_CONTEXT`, which holds every definition. Say that the documents in test_zarr_python_scope.py leave out the compressors zarr-python adds. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(zarr-metadata): pin the registered extensions to CORE_AND_EXTENSIONS_CONTEXT The strategies are now checked against `ZARR_PYTHON_CONTEXT`, which no longer proves that `fixed_length_utf32`, `vlen-utf8` and `vlen-bytes` are in `CORE_AND_EXTENSIONS_CONTEXT`. Assert it directly. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(zarr-metadata): let the vlen strategies ask their codec's rules Each vlen codec strategy named its compatible data types a second time; draw nothing for exactly the chunks the codec's own chunk rules refuse, so the two cannot drift. Draw `fixed_length_utf32` fill values with trailing U+0000 padding too, which the canonical spelling removes. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…v3-package-reexports # Conflicts: # packages/zarr-metadata/src/zarr_metadata/v3/codec/__init__.py
…r-developers#4560) * fix(codecs): reshape flattened numcodecs filter output on encode (zarr-developersgh-4553) * chore: add release notes fragment for zarr-developersgh-4553 * fix(codecs): reshape filter output to the resolved output shape --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Owner
Author
|
🤖 AI text below 🤖 Superseded by zarr-developers#4588, which merged this branch upstream. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
zarr_metadata.v3.data_type,zarr_metadata.v3.codec,zarr_metadata.v3.chunk_gridandzarr_metadata.v3.chunk_key_encodingre-export every name their submodules export, sofrom zarr_metadata.v3.data_type import BoolDataTypeName, Uint8DataTypeNameworks again, as it did before 0.6.0. Each name is the same object as in its submodule, which stays its documented home.Why
0.6.0 emptied these packages'
__all__so that each name had one public home. Importing from the package is a convenient way to get several data types or codecs at once, and downstream code relies on it: developmentseed/zarrista (zarr-metadata>=0.4) imports the 17*DataTypeNameliterals fromzarr_metadata.v3.data_typein its type stubs, which fail to resolve against 0.6.0.How
__all__(no name collides across submodules).test_an_alias_package_exports_its_submodules_names, pins that each package exports exactly its submodules' names, as the same objects.members: false, so no name appears twice in the docs.Checks
just checkinpackages/zarr-metadata(ruff, pyright, the suite on Python 3.11 and 3.14,mkdocs build --strict) passes. Every one of zarrista's 31zarr_metadataimports resolves at runtime and under pyright on this branch, and no import order creates a cycle. roborev found no issues.Stack notes
Independent of #432 (the field/declaration rename); both edit the
v3/codec/__init__.pydocstring, so whichever lands second resolves a small conflict there. The changelog fragment is the orphanchanges/+v3-package-reexports.feature.mduntil an upstream PR gives it a number.🤖 Generated with Claude Code