Skip to content

fix: zero default scalar for bytes fields in structured dtypes - #4495

Merged
d-v-b merged 2 commits into
zarr-developers:mainfrom
safaksm:fix/struct-default-scalar-bytes
Oct 9, 2026
Merged

d-v-b merged 2 commits into
zarr-developers:mainfrom
safaksm:fix/struct-default-scalar-bytes

Conversation

@safaksm

@safaksm safaksm commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Structured.default_scalar builds each field's zero with np.array([0], dtype)[0], which breaks for bytes fields:

  • V (raw bytes): numpy raises TypeError: a bytes-like object is required, not 'int', so create_array fails for any struct with a V field unless fill_value is passed.
  • S (byte string): numpy returns b"0", so unwritten chunks read back as "0".

The same types outside a struct already default to zero bytes (S4 to b"", V4 to four zero bytes). This PR uses np.zeros(1, dtype)[0], which gives zero bytes for every fixed-size field.

import numpy as np, zarr

dt = np.dtype([("a", "V4"), ("b", "<f4")])
zarr.create_array(zarr.storage.MemoryStore(), shape=(2,), dtype=dt)  # TypeError

dt = np.dtype([("a", "S4"), ("b", "<f4")])
zarr.create_array(zarr.storage.MemoryStore(), shape=(2,), dtype=dt)[:]  # [(b'0', 0.), (b'0', 0.)]

I hit this while mapping fixed-size HDF5 records (WOD plankton data) to a Zarr struct.

For reviewers

Only new struct arrays created without fill_value change; stored arrays keep theirs.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions (no API change)
  • New/modified features documented in docs/user-guide/*.md (no feature change)
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.69%. Comparing base (246fa20) to head (83ad7e8).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4495   +/-   ##
=======================================
  Coverage   94.69%   94.69%           
=======================================
  Files          94       94           
  Lines       13619    13619           
=======================================
  Hits        12897    12897           
  Misses        722      722           
Files with missing lines Coverage Δ
src/zarr/core/dtype/npy/structured.py 97.14% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@d-v-b

d-v-b commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

thanks @safaksm, I'll see if we can get this released soon

@d-v-b
d-v-b merged commit 3d1a839 into zarr-developers:main Oct 9, 2026
40 checks passed
d-v-b added a commit that referenced this pull request Oct 9, 2026
…st (#4496)

#4495 also changed the default of `U` fields in structured dtypes from
"0" to "". Add `U4` to the regression test and name string fields in
the changelog entry.

Assisted-by: ClaudeCode:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants