Read remote netCDF4 rasters chunk-wise through h5py

[SmithB]

from_nc() on an s3:// URI downloaded the whole object to return one tile: netCDF4.Dataset takes a filename or an in-memory buffer but never a file object, so a 60 km window out of a 2.9 GiB ATL15 granule cost 2.9 GiB.

These files are HDF5 underneath and h5py does take a file object, so the remote case now goes through h5py and range requests, and a windowed read costs the chunks the window touches. grid/nc_h5.py presents an h5py.File through the netCDF4 API from_nc() uses, so the body of from_nc() is unchanged. Four differences between the libraries live there:

• netCDF4 applies scale_factor/add_offset unless set_auto_scale(False) is called, and from_nc() only turns off masking, so the adapter replicates that arithmetic -- otherwise packed variables silently change value.
• h5py rejects the negative-step slices select_slices() emits for a descending y axis, and broadcasts multiple index arrays.
• netCDF dimensions that are not netCDF variables, and the HDF5 bookkeeping attributes netCDF4 hides, have to stay hidden.
• netCDF4 lets a scalar variable be indexed with [:].

nc_open() is now resolve-source / open rather than a remote x compression matrix, and takes engine ('auto', 'h5py', 'netcdf4'), block_size and rdcc_nbytes. A file h5py cannot open (NETCDF3/classic) falls back to netCDF4. Local uncompressed reads take exactly the path they did before. External whole-file compression still decompresses whole: a gzip or bzip2 stream carries no index, so there is nothing to read chunk-wise.

Block size is the difference between a win and a loss, not a tuning knob: fsspec caches one block, so scattered chunk reads at the 5 MiB default re-fetch enough to lose to downloading the file outright. Measured on a 29 MiB gzip-chunked granule over https, a 501x501 window fetches 91.1 MiB at 5 MiB blocks, 7.4 MiB at the 256 KiB default, 3.9 MiB at 64 KiB. io_utils.open_remote() passes block_size per file rather than to the session, so it reaches an earthaccess DAAC session too, whose constructor takes no such argument. h5_open(), from_h5() and io_utils.open_h5() gained the same parameter.

Also fixes a fill-value bug this surfaced: from_nc() compared scaled data against a raw _FillValue, so a packed variable's fills were never converted. On a real GOES granule that left 91% of a window reading as 179.9976 K instead of NaN. Unpacked variables are unaffected.

Tests are network-free -- engine='h5py' reads a local file through the same reader, and a stand-in filesystem maps s3:// URIs onto local files -- and assert the two readers return identical arrays under bounds, bands, skip, group, meta_only and t_axis, on to_nc() fixtures and on a netCDF4-written packed fixture. tests/test_nc_h5_remote.py measures bytes actually transferred against real granules; it is opt-in (PC_TEST_REMOTE for the https test, PC_TEST_S3 for the NSIDC ones, which need us-west-2).

Claude-Session: https://claude.ai/code/session_016WPhgmqfUv5vYpBfnm592f