Metadata-Version: 2.4
Name: xarray-blosc2
Version: 1.7.1
Summary: Blosc2 storage backend for xarray — single-file, Pyodide-ready
Author: Rolv Erlend Bredesen
License: BSD-3-Clause
Project-URL: Homepage, https://github.com/rolveb/xarray-blosc2
Project-URL: Repository, https://github.com/rolveb/xarray-blosc2
Keywords: xarray,blosc2,compression,scientific-data,meteorology
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: xarray>=2024.9.0
Requires-Dist: blosc2>=3.12.0
Requires-Dist: numpy>=1.26.0
Provides-Extra: all
Requires-Dist: httpx; extra == "all"
Requires-Dist: caterva2; extra == "all"
Requires-Dist: starlette; extra == "all"
Requires-Dist: uvicorn; extra == "all"
Requires-Dist: tyro>=0.9.0; extra == "all"
Requires-Dist: uncertainties; extra == "all"
Requires-Dist: pandas; extra == "all"
Requires-Dist: blosc2-btune; extra == "all"
Requires-Dist: zstandard>=0.22.0; extra == "all"
Provides-Extra: btune
Requires-Dist: blosc2-btune; extra == "btune"
Provides-Extra: crypto
Requires-Dist: cryptography>=41.0; extra == "crypto"
Provides-Extra: lossy
Requires-Dist: blosc2-grok; extra == "lossy"
Provides-Extra: zstd-dict
Requires-Dist: zstandard>=0.22.0; extra == "zstd-dict"
Provides-Extra: icechunk
Requires-Dist: icechunk>=2.0; extra == "icechunk"
Requires-Dist: zarr>=3.0; extra == "icechunk"
Provides-Extra: dev
Requires-Dist: pytest>=8.3.0; extra == "dev"
Requires-Dist: pytest-cov>=6.0.0; extra == "dev"
Requires-Dist: pytest-playwright>=0.5.0; extra == "dev"
Requires-Dist: ruff>=0.7.0; extra == "dev"
Dynamic: license-file

# xarray-blosc2

[![Python Version](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![Tests](https://img.shields.io/badge/tests-197%20passed-success.svg)](tests/)
[![License](https://img.shields.io/badge/license-BSD--3--Clause-blue.svg)](LICENSE)
[![Blosc2](https://img.shields.io/badge/blosc2-%E2%89%A53.12-orange.svg)](https://www.blosc.org/)
[![Version](https://img.shields.io/badge/version-1.7.1-green.svg)](CHANGELOG.md)

**Blosc2 storage backend for xarray — single-file, Pyodide-ready, with ACID versioned storage.**

## Why xarray-blosc2?

| Capability              | xarray-blosc2                          | Zarr v3         | NetCDF-4       | icechunk         |
|-------------------------|----------------------------------------|-----------------|----------------|------------------|
| Compression             | blosc2 (5–90×), geogrid, ZSTD dict     | blosc/zstd      | HDF5 filters   | Zarr codecs      |
| Random access           | chunk + block level                    | chunk level     | chunk level    | chunk level      |
| Single-file archive     | `.b2z` (zip)                           | `.zip` (limited)| yes (HDF5)     | no               |
| Versioned storage       | `VersionedStore` (ACID)                | no              | no             | yes (core)       |
| CF conventions          | CF-1.11 automatic                      | manual          | manual         | manual           |
| Encryption              | per-chunk AES-256-GCM                  | no              | no             | no               |
| Browser / Pyodide       | yes                                    | yes (partial)   | no             | no               |
| Fast reductions         | on compressed data                     | no              | no             | no               |
| Uncertainty propagation | built-in                               | no              | no             | no               |
| icechunk bridge         | `to_icechunk` / `from_icechunk`        | (native)        | no             | (native)         |

### vs icechunk

icechunk provides production-grade version control for Zarr v3 stores. `xarray_blosc2.VersionedStore` offers the same API surface (snapshots, branches, tags, garbage collection) but uses blosc2 as the storage engine, giving you:

- **Block-level random access** — blosc2 chunks contain blocks (~KB, L2-cache-sized); a point extraction touches ~140 bytes, not a 4 MB Zarr chunk.
- **Geogrid compression** — 3.4× on weather grids, 8 % smaller than GRIB2 + CCSDS, 17× faster decompression.
- **Per-chunk encryption** — AES-256-GCM per chunk, preserving chunk-level random access.
- **Single-file deployment** — freeze to `.b2z` for archival or sharing.

For interop the other way, `xarray_blosc2.to_icechunk(ds, repo, message=...)` writes any xarray Dataset into an icechunk repository, and `from_icechunk(repo)` reads it back — see [icechunk bridge](#icechunk-bridge) below.

## Features

- **5–90× compression** on structured scientific data
- **Geogrid preset** — CCSDS-inspired pipeline (int16 quantize + row-scan differencing + ZSTD+shuffle) for weather/climate grids
- **ZSTD dictionary compression** — train domain-specific dictionaries for 15–40 % extra compression
- **Lazy loading** via blosc2 NDArray (no data in memory until sliced)
- **Fast reductions** on compressed data (`sum`, `mean`, `std`, `min`, `max`)
- **CF-1.11 conventions** — automatic `Conventions` attribute, axis detection, per-variable metadata preservation
- **Versioned storage** — ACID transactions, snapshots, branches, tags, garbage collection (local FS or S3)
- **Unified `open()`** — one entry-point reading `.b2` / `.b2z` / `.b2nd` / bytes / caterva2 URL, with optional AES-256-GCM decryption
- **icechunk bridge** — `to_icechunk` / `from_icechunk` for interop
- **Memory guard** — `memory_guard(max_gb=…)` context that caps peak RSS during pipelines
- **Pipeline lineage** — `save_dataset(..., source_files=, history=)` records provenance; `load_pipeline` reads it back
- **Uncertainty propagation** with `UncertainDataArray` and the `.unc` accessor
- **Lossy compression** — CF-compliant scale/offset packing (optional)
- **Encryption** — per-chunk AES-256-GCM (optional `cryptography`)
- **Isoweek sharding** — time-series appending without rewriting existing data
- **Binary blob serialization** for APIs and databases (msgpack)
- **Remote access** via Caterva2 (`open_remote`) or the unified `open("http://...")`
- **Zarr Store wrapper** — `xr.open_zarr()` compatibility
- **Format conversion** — convert between blosc2, NetCDF, Zarr, and GRIB
- **Single file** — copy `xarray_blosc2.py` anywhere, including Pyodide/browser

## Installation

```bash
pip install xarray-blosc2
```

With optional features:

```bash
pip install xarray-blosc2[btune]      # Auto-tuning (blosc2-btune)
pip install xarray-blosc2[crypto]     # Encryption (cryptography)
pip install xarray-blosc2[lossy]      # JPEG2000 lossy (blosc2-grok)
pip install xarray-blosc2[zstd_dict]  # ZSTD dictionary compression
pip install xarray-blosc2[icechunk]   # icechunk bridge
pip install xarray-blosc2[all]        # All optional deps
```

For development:

```bash
git clone https://github.com/rolveb/xarray-blosc2.git
cd xarray-blosc2
pip install -e ".[dev]"
```

## Quick Start

```python
import numpy as np
import xarray as xr
import xarray_blosc2 as xb  # registers the "blosc2" engine automatically

# Create and save
ds = xr.Dataset({
    "temperature": (["time", "x"], np.random.randn(100, 50).astype("f4")),
    "pressure":    (["time", "x"], np.random.randn(100, 50).astype("f4") + 1013),
})
xb.save_dataset(ds, "weather.b2")

# Load — one entry-point, dispatches on shape/extension
ds2 = xb.open("weather.b2")             # → directory of .b2nd
print(ds2["temperature"].shape)         # (100, 50) — lazy, not in memory
subset = ds2["temperature"].isel(time=slice(0, 5))  # reads only these chunks

# Freeze to a single-file archive
xb.save_dataset(ds, "weather.b2", freeze=True)   # writes weather.b2z, removes dir
ds3 = xb.open("weather.b2z")            # → same Dataset from the zip

# The classic engine still works
ds4 = xr.open_dataset("weather.b2", engine="blosc2")
```

`xb.open()` also accepts raw bytes (from `to_blob`), an encrypted archive (with `key=`), a caterva2 URL, and glob patterns — one function, many sources.

## CF-1.11 Conventions

xarray-blosc2 follows the [CF Conventions](https://cfconventions.org/) (version 1.11) for self-describing scientific data. CF compliance is **automatic** — no manual configuration needed.

### What happens on save

When you call `save_dataset()`, the library:

1. **Sets `Conventions: "CF-1.11"`** in the dataset's global attributes (in `dataset.json`). If the dataset already has a `Conventions` attribute, it is preserved.
2. **Preserves per-variable CF attributes** — `standard_name`, `long_name`, `units`, `cell_methods`, `ancillary_variables`, `coordinates`, and any other attributes are stored in the manifest's `variable_attrs` section. The vlmeta on each `.b2nd` file carries `units` and `long_name` for quick inspection; the full attribute set lives in `dataset.json` (no key-count limit).
3. **Auto-detects coordinate axes** — coordinates named `time`/`t`/`datetime`/`valid_time` get `axis: "T"`, `lon`/`longitude` get `axis: "X"`, `lat`/`latitude` get `axis: "Y"`, `level`/`depth`/`pressure`/`height` get `axis: "Z"`. Datetime-typed coordinates are also detected as `axis: "T"` regardless of name.
4. **CF-compliant lossy compression** — when using `lossy={"temperature": {"max_error": 0.01}}`, the packing uses the CF convention `unpacked = packed * scale_factor + add_offset`. The `scale_factor` and `add_offset` attributes are set on the packed variable, so any CF-aware tool can unpack it.
5. **Lineage tracking** — every save records `change_time`, `valid_time` range, `source_files`, `history`, and library versions in the manifest. This maps naturally to CF's `history` attribute and is readable back via `xb.load_pipeline("weather.b2")`.

### Storage layout

```
weather.b2/
  dataset.json          # manifest: dims, coords, global attrs (Conventions, history, ...),
                        #           variable_attrs (per-var CF metadata), lineage
  temperature.b2nd      # blosc2 NDArray with vlmeta: dims, units, long_name, dtype
  pressure.b2nd
  time.b2nd             # coordinate with axis="T" auto-detected
  x.b2nd
```

### Custom CF metadata

```python
xb.save_dataset(ds, "output.b2",
    cf_metadata={
        "institution": "Norwegian Meteorological Institute",
        "source": "NORA3 hindcast",
        "references": "https://doi.org/10.xxxx",
    },
)
```

These are merged into the global attributes alongside `Conventions`.

## Geogrid Compression

The geogrid preset implements a CCSDS-inspired pipeline optimized for regularly-gridded weather/climate data:

```
float32 → int16 quantize → row-scan differencing → ZSTD + byte shuffle
```

```python
# Save with geogrid preset (0.001 precision, matching GRIB native)
xb.save_dataset(ds, "era5.b2", preset="geogrid")

# Per-variable precision
xb.save_dataset(ds, "era5.b2",
    preset="geogrid",
    precision={"u100": 0.01, "msl": 1.0},
)
```

| Pipeline              | Bits/value | Ratio    | Decompress   |
|-----------------------|-----------:|---------:|-------------:|
| float32 raw           |       32.0 |     1.0× |           -- |
| ZSTD+shuffle (plain)  |       12.1 |     2.6× |     3.2 GB/s |
| **Geogrid (ours)**    |    **9.1** | **3.4×** | **2.7 GB/s** |
| GRIB2+CCSDS           |        9.9 |     3.2× |     160 MB/s |
| Shannon bound         |        8.5 |     3.8× |  theoretical |

The geogrid pipeline is automatically decoded on load — no special reader code needed.

## Versioned Storage (VersionedStore)

ACID transactional storage inspired by icechunk. Immutable frozen shards + atomic manifest swap.

```python
from xarray_blosc2.versioned_store import Repository, local_filesystem_storage

# Create a versioned repository
repo = Repository.create(local_filesystem_storage("./era5_store"))

# Write data transactionally
with repo.writable_session() as session:
    session.add_shard("2026W13", ds_week13, preset="geogrid")
    snap = session.commit("added week 13 ERA5 data")

# Read (pinned to a snapshot — isolation)
ds = repo.open_dataset()

# Time travel
for s in repo.ancestry():
    print(f"{s.written_at}: {s.message} ({len(s.shards)} shards)")

# Branches and tags
repo.create_branch("experimental")
repo.create_tag("v1.0", snap.snapshot_id)

# Garbage collection
from datetime import datetime, timezone
repo.expire_snapshots(older_than=datetime(2026, 1, 1, tzinfo=timezone.utc))
summary = repo.garbage_collect()
print(f"Freed {summary.bytes_deleted} bytes")
```

### S3 backend

```python
from xarray_blosc2.versioned_store import Repository, s3_storage

storage = s3_storage("my-bucket", prefix="era5/", region="eu-north-1")
repo = Repository.open_or_create(storage)
```

### Layout options

- **`"bundled"`** (default) — all variables in one shard. Atomic updates, simpler.
- **`"per_variable"`** — one shard tree per variable. Independent update cadence.

```python
repo = Repository.create(storage, layout="per_variable")
```

## icechunk bridge

Bridge to `icechunk` for teams that want Zarr v3 tooling alongside blosc2's random access:

```python
import icechunk
import xarray_blosc2 as xb

repo = icechunk.Repository.create(icechunk.local_filesystem_storage("./ic_repo"))

# Write any xarray Dataset into icechunk (blosc2 codec by default)
xb.to_icechunk(ds, repo, message="initial NORA3 slab", branch="main")

# Read back
ds2 = xb.from_icechunk(repo, branch="main")
```

Requires `pip install xarray-blosc2[icechunk]`.

## Encryption

Per-chunk AES-256-GCM encryption preserves chunk-level random access. Each chunk is encrypted independently — accessing a single chunk only decrypts that chunk.

```python
key = xb.generate_encryption_key()

# Save encrypted (creates .b2enc files)
xb.save_encrypted_dataset(ds, "secret.b2enc", key=key)

# Open with the unified reader — pass the key
ds = xb.open("secret.b2enc", key=key)

# Or use the typed reader
ds = xb.open_encrypted_dataset("secret.b2enc", key=key)

# Freeze to encrypted archive
xb.save_encrypted_dataset(ds, "secret.b2enc", key=key, freeze=True)
```

Block-level random access within a chunk is not available when encrypted (the entire chunk must be decrypted). For unencrypted data, both chunk- and block-level access are available.

## ZSTD Dictionary Compression

Train a domain-specific dictionary for 15–40 % extra compression on repetitive data:

```python
# Auto-train dictionary on first variable
xb.save_dataset(ds, "data.b2", zstd_dict=True)

# Reuse a pre-trained dictionary
dict_bytes = xb.train_zstd_dict(training_data)
xb.save_dataset(ds, "data.b2", zstd_dict=dict_bytes)
```

Requires `pip install xarray-blosc2[zstd_dict]`.

## More Features

### Save, Load, Append

```python
# Save with lineage tracking
xb.save_dataset(ds, "data.b2",
    source_files=["/data/model_run_42.nc"],
    history=["remap to 0.1°", "clip to Norway box"],
)

# Append new timesteps atomically
xb.append_dataset(ds_new, "data.b2", dim="time")

# Freeze to single-file archive (.b2z)
xb.save_dataset(ds, "data.b2", freeze=True)   # writes data.b2z, removes data.b2/
# Or freeze an existing directory in place:
xb.freeze_dataset("data.b2")                  # data.b2 → data.b2z (dir removed)

# Open frozen archive — same one-liner, different extension
ds = xb.open("data.b2z")

# Inspect lineage
pipeline = xb.load_pipeline("data.b2z")
print(pipeline["history"])
```

### Isoweek Sharding

Solves the append-vs-read-order conflict: each shard is written once in the optimal memory order. "Appending" = creating a new shard.

```python
xb.save_dataset(ds, "data.b2",
    shard_by="isoweek",
    freeze=True,   # each shard becomes a .b2z
)
```

### Fast Reductions (on compressed data)

Fast reductions operate on the compressed blosc2 array directly — no full decompression. Use the `.b2` accessor to hand the underlying `blosc2.NDArray` to the reducer:

```python
arr = ds["temperature"].b2.array          # → blosc2.NDArray
total = xb.fast_sum(arr)
avg   = xb.fast_mean(arr, axis=0)
std   = xb.fast_std(arr)
lo    = xb.fast_min(arr)
hi    = xb.fast_max(arr)
```

### Blob Serialization

```python
blob = xb.to_blob(ds)                # → bytes (.b2z zip)
ds   = xb.from_blob(blob)            # roundtrip
ds   = xb.open(blob)                 # or via the unified reader
```

`to_blob(ds, key=key)` produces an AES-256-GCM sealed blob; `open(blob, key=key)` reads it back.

### Uncertainty Propagation

```python
from xarray_blosc2 import UncertainDataArray

a = UncertainDataArray.from_nom_std(
    xr.DataArray([1.0, 2.0, 3.0], dims=["x"]),
    xr.DataArray([0.1, 0.2, 0.1], dims=["x"]),
    name="strain",
)
result = a * 2.0 + a
print(result.std)  # propagated uncertainties
```

### Lossy Compression

```python
# CF-compliant float-to-int packing with bounded error
xb.save_dataset(ds, "data.b2", lossy={
    "temperature": {"max_error": 0.01},
    "pressure":    {"max_error": 0.1},
})
```

### Memory guard

Wrap a pipeline to cap peak resident memory during a batch of operations:

```python
with xb.memory_guard(max_gb=8.0):
    for path in paths:
        ds = xb.open(path)
        xb.save_dataset(ds.sel(time=slice("2026", "2027")), path.replace(".b2", "_2026.b2"))
```

The guard raises `MemoryError` if the process crosses the limit, letting the caller decide whether to retry with smaller chunks.

### Remote Access (Caterva2)

```python
ds_remote = xb.open_remote("https://server:8081", "datasets/@public/era5")
ds_local  = ds_remote.materialize(time=slice(0, 10))

# Or via the unified reader
ds = xb.open("https://server:8081/api/datasets/@public/era5")
```

### Compression Tuning

```python
results, fig = xb.benchmark_compression(
    data,
    chunks=[(24, 200, 300), (8760, 10, 10), (168, 50, 50)],
    read_slice=(slice(0, 24), slice(None), slice(None)),
    plot=True,
)

# Auto-optimize with Btune
cparams = xb.optimize_cparams(data, priority="balanced")
```

### Native blosc2 Access (`.b2` accessor)

```python
ds  = xb.open("data.b2")
arr = ds["temperature"].b2.array          # → blosc2.NDArray
print(arr.schunk.cparams)                 # codec, filters, clevel
```

## Tabular data (experimental)

xarray-blosc2's data model is multi-dimensional arrays (`.b2nd`). For
**columnar tabular data** — heterogeneous rows, block-indexed queries,
Parquet / Arrow interchange — the right primitive lives upstream in
[`blosc2.CTable`](https://blosc.org/python-blosc2/reference/ctable.html).
CTable provides `from_arrow` / `from_parquet` / `from_pandas`
constructors, `to_b2z` / `to_parquet` / `to_arrow` exporters, block-level
`create_index` for pruning, `where` / `group_by` / `sort_by` /
`add_computed_column` for compute-on-compressed, and tombstone-based
`delete` + `compact()`.

Read these upstream references before reaching for tabular in
xarray-blosc2:

- **API reference** — <https://blosc.org/python-blosc2/reference/ctable.html>
- **Tutorial** — <https://blosc.org/python-blosc2/tutorials/13.ctable-basics.html>
- **Benchmark blog post** (CTable + `.b2z` vs DuckDB / Parquet / PyArrow / polars / pandas on Chicago Taxi 24.3 M rows — cold-cache 1.9× faster than DuckDB, 2.5 % storage premium vs Parquet) — <https://blosc.org/posts/ctable-b2z-queries/>

xarray-blosc2 1.7 ships only the pandas-roundtrip helpers
(`save_dataframe` / `load_dataframe`) — no ctable-specific API. For
columnar tabular use today, reach for upstream `blosc2.CTable`
directly (the three links above).

**xarray-blosc2 1.8 will land a proper tabular story** — delegating
storage to upstream `blosc2.CTable`, adding new APIs (`render_view`,
hydra-zen config load, GeoJSON interchange, tritemporal ctable
wrapper) inside an opt-in `xarray_blosc2.experimental` subpackage. The stable public API
(`open`, `save_dataset`, `save_encrypted_dataset`, engine registration,
`save_dataframe` / `load_dataframe`) stays where it is; anything
whose shape may still evolve lives behind the `experimental` import,
signaling to users that it does not follow the stable semver contract.

xarray-blosc2's addition is what upstream doesn't cover — a CF-1.11
manifest, nested-group layout inside one `.b2z`,
`layers/features.geojson.zst` companion geometry, `views/` render
manifest, and `UncertainDataArray` pairs auto-recombined from
`{col}_mean`/`{col}_std`. That's the "xarray + orchestration cap over
upstream storage" split, the same way xarray-zarr relates to zarr.

Encryption follows the same env-var default as the rest of xarray-blosc2:
if `key=` is not passed, `_get_encryption_key()` walks
`B2_ENCRYPTION_KEY_{NAME}` → `B2_ENCRYPTION_KEY` → `REBOOT_CRYPTO_KEY`.
Setting `REBOOT_CRYPTO_KEY` once makes both xarray-blosc2 (whole-blob
sealing on `.b2z`) and `rbt.io.encryption` (columnar AES-GCM-V1 on
`.reb` Parquet) pick up the same 32-byte key — same key material,
different file framings. 1.8's `encrypt_columns=[...]` option on the
tabular writer defaults to the same fallback chain.

## CLI

```bash
python -m xarray_blosc2 --test                              # Self-test
python -m xarray_blosc2 --info data.b2                      # Inspect dataset
python -m xarray_blosc2 --serve --path /data --port 8081    # HTTP server
```

## Sample data & downloaders

A `.b2` archive of a well-known reanalysis is often the fastest way to
benchmark a solver, a compression preset, or a client-side viewer against
a "reference climate". xarray-blosc2 is well-suited to this — the archive
lives in one directory (or one `.b2z` file per ISO-week shard), keeps its
CF-1.11 metadata, streams from S3 or HTTP with Range requests, and stays
readable from Pyodide.

### ERA5 100 m wind climatology (30-year)

```bash
uv run scripts/download_era5_climatology.py --start 1991 --end 2020 --freeze
```

Prerequisites: a CDS API account and `~/.cdsapirc` with your token — see
https://cds.climate.copernicus.eu/api-how-to. Wall-clock is dominated by
CDS queue latency (hours to days depending on load); each year is a
separate ~800 MB GRIB request and blosc2 conversion takes a minute or two
per year.

Output lands at `./era5/grib/` (raw) + `./era5/blosc2/` (converted), or
under `$XB2_ERA5_DIR` if set, or `--out <path>`. Raw GRIB is kept by
default; pass `--no-keep-grib` to drop it after conversion (~15 GB saved).

With `--freeze`, each ISO-week shard is written as one `.b2z` file rather
than a directory of `.b2nd` files, making it drop-in readable via
`xr.open_mfdataset`:

```python
import xarray as xr

ds = xr.open_mfdataset(
    "era5/blosc2/era5_100m_wind_*/*.b2z",
    engine="blosc2",
    combine="by_coords",
)
```

`--test` fetches only 2020 for a smoke run; `--year YYYY` or `--start /
--end` for arbitrary ranges.

### AIFS forecast weeks

`sandbox/wind_index/download_aifs_week.py` (still in sandbox, not
promoted yet) fetches ECMWF AIFS forecast weeks — 100 m wind at 6-hourly
cadence to match the ERA5 climatology's grid. Pair with the ERA5 archive
above to compute wind-index anomalies.

## Format Conversion

```bash
python -m xarray_blosc2.convert input.nc   output.b2                          # NetCDF → blosc2
python -m xarray_blosc2.convert input.b2   output.nc                          # blosc2 → NetCDF
python -m xarray_blosc2.convert input.zarr output.b2                          # Zarr   → blosc2
python -m xarray_blosc2.convert input.grib output.b2                          # GRIB   → blosc2
python -m xarray_blosc2.convert input.b2   output.nczarr                      # blosc2 → NCZarr
python -m xarray_blosc2.convert input.b2   out.nc      --output-format nczarr # NCZarr with .nc ext
python -m xarray_blosc2.convert input.b2   output.b2   --preset geogrid       # Re-compress
```

NCZarr output writes Zarr v2 with `_NCZARR_SUPERBLOCK` metadata, readable by `ncdump` and any netCDF-C ≥ 4.8 tool.

## Testing

```bash
pytest tests/ -v                        # All 197 tests
pytest tests/ -v -m benchmark           # Memory + speed benchmarks
pytest tests/ -v --cov=xarray_blosc2    # With coverage
pytest tests/test_readme_examples.py    # Run every README code block
```

`tests/test_readme_examples.py` executes every code block in this README as a live test — the README is a CI contract, not a promise.

## Standalone / Pyodide

The library is a single file. For Pyodide or environments without pip:

```python
# Copy xarray_blosc2.py into your project, then:
import xarray_blosc2
```

The root `xarray_blosc2.py` includes a PEP 723 header for direct execution with `uv`:

```bash
uv run xarray_blosc2.py --test
```

## Dependencies

| Package        | Version    | Required | Feature                       |
|----------------|-----------:|:--------:|-------------------------------|
| `xarray`       | ≥ 2024.9.0 |   yes    | core                          |
| `blosc2`       | ≥ 3.12.0   |   yes    | core                          |
| `numpy`        | ≥ 1.26.0   |   yes    | core                          |
| `msgpack`      | latest     |    no    | blob serialization, VersionedStore |
| `cryptography` | ≥ 41.0     |    no    | encryption                    |
| `blosc2-grok`  | latest     |    no    | JPEG2000 lossy                |
| `blosc2-btune` | latest     |    no    | auto-tuning (Btune)           |
| `zstandard`    | ≥ 0.22.0   |    no    | ZSTD dictionary               |
| `caterva2`     | latest     |    no    | remote access                 |
| `uncertainties`| latest     |    no    | ufloat interop                |
| `pandas`       | latest     |    no    | DataFrame I/O                 |
| `boto3`        | latest     |    no    | S3 storage backend            |
| `icechunk`     | ≥ 0.2      |    no    | icechunk bridge               |

**Author:** Rolv Erlend Bredesen
**License:** BSD-3-Clause
