Metadata-Version: 2.4
Name: drb-chunk
Version: 0.6.1
Summary: DRB N-D lazy chunk access add-on
Author: GAEL Systems
Author-email: drb-python@gael.fr
License: LGPLv3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Environment :: Plugins
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Requires-Python: <3.14,>=3.11
Description-Content-Type: text/markdown
License-File: LICENCE.txt
Requires-Dist: drb<3,>=2.3.5
Requires-Dist: drb-extractor~=1.3.0
Requires-Dist: rdflib
Requires-Dist: numpy
Requires-Dist: xarray
Requires-Dist: dask
Dynamic: license-file

# drb-chunk

DRB N-D lazy chunk access add-on. It exposes N-dimensional, lazily-read data
chunks (raster tiles, windows, byte ranges, time-series blocks) as first-class
DRB objects, driven **declaratively** from a topic's RDF/Turtle descriptor and
materialised on demand through `numpy` or a dask-backed `xarray`.

The add-on reads nothing until you ask for bytes. Selection (`select`,
`__getitem__`) only narrows a manifest; the actual I/O happens when you call
`get_impl(...)` / `to_xarray()`, always through a coalesced, physically-aligned
window read (`RemapEngine`) — the add-on refuses to silently fall back to a
full read.

```bash
pip install drb-chunk
```

Requires Python 3.11–3.13. The core itself has **no format dependency** (no
rasterio, no GDAL): `numpy`, `dask` and `xarray` back lazy assembly; the actual
content read comes from a format add-on's leaf reader (e.g. `drb-chunk-image`
pulls in `rasterio`), needed only at materialisation time.

---

## Why it exists

DRB drivers already turn a file or URL into a tree of `DrbNode`s. But a single
raster band or a large array is not naturally a "node" — it is an N-D grid you
want to slice cheaply, possibly remotely, without loading the whole thing.

`drb-chunk` adds that missing layer:

- **Declarative** — a topic says, in its `cortex.ttl`, *what* its chunks are
  (dims, dtype, transformer, where the bytes live). No code per product.
- **Lazy** — narrowing a chunk is pure metadata arithmetic over the
  transformer. I/O is deferred to materialisation.
- **Driver-reusing** — a format's leaf reader reads through the source
  node's **existing** `get_impl` (e.g. `rasterio.io.DatasetReader`). No
  driver is modified to support chunking.

---

## Core concepts

| Object | Role |
| --- | --- |
| `ChunkAddon` | Entry point (`drb.addon` = `chunk`). Reads a topic's descriptors and builds `Chunk`s from a node. |
| `ChunkDescriptor` | The parsed `drb:chunk` declaration: name, source, dims, dtype, transformer, optional selection/collection. |
| `Chunk` | A handle on one declared chunk of a node. Slice it, then materialise it. |
| `ChunkArray` | Pure metadata of the array: `dims`, `shape`, `dtype`, `transformer`. |
| `Transformer` / `WindowingTransformer` / `RegularGrid` | The user-facing representation layer: geometry + how a `Selection` resolves into tile keys. |
| `ChunkManifest` | Lazy `Mapping[key -> ChunkRef]` over the transformer's keys. |
| `ChunkRef` | One tile's locator: `key`, `source` node, and a `window` *or* `byte_range`. |
| `Selection` | Serialisable, declarative constraint (`isel`, `sel`, `window`, `band`, `range`). |
| `RemapEngine` | Internal mechanics: coalesces a logical window onto physical-chunk boundaries, reads once, caches, crops. |
| `PhysicalChunkModel` / `PhysicalTilingProvider` | The source's native chunking (unit of I/O), discovered from a format add-on, never declared in a product descriptor. |

Data flow:

```
topic (cortex.ttl)                       node (DrbNode)
      │  drb:chunk …                            │
      ▼                                          │
ChunkDescriptor ──ChunkAddon.apply(node)────────►  Chunk (Transformer + RemapEngine)
                                                   │  .select(Selection)   (lazy: subsets the manifest)
                                                   ▼
                                                 Chunk (narrowed)
                                                   │  .get_impl(np.ndarray) / .to_xarray()
                                                   ▼
                                           RemapEngine.materialize(window)  ──►  numpy / xarray
                                           (coalesced read via the format's leaf reader)
```

---

## Architecture

The add-on is organised in **three layers**, each ignoring the details of the
one below it and each with its own extension seam. See
[`docs/dev/architecture-layers.md`](docs/dev/architecture-layers.md) for the
full write-up; the summary:

```
┌─────────────────────────────────────────────────────────────────────┐
│  TRANSFORMER  (user-facing, drb/chunk/transformer.py)                 │
│  Transformer / WindowingTransformer / RegularGrid                    │
│  TransformerProvider  ← entry point drb.chunk.transformer              │
│  descriptor.py  ChunkDescriptor + retrieve_chunks()  ←── cortex.ttl   │
│  selection.py   Selection (isel/window/band/range/sel) + aggregator   │
│        "what geometry a chunk presents; how a Selection becomes keys"  │
├─────────────────────────────────────────────────────────────────────┤
│  LAZINESS (metadata only, zero bytes read)                            │
│  model.py       ChunkArray, ChunkManifest, ChunkRef                   │
│  chunk.py       Chunk  (select / __getitem__ / tiles / locator)       │
│        "a handle you narrow; a lazy key→ref manifest"                  │
├─────────────────────────────────────────────────────────────────────┤
│  MECHANICS  (internal, not user-facing, drb/chunk/engine.py)          │
│  RemapEngine   ← built via BuildContext.build_engine                  │
│  remap.py       aligned_window / plan_remap / materialize              │
│  cache.py       BytesLRU (per-source region cache) + plan_cache        │
│        "coalesce a logical window onto physical-chunk boundaries,      │
│         read once, cache, crop"                                        │
├─────────────────────────────────────────────────────────────────────┤
│  PHYSICAL  (format add-on, drb/chunk/physical.py)                     │
│  PhysicalChunkModel(chunk_shape, reader, array_shape)                 │
│  PhysicalTilingProvider  ← entry point drb.chunk.physical               │
│  leaf reader  ← register_physical_reader(token, fn)                   │
│  chunk.py       get_impl(numpy) / to_xarray()  + interop.to_kerchunk  │
│        "the source's native chunking; how to read one native block"    │
└─────────────────────────────────────────────────────────────────────┘
```

The guiding rule, enforced in the code: **nothing is read before
materialisation** — `select()` is pure key arithmetic; `RemapEngine.materialize()`
(via the format's leaf reader) is the first place that touches the source.

### Declaration & discovery

`ChunkAddon` (`core.py`) is a singleton registered under the `drb.addon` entry
point; DRB loads it into `AddonManager`. It implements the `Addon` contract
(`identifier`, `return_type`, `can_apply`, `apply`). Its core job is the **build
pipeline** behind `apply()`:

```
topic ──retrieve_chunks()──► {name: ChunkDescriptor}
                                   │  for the requested chunk:
cd.source.extract(node) ───────────┤  where are the bytes?
_resolve_source() ─────────────────┤  "." → the node itself ; else resolver.create(url)
_build_engine(source_node) ────────┤  physical model + RemapEngine (or None)
                                   ▼
ChunkArray(dims, shape, dtype, transformer)
RegularGridManifest(array, source_node)
                                   ▼
       Chunk(name, array, node, manifest, engine, topic_uri)
```

`retrieve_chunks()` (`descriptor.py`) **inherits descriptors through
`rdfs:subClassOf`** (recursing into parents) then **overrides** them with the
topic's own chunks, keyed by `drb:chunkName` — the same pattern as
`MetadataAddon`. It reads the public RDF graph exposed by the topic's `ManagerDao`.

> A descriptor-level `drb:selection` is **rejected** at build time; apply
> selections explicitly via `Chunk.select()`.

### Transformer — geometry and selection

This is "the only place grid geometry lives" for a windowing transformer
(`WindowingTransformer` docstring). Two responsibilities: enumerate tile keys
(`keys(array)`) and resolve a selection into keys + residual
(`resolve(selection, array) → ResolvedSelection`). `RegularGrid` is the one
built-in implementation:

```
grid_shape = ceil(shape / chunk_shape)   per dim
keys       = itertools.product(range(n) for n in grid_shape)

resolve(WindowSelection(x,y,w,h)):
    x → [x, x+w)   y → [y, y+h)
    per dim:  first = start // chunk ;  last = (stop-1) // chunk
    keys = product(range(first, last+1) ...)
```

`Selection` (`selection.py`) is deliberately **pure, serialisable data**
(`to_dict()` / `parse_selection()`) — it does *not* know how to turn itself into
keys; that is the transformer's job. This separation is what makes a `Chunk`
serialisable (locator) and a selection replayable. `SelectionAggregator` composes
several per-dimension constraints. A content-driven transformer (geometry only
known from the product, e.g. Sentinel-1 `burst`) is supplied by an add-on's
`TransformerProvider` — see
[`docs/dev/writing-a-transformer.md`](docs/dev/writing-a-transformer.md).

### Laziness

- `ChunkArray` — pure metadata (`dims`, `shape`, `dtype`, `transformer`). No bytes.
- `ChunkManifest` — a **lazy** `Mapping[key → ChunkRef]`. `RegularGridManifest.ref(key)`
  *computes* the window on the fly (`key * chunk_shape`); it stores nothing.
- `ChunkRef` — one tile's locator: `key`, `source` node, and **either** a
  `window` (the mainstream, format-native path) **or** a `byte_range` (used by
  `to_kerchunk` export, see *Locators and interop* below).
- `Chunk.select()` — laziness in action: resolve the selection into keys, then
  `manifest.subset(resolved)` returns a `_SubsetManifest` keeping only those keys,
  wrapped in a new immutable `Chunk`. **No read happens.**

### Mechanics — `RemapEngine`

Materialisation is the only moment bytes are read, and it always goes
through a **`RemapEngine`** (`drb/chunk/engine.py`) bound to the chunk's
source node — internal plumbing, not a transformer, not user-facing, never
constructed by hand. For a requested logical window, `RemapEngine.materialize`:

```
aligned_window(window, phys_chunk_shape, array_shape)  →  physically-aligned region
BytesLRU.get_or_read((source_id, region), lambda: leaf_reader(node, region))
crop the coalesced region down to the requested window
```

so adjacent logical tiles sharing one native (physical) block only pay for
that block once. A `RemapEngine` is built once per source via
`BuildContext.build_engine`, which resolves the source's `PhysicalChunkModel`
(next layer) and returns `None` when the format declares none — in which
case the chunk has no engine and cannot be materialised (enumeration,
`locator()` and `to_kerchunk` still work without one).

### Physical — native chunking and the leaf reader

`PhysicalTilingProvider.probe(node, multiplier=1)` (a format add-on) opens the
source through its existing `get_impl` and returns a `PhysicalChunkModel`
(`chunk_shape`, `reader` token, `array_shape`) describing its native
chunking — discovered from the implementation, never declared in a product
descriptor. The paired **leaf reader**, `fn(node, phys_window) -> ndarray`
registered via `register_physical_reader(token, fn)`, is **the only
content-reading path for that format**: the core ships none itself and never
imports a format library. See
[`docs/dev/writing-a-physical-tiling-provider.md`](docs/dev/writing-a-physical-tiling-provider.md).

From `Chunk` there are two exits: `get_impl(np.ndarray)` (single-tile direct
read through the engine) and `to_xarray()` (dask-backed
`da.from_delayed(dask.delayed(engine.materialize))`, dims `("band",) +
array.dims`). **No driver is modified** to support chunking.

### Design decisions to remember

| Decision | Why |
| --- | --- |
| Selection = pure data, resolution = transformer | makes `Chunk` serialisable and selections replayable; one home for geometry |
| Lazy manifest (`ref()` computes the window) | no materialised tile list for very large grids |
| Physical model discovered, not declared; leaf reader is the only content-reading path | zero driver changes; chunking is an additive layer, format libraries stay out of the core |
| Reads always coalesced onto physical-chunk boundaries via `RemapEngine` | adjacent logical tiles sharing a native block pay for it once |
| No fallback to a full read | guarantees a chunk stays a chunk (no accidental full read) |
| Explicit v1 deferrals with clear errors | bounded scope: `regular` transformer + provider-supplied windowing transformers only, single-tile materialisation; `drb:selection` in descriptor rejected |

---

## Declaring chunks in a topic (`cortex.ttl`)

Chunks are declared on a `DrbTopic` with the `drb:chunk` predicate. Each
`drb:chunk` blank node describes one chunk. Descriptors are **inherited** through
`rdfs:subClassOf` and a child topic may override a parent's chunk by reusing its
`drb:chunkName`.

```turtle
@prefix owl:  <http://www.w3.org/2002/07/owl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix drb:  <http://www.gael.fr/drb#> .

drb:raster-base a owl:Class ;
    rdfs:label "raster-base" ;
    drb:chunk [
        drb:chunkName    "data" ;
        drb:source       "." ;          # bare literal: the node itself
        drb:dims         ( "y" "x" ) ;
        drb:dtype        "uint16" ;
        drb:transformer  "regular" ;    # optional: "regular" is the default
        drb:tileHeight   512 ;
        drb:tileWidth    512
    ] .

drb:my-image a owl:Class ;
    rdfs:label "my-image" ;
    rdfs:subClassOf drb:raster-base ;   # inherits "data", adds "b04"
    drb:chunk [
        drb:chunkName    "b04" ;
        drb:source       [ drb:xquery
            "GRANULE/*/IMG_DATA/R10m/*[fn:matches(fn:name(),'.*_B04_10m\\.jp2$')]"
        ] ;                             # typed blank node: XQuery navigates the product
        drb:dims         ( "y" "x" ) ;
        drb:dtype        "uint16" ;
        drb:chunkShape   ( 256 256 )    # equivalent to tileHeight/tileWidth
    ] .
```

### `drb:chunk` vocabulary

| Predicate | Meaning | Required |
| --- | --- | --- |
| `drb:chunkName` | Unique chunk identifier within the topic. | yes |
| `drb:source` | Where the bytes are. **Bare literal**: `"."` / `""` = the node itself; a path/URL resolved through DRB (`ConstantExtractor`). **Typed blank node**: `[ drb:xquery "…" ]` — an XQuery evaluated against the product node, returning the band `DrbNode` (`XQueryExtractor`). Also accepts `drb:python`, `drb:script`, `drb:constant` via `parse_extractor`. Note: drb's XQuery engine uses full-match semantics, so patterns must carry a leading `.*` (e.g. `fn:matches(fn:name(),'.*_B04_10m\\.jp2$')`). | yes |
| `drb:dims` | RDF list of dimension names, e.g. `( "y" "x" )`. | yes |
| `drb:dtype` | NumPy dtype string, e.g. `"uint16"`. | yes |
| `drb:transformer` | Transformer name (`drb:transformer` token). Defaults to `"regular"` when omitted and the chunk declares a `drb:chunkShape`/`drb:tileHeight`+`drb:tileWidth`; omit entirely (no shape either) to delegate the grid to the source's physical chunk model. Non-`"regular"` values dispatch to a `TransformerProvider` (see [`docs/dev/writing-a-transformer.md`](docs/dev/writing-a-transformer.md)). Can be overridden per-build via `apply(transformer=…)`. | no |
| `drb:chunkShape` | RDF list giving the tile shape per dim. | one of chunkShape / (tileHeight + tileWidth) / delegated to the physical model |
| `drb:tileHeight`, `drb:tileWidth` | Convenience for 2-D `(y, x)` grids; equivalent to `drb:chunkShape ( h w )`. | |
| `drb:nativeMultiplier` | Integer scaling the source's native (physical) chunk unit up before it drives reads (e.g. read 2×2 native tiles as one physical chunk). Defaults to `1`. | no |
| `drb:collection` | Logical group name. Chunks sharing the same `drb:collection` value can be built together with `apply(collection=…)`. | no |
| `drb:selection` | **v1 deferral** — a descriptor-level default selection is rejected at build time; apply selections explicitly via `Chunk.select()`. | no |

> The legacy `drb:tilingScheme` / `drb:reader` predicates are **no longer
> supported**: a descriptor still carrying either raises `DrbChunkError` at
> load time ("upgrade the descriptor to `drb:transformer`"). Reading is now
> always mediated by the format's registered physical leaf reader through
> `RemapEngine`, not a per-chunk reader hint.

### Collections

A **collection** groups related chunks under a shared name so callers can build
all of them in one call. Declare it with `drb:collection` on each chunk that
belongs to the group:

```turtle
drb:sentinel2-l2a-r10m a owl:Class ;
    rdfs:label "sentinel2-l2a-r10m" ;
    drb:chunk [
        drb:chunkName  "B02" ;
        drb:source     [ drb:xquery
            "GRANULE/*/IMG_DATA/R10m/*[fn:matches(fn:name(),'.*_B02_10m\\.jp2$')]"
        ] ;
        drb:dims       ( "y" "x" ) ;
        drb:dtype      "uint16" ;
        drb:tileHeight 512 ; drb:tileWidth 512 ;
        drb:collection "R10m"
    ] ;
    drb:chunk [
        drb:chunkName  "B04" ;
        drb:source     [ drb:xquery
            "GRANULE/*/IMG_DATA/R10m/*[fn:matches(fn:name(),'.*_B04_10m\\.jp2$')]"
        ] ;
        drb:dims       ( "y" "x" ) ;
        drb:dtype      "uint16" ;
        drb:tileHeight 512 ; drb:tileWidth 512 ;
        drb:collection "R10m"
    ] .
```

`ChunkAddon` exposes two collection-aware API calls:

```python
# Map collection name -> list of chunk names (None key = ungrouped chunks).
chunk_addon.available_collections(topic)
# {'R10m': ['B02', 'B04'], None: ['QI']}

# Build every chunk in a collection.
chunks = chunk_addon.apply(node, collection="R10m", topic=topic)
# -> [Chunk("B02"), Chunk("B04")]
```

`apply(collection=…)` raises `DrbChunkError` listing available collections if
the requested name is unknown. `chunk_name` and `collection` are mutually
exclusive — passing both raises `DrbChunkError`.

---

## Quick start (through the DRB resolver)

```python
from drb.topics import resolver
from drb.addons.addon import AddonManager

# Resolve any source DRB knows how to type.
topic, node = resolver.resolve("/data/S2/IMG_DATA/T31TCJ_B04.jp2")

chunk_addon = AddonManager().get_addon("chunk")   # the registered singleton

if chunk_addon.can_apply(topic):
    # What chunks does this topic declare?
    for name, transformer in chunk_addon.available_chunks(topic):
        print(name, transformer)       # ('data', {'regular': {'chunk_shape': [512, 512]}})

    # What collections are declared? (None key = ungrouped chunks)
    print(chunk_addon.available_collections(topic))
    # {'R10m': ['B02', 'B04'], None: ['QI']}

    # Build one named chunk (or omit chunk_name to get a list of all chunks).
    chunk = chunk_addon.apply(node, chunk_name="data")

    # Build every chunk in a collection.
    chunks = chunk_addon.apply(node, collection="R10m")
```

`apply()` returns a single `Chunk` when `chunk_name` is given, a `list[Chunk]`
when `collection` is given, or a `list[Chunk]` of every declared chunk when
neither is given. It raises `DrbChunkError` if the topic declares no chunk, if
`chunk_name` is unknown, if `collection` is unknown, or if both are given.

---

## Working with a `Chunk`

### Inspect the grid

```python
chunk.name              # "data"
chunk.array.dims        # ("y", "x")
chunk.array.shape       # (10980, 10980)   — inferred from the source raster
chunk.array.dtype       # "uint16"
chunk.grid_shape        # (22, 22)         — ceil(shape / chunk_shape), RegularGrid only

list(chunk.tiles())     # [(0, 0), (0, 1), …]   — tile keys
chunk.tile((0, 0))      # ChunkRef(key=(0,0), source=…, window=((0,512),(0,512)))
```

### Select (lazy — reads nothing)

`select` resolves the selection into tile keys via the transformer and returns
a new `Chunk` over the narrowed manifest. `chunk[sel]` is sugar for
`chunk.select(sel)`.

```python
from drb.chunk.selection import WindowSelection, IselSelection, BandSelection

# A pixel window (x, y, w, h); maps to the x/y dims of a RegularGrid.
roi = chunk.select(WindowSelection(x=512, y=0, w=512, h=512))

# Integer-position selection per dim: int -> single index, [start, stop] -> range.
sub = chunk.select(IselSelection({"y": [0, 1024], "x": [0, 512]}))
```

### Materialise (this is where I/O happens)

```python
import numpy as np

# Single tile -> numpy. Multi-tile numpy materialisation is rejected on purpose.
arr = roi.get_impl(np.ndarray)          # windowed read via RemapEngine, shape (bands, h, w)

# Lazy, dask-backed xarray.DataArray (dims = ("band",) + array.dims).
xda = roi.to_xarray()
xda = roi.get_impl(__import__("xarray").DataArray)   # equivalent
```

`get_impl(np.ndarray)` requires the selection to resolve to a **single tile** —
otherwise it raises and points you to `to_xarray()`. (In v1, `to_xarray()` itself
also only assembles a single tile; see *Limitations*.)

---

## More usage examples

### Manual construction with a `RemapEngine`

A `Chunk` materialises through a `RemapEngine` bound to its source node — a
`Chunk` built without one (`engine=None`) still supports enumeration,
`select()` and `locator()`, but raises `DrbChunkError` on
`get_impl`/`to_xarray`. Building both by hand (no topic/descriptor involved)
mirrors what `ChunkAddon._build` does internally (see
[`docs/dev/architecture-layers.md`](docs/dev/architecture-layers.md)); the
windowed read returns exactly the same data as a full slice, without
loading the whole source, and adjacent tiles sharing a native (physical)
block only pay for it once (proven in `tests/test_chunk.py` and
`tests/test_engine.py`):

```python
import numpy as np

from drb.chunk.cache import BytesLRU
from drb.chunk.chunk import Chunk
from drb.chunk.engine import RemapEngine
from drb.chunk.model import ChunkArray, RegularGridManifest
from drb.chunk.physical import PhysicalChunkModel, register_physical_reader
from drb.chunk.selection import WindowSelection
from drb.chunk.transformer import RegularGrid

# The leaf reader is the format add-on's only content-reading path; here a
# stand-in for e.g. drb-chunk-image's rasterio-backed "raster" reader.
def read_block(node, phys_window):
    (y0, y1), (x0, x1) = phys_window
    return node.get_impl(np.ndarray)[:, y0:y1, x0:x1]

register_physical_reader("my-reader", read_block)

model = PhysicalChunkModel(chunk_shape=(512, 512), reader="my-reader",
                           array_shape=(1024, 1024))
engine = RemapEngine(node, model, (1024, 1024), cache=BytesLRU(64 * 2**20))

array = ChunkArray(dims=("y", "x"), shape=(1024, 1024), dtype="uint16",
                   transformer=RegularGrid(chunk_shape=(512, 512)))
chunk = Chunk(name="data", array=array, node=node,
              manifest=RegularGridManifest(array, node), engine=engine)

out = chunk.select(WindowSelection(x=512, y=0, w=512, h=512)).get_impl(np.ndarray)
# out == full_data[:, 0:512, 512:1024]
```

In practice you rarely construct a `RemapEngine` yourself: `ChunkAddon.apply()`
builds it from the topic's `drb:chunk` declaration and the source's
`drb:physicalTiling`, and a `TransformerProvider.build()` gets one through
`ctx.build_engine(...)`.

### Lazy xarray / dask compute

```python
xda = roi.to_xarray()              # dask-backed DataArray, dims ("band", "y", "x")
result = xda.mean().compute()      # the deferred read fires only on .compute()
```

---

## Selection types

All selections are pure, serialisable data (`to_dict()` / `parse_selection()`);
turning them into tile keys is the transformer's job (`WindowingTransformer.resolve`).

| Type | `to_dict()` shape | Resolvable by `RegularGrid` in v1 |
| --- | --- | --- |
| `IselSelection(per_dim)` | `{"isel": {"y": [0,1024], "x": 5}}` | ✅ |
| `WindowSelection(x,y,w,h)` | `{"window": {"x":…, "y":…, "w":…, "h":…}}` | ✅ (maps to x/y dims) |
| `SelSelection(per_dim)` | `{"sel": {…}}` (label/coord based) | ❌ (needs coords) |
| `BandSelection(bands)` | `{"band": [0, 3]}` | ❌ |
| `RangeSelection(offset,length)` | `{"range": {"offset":…, "length":…}}` | ❌ |
| `SelectionAggregator(parts)` | merged keys of its parts | per-part |

Round-tripping:

```python
from drb.chunk.selection import parse_selection

sel = parse_selection({"window": {"x": 0, "y": 0, "w": 256, "h": 256}})
sel.to_dict()           # {'window': {'x': 0, 'y': 0, 'w': 256, 'h': 256}}

# Several keys -> a SelectionAggregator composing leaf constraints.
parse_selection({"isel": {"y": [0, 512]}, "band": [0]})
```

Out-of-bounds or unknown-dimension selections raise `DrbSelectionError`.

---

## Materialisation: `RemapEngine` and the physical leaf reader

A `Chunk` materialises through the `RemapEngine` bound to its source node
(`Chunk(engine=...)`), not through a per-chunk reader hint (there is no
`drb:reader` predicate any more). For the tile(s) a selection resolves to,
`RemapEngine.materialize(window)`:

1. expands the window to the enclosing physically-aligned region
   (`drb/chunk/remap.py`'s `aligned_window`, clamped to the source's
   `PhysicalChunkModel.array_shape`);
2. reads that region **once** through the format's registered physical leaf
   reader (`get_physical_reader(model.reader)`) — the format add-on's only
   content-reading path; the core never imports a format library and calls
   no I/O itself;
3. caches it in a per-source `BytesLRU` region cache, so adjacent tiles
   sharing a native block only pay for it once;
4. crops the cached region down to the requested window.

A `Chunk` built with `engine=None` (no physical model on its source) is
still valid for enumeration, `select()` and `locator()`, but raises
`DrbChunkError` on `get_impl`/`to_xarray` — the add-on never silently falls
back to reading the whole source. See
[`docs/dev/architecture-layers.md`](docs/dev/architecture-layers.md) and
[`docs/dev/writing-a-physical-tiling-provider.md`](docs/dev/writing-a-physical-tiling-provider.md)
for how a format add-on supplies that leaf reader.

---

## Locators and interop

```python
chunk.locator()
# {'source': '/data/…/B04.tif', 'topic': 'http://…#my-image',
#  'chunk': 'b04', 'selection': {'window': {...}} | None}

from drb.chunk import to_kerchunk
to_kerchunk(chunk)      # {'version': 1, 'refs': {'1.0': [path, offset, length], …}}
```

`to_kerchunk` emits a [kerchunk](https://fsspec.github.io/kerchunk/) reference-spec
v1 dict — a **physical-address annex**, orthogonal to the read path above.
Because kerchunk addresses chunks by byte range, it only works for a
`ChunkManifest` whose `ChunkRef.byte_range` is set; window-only
(format-native) chunks, which is what every manifest shipped in the core
produces, raise `DrbChunkError`. Populating `byte_range` is a manifest's
choice (e.g. a content-driven add-on's `ChunkManifest`), independent of
whether the chunk also has a `RemapEngine` for direct reads.

---

## Exceptions

```python
from drb.chunk import DrbChunkError, DrbSelectionError
```

- `DrbChunkError` — base error for the add-on (unknown chunk name, no physical
  model/engine for materialisation, unsupported `get_impl`, v1 deferrals, …).
  Subclasses `drb.exceptions.core.DrbException`.
- `DrbSelectionError` — unknown selection type or an out-of-bounds region. Subclasses `DrbChunkError`.

---

## v1 limitations (intentional deferrals)

These are explicit, raise clear errors, and are tracked as follow-ups:

- **Transformer**: `regular` (`RegularGrid`) is the only built-in transformer;
  content-driven transformers (e.g. Sentinel-1 `burst`) come from add-ons.
- **Multi-tile assembly**: `to_xarray()` assembles a single tile; concat/mosaic
  across tiles is not yet implemented. `get_impl(np.ndarray)` likewise requires a
  single-tile selection.
- **Descriptor-level `drb:selection`**: a default selection in the descriptor is
  rejected at build time — apply selections explicitly via `Chunk.select()`.
- **`Chunk.from_locator()`**: locator round-trip is deferred (`locator()` works).
- **Label/coord & band/range resolution**: `SelSelection`, `BandSelection` and
  `RangeSelection` are not resolved by `RegularGrid` in v1.
- **Resampling transformers**: a non-windowing transformer kind (e.g. a
  healpix resampling representation) would need its own mechanic distinct
  from `RemapEngine`; not designed or implemented yet.

---

## Public API

```python
from drb.chunk import (
    Chunk, ChunkAddon, ChunkArray, ChunkRef, ChunkManifest,
    Selection, parse_selection,
    Transformer, WindowingTransformer, RegularGrid, RemapEngine,
    TransformerProvider, BuildContext,
    register_transformer, get_transformer,
    PhysicalChunkModel, PhysicalTilingProvider,
    register_physical_provider, get_physical_provider,
    build_source_extractor,
    to_kerchunk, DrbChunkError, DrbSelectionError, __version__,
)

# ChunkAddon public methods (singleton via AddonManager().get_addon("chunk")):
#   .can_apply(topic)                                    -> bool
#   .available_chunks(topic)                             -> list[tuple[str, dict]]
#   .available_collections(topic)                        -> dict[str | None, list[str]]
#   .apply(node, *, chunk_name=…, transformer=…)          -> Chunk
#   .apply(node, *, collection=…, transformer=…)          -> list[Chunk]
#   .apply(node)                                          -> list[Chunk]  (all declared chunks)
```

The physical leaf-reader registry (`register_physical_reader`/
`get_physical_reader`) lives in `drb.chunk.physical` alongside
`PhysicalTilingProvider` but is not re-exported at the top-level package;
import it from `drb.chunk.physical` directly (as every format add-on does).

## Development

```bash
python3 -m venv venv && source venv/bin/activate
pip install -e . -r requirements-test.txt
python3 -m unittest discover        # unittest suite under tests/
```

### Three layers, two extension seams

`drb-chunk` is organised in three layers — **transformer** (user-facing
representation), **mechanics** (`RemapEngine`, internal), **physical**
(the source's native chunking) — see
[`docs/dev/architecture-layers.md`](docs/dev/architecture-layers.md) for the
overview. Add-ons extend the two outer layers:

### Extending: content-driven transformers

`regular` is the only transformer built into the core. An add-on can teach
the engine a new one — geometry read from the product itself at
apply-time — by shipping a `drb.chunk.transformer` entry point. See
[`docs/dev/writing-a-transformer.md`](docs/dev/writing-a-transformer.md)
for the `TransformerProvider` contract and a worked example;
`drb-chunk-sentinel1` (the `burst` transformer) is the reference
implementation.

### Extending: physical-tiling providers

`drb-chunk` discovers a source's *native* chunking (the unit of I/O) rather
than taking it from the descriptor. An add-on teaches the engine a new
format's native chunking by shipping a `drb.chunk.physical` entry point and
attaching `drb:physicalTiling "<token>"` to the format's topic. See
[`docs/dev/writing-a-physical-tiling-provider.md`](docs/dev/writing-a-physical-tiling-provider.md)
for the `PhysicalTilingProvider` contract and a worked example;
`drb-chunk-image` (the `gdal-blocks` provider, for GDAL rasters) is the
reference implementation.

### Add-on naming convention

A chunk add-on's **PyPI dist name** is `drb-chunk-<name>` and its **Python
module** is `drb.addons.chunk.<name>`. The product suffix keeps the
**unhyphenated** form — `sentinel1`, `sentinel2` — matching the sibling DRB
families (`drb-iceberg-sentinel1`, `drb-image-sentinel1`,
`drb-metadata-sentinel1`, …); the git repository directory is historically
`sentinel-1`/`sentinel-2` and is left as-is. Two kinds of add-on live under
`add-ons/`:

- **product** add-ons (`drb-chunk-sentinel1`, `drb-chunk-sentinel2`) —
  content-driven transformers / `drb:chunk` descriptors;
- **format** add-ons (`drb-chunk-image`, and future `drb-chunk-netcdf` /
  `drb-chunk-zarr`) — physical-tiling providers, one per implementation that
  exposes a native chunking.

## License

LGPLv3 — see `LICENCE.txt`.
