Metadata-Version: 2.4
Name: zarr-vectors
Version: 0.9.2
Summary: Python utilities for Zarr Vectors — cloud-native storage for points, lines, streamlines, graphs, and meshes.
Author: Andrew Keenlyside, Forrest Collman
License: BSD-3-Clause
Project-URL: Homepage, https://github.com/AllenInstitute/zarr-vectors-py
Project-URL: Documentation, https://zarr-vectors-py.readthedocs.io
Project-URL: Repository, https://github.com/AllenInstitute/zarr-vectors-py
Project-URL: Issues, https://github.com/AllenInstitute/zarr-vectors-py/issues
Project-URL: Specification, https://github.com/AllenInstitute/zarr_vectors
Keywords: zarr,point-cloud,mesh,skeleton,streamline,connectomics,spatial
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: numcodecs>=0.12
Requires-Dist: zarr>=3.0
Provides-Extra: draco
Requires-Dist: DracoPy>=1.3; extra == "draco"
Provides-Extra: obstore
Requires-Dist: obstore>=0.3; extra == "obstore"
Provides-Extra: fsspec
Requires-Dist: fsspec>=2024.0; extra == "fsspec"
Requires-Dist: s3fs>=2024.0; extra == "fsspec"
Requires-Dist: gcsfs>=2024.0; extra == "fsspec"
Requires-Dist: adlfs>=2024.0; extra == "fsspec"
Provides-Extra: icechunk
Requires-Dist: icechunk>=2; python_version >= "3.12" and extra == "icechunk"
Provides-Extra: cloud
Requires-Dist: zarr-vectors[obstore]; extra == "cloud"
Provides-Extra: all
Requires-Dist: zarr-vectors[draco,fsspec,icechunk,obstore]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: linkml<2,>=1.7; extra == "dev"
Requires-Dist: jsonschema<5,>=4.0; extra == "dev"
Dynamic: license-file

> [!NOTE]
> This package is under active development.

<img src="assets/zarr-vectors.png" alt="zarr-vectors" width="60%" />

**Tools for Zarr Vectors Data**

`zarr-vectors-py` is a Python package for reading, writing, and managing large-scale vector geometry data in the Zarr Vectors format — a chunked, cloud-native format built on Zarr v3 for multiscale points, lines, streamlines, graphs, skeletons, and meshes.

The package supports supervoxel-level spatial binning with separated chunk and bin sizes, per-level object sparsity for balanced multi-resolution pyramids, and OME-Zarr-compatible multiscale metadata.

*Aligned to the Zarr Vectors specification by Forrest Collman, Allen Institute for Brain Sciences*
[Link to specification GitHub](https://github.com/AllenInstitute/zarr_vectors)

---
## Documentation

**link to readthedocs:** https://zarr-vectors-py.readthedocs.io/en/latest

---

## Install

```bash
pip install zarr-vectors
```

---

## Quick start

```python
import numpy as np
import zarr_vectors as zv

# A Schema says what the data is; a Layout says how it is cut up.
schema = zv.Schema(
    bounds=([0.0, 0.0, 0.0], [1000.0, 1000.0, 1000.0]),
    kind="point_cloud",
    layout=zv.Layout(cells=8),  # ~8 chunks per axis
)

ds = zv.create("demo.zarrvectors", schema=schema)

rng = np.random.default_rng(0)
positions = rng.uniform(0, 1000, size=(10_000, 3)).astype("float32")
ds.add_points(
    positions,
    attributes={"intensity": rng.random(10_000).astype("float32")},
    object_ids=np.repeat(np.arange(100), 100),
)
ds.build_pyramid(factors=[(2.0, 1.0)])  # add a coarser level

# Read back only what falls inside a box.
result = zv.open("demo.zarrvectors").select(bbox=([0, 0, 0], [250, 250, 250])).read()

print(result.kind, result.positions.shape, result.attributes.names())
print(result.attributes["intensity"][:3])
```

```
point_cloud (122, 3) ('intensity',)
[0.58416754 0.17340323 0.28906345]
```

A read always comes back as a `ReadResult`, whatever the geometry: `positions` is
`(N, D)`, `parts` slices it into polylines or segments, and `edges` / `faces` /
`attributes` are filled in when the store carries them. Per-vertex `object_ids`
are populated only on the reader paths that supply them — a whole-store read of a
point cloud leaves them `None` even when ids were written. Reach for
`level.objects[...]` to read by id.

There are two supported surfaces. `zarr_vectors.api` — re-exported from the top-level
package, which is what the example above uses — is for reading and writing data.
`zarr_vectors.building` is for tools that construct stores. Alongside them
`constants`, `exceptions`, `typing` and `headers` are supported too. `core`,
`encoding`, `spatial`, `lazy`, `ops`, `sharding`, `multiresolution` and `rechunk` are
internal and may change between releases, and `types`, `validate` and `composite` are
undecided — neither promised nor disowned. `zv.stability("zarr_vectors.<module>")`
answers for any of them at runtime.

Fuller walkthroughs live in the
[documentation](https://zarr-vectors-py.readthedocs.io/en/latest).

---

## Store Layout

A store is a plain Zarr v3 group. `.zarrvectors` is the canonical extension and
`.zv` the accepted short form — nothing in the format reads either, so a store
opens the same whichever it carries. Only five entries always exist; everything
else appears when the matching data is written.

```
dataset.zarrvectors/
├── zarr.json                     [always]  Zarr v3 group metadata.  Store-level fields sit under
│                                           attributes.zarr_vectors (zv_version, bounds, chunk_shape,
│                                           base_bin_shape, geometry_types, conventions, optional crs
│                                           and attribute_specs); per-level scale/translation under
│                                           attributes.multiscales (OME-NGFF 0.4).  attributes.ome
│                                           carries an OME-Zarr RFC 8 node so a collection elsewhere
│                                           can name this store by path — additive, and read by
│                                           nothing here (see zarr_vectors/core/ome.py).
├── 0/                            [always]  Full resolution.  Levels are bare integers, no prefix.
│   ├── zarr.json                 [always]  Level metadata under attributes.zarr_vectors_level
│   │                                       (vertex_count, object_sparsity, coarsening_method,
│   │                                       parent_level, arrays_present, ...).
│   │
│   │  ── spatial arrays: each is one Zarr v3 array whose shape IS the level's chunk grid, holding
│   │     one vlen-bytes cell per occupied chunk at  c/<i>/<j>/<k> .  Occupancy is listed in the
│   │     array's nonempty_chunks attribute; chunk_grid_origin (absent means 0) maps negative
│   │     chunk coordinates onto it.
│   │
│   ├── vertices/                 [always]  c/i/j/k = every vertex in that chunk, packed.  One cell
│   │   └── c/0/0/0 ...                     per chunk, not per bin.
│   ├── vertex_fragments/         [always]  Fragment index into the sibling vertices cell: per
│   │                                       fragment either a row range [start, count) or an explicit
│   │                                       list of row indices (which is how two fragments can share
│   │                                       a vertex).  Row indices, not byte offsets.  Fragments are
│   │                                       per-bin, or per-(object, chunk) once object ids exist.
│   ├── vertex_attributes/        [written] One child array per name, same grid, rows aligned 1:1
│   │   └── intensity/                      with the vertices cell.
│   ├── links/                    [written] Connectivity.  A GROUP two levels deep, never an array:
│   │   ├── 0/0.0.0/                        links/<delta>/<offsets>/.  <delta> is how many pyramid
│   │   ├── 0/0.0.+1/                       levels the record spans: 0 intra-level, +1/-1 for the
│   │   └── +1/0.0.0/                       parent/child metavertex edges a pyramid adds.  <offsets>
│   │                                       is where the record's other endpoints sit relative to its
│   │                                       source chunk (the cell holding it): link_width-1 signed
│   │                                       tuples, components joined by ".", offsets by "_", or
│   │                                       "self" when link_width is 1.  So 0.0.0 is intra-chunk and
│   │                                       0.0.+1 reaches one chunk along +z.  A cross-chunk link is
│   │                                       simply one with non-zero offsets — there is no separate
│   │                                       array family for them.
│   ├── link_fragments/           [written] Fragment index over the one intra-chunk link array
│   │                                       (delta 0, all-zero offsets), so it is keyed by chunk alone
│   │                                       and carries no delta or offsets segment.  Every other
│   │                                       offsets array self-describes and needs no sidecar.
│   ├── link_attributes/          [written] Mirrors links/ exactly — <name>/<delta>/<offsets>, same
│   │   └── weight/0/0.0.0/                 cells, same row order, so rows align 1:1 with links.
│   ├── fragment_attributes/      [builder] One row per fragment in the chunk.
│   │   └── object_id/
│   │
│   │  ── non-spatial arrays: rank-1 over object or group ids, so their chunk keys are  c/0, c/1, ...
│   │
│   ├── object_index/             [written] A GROUP whose only child is manifests: a ragged array with
│   │   └── manifests/                      one row per object slot, row i being object i's ordered
│   │       └── c/0                         (chunk_coords, fragment_index) references.  num_objects
│   │                                       and num_present sit on the group.
│   ├── object_attributes/        [written] One (O,) or (O,C) array per name; absent objects hold the
│   │   └── cell_type/                      array's fill value.
│   ├── groups/                   [written] One ragged (G,) array: row g is that group's object ids.
│   └── group_attributes/         [builder] One (G,) or (G,C) array per name.
│       └── region/
├── 1/                            [written] A coarser level from build_pyramid.  Same layout, but not
│   ├── vertices/                           the same arrays: a coarsened level gets vertices,
│   ├── vertex_fragments/                   vertex_fragments, object_index and links/-1, and inherits
│   ├── object_index/manifests/             object_attributes.  It does not get vertex_attributes,
│   ├── object_attributes/                  groups, group_attributes, link_attributes or
│   └── links/-1/0.0.0/                     link_fragments.
├── parametric/                   [written] Root-level, created lazily on the first parametric write.
│   ├── zarr.json                           Its attributes carry the type registry (plane, line,
│   ├── objects/                            sphere).  objects/ is (O, 1 + max_coeffs): a type id, then coefficients.
│   ├── names/
│   ├── object_attributes/
│   ├── groups/
│   └── group_attributes/
└── headers/                      [written] Root-level, one group per source format; the header dict
    └── swc/                                is stored as that group's attributes.
```

- `[always]` — there the moment `zv.create(...)` returns, before anything is written.
  A fresh store is exactly these five entries, and a bare `add_points(positions)` with
  no ids, attributes, groups or pyramid adds nothing more.
- `[written]` — appears only once the matching data is written, so which of these a
  store has depends on its geometry kind and on what the caller passed. A point cloud
  grows a `links/` only if you build a pyramid; polylines and streamlines never write
  `link_fragments/`, because their intra-chunk edges are implicit.
- `[builder]` — no `zarr_vectors.api` writer produces it; only `zarr_vectors.building`
  does. The api can still read them back: `Level.attribute_names("fragment")` and
  `Level.attribute_names("group")` are both accepted.

Sharding does not change this tree. The keys stay `c/i/j/k`; each file just becomes a
shard covering several cells.
