Metadata-Version: 2.4
Name: edgefirst-schemas
Version: 4.0.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Software Development :: Libraries
Requires-Dist: pytest>=7 ; extra == 'test'
Requires-Dist: numpy>=1.21 ; extra == 'test'
Provides-Extra: test
Summary: Zero-copy CDR message types for EdgeFirst Perception (ROS 2 + Foxglove + custom)
Keywords: ros2,cdr,foxglove,zenoh,perception,edge-ai,zero-copy
Home-Page: https://doc.edgefirst.ai
Author-email: Au-Zone Technologies <support@au-zone.com>
License: Apache-2.0
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/EdgeFirstAI/schemas/blob/main/CHANGELOG.md
Project-URL: Homepage, https://doc.edgefirst.ai
Project-URL: Issues, https://github.com/EdgeFirstAI/schemas/issues
Project-URL: Repository, https://github.com/EdgeFirstAI/schemas

# edgefirst-schemas (Python)

Zero-copy CDR message types for EdgeFirst Perception. Wraps the
[`edgefirst-schemas`](https://github.com/EdgeFirstAI/schemas) Rust crate via PyO3.

```bash
pip install edgefirst-schemas
```

The wheel ships two stable-ABI families per OS/architecture:

- `cp311-abi3` — Python 3.11+, zero-copy `Py_buffer` path (pip prefers this on 3.11+)
- `cp38-abi3` — Python 3.8–3.10 (and 3.11+ if the other wheel is absent); constructors copy and `BorrowedBuf.view()` returns `bytes`

See [abi3-py38 builds](#abi3-py38-builds--typed-numpy-caveats) for the older-ABI tradeoffs.

## Quick start

```python
from edgefirst.schemas.builtin_interfaces import Time
from edgefirst.schemas.std_msgs import Header
from edgefirst.schemas.sensor_msgs import Image
import numpy as np

pixels = np.zeros((720, 1280, 3), dtype=np.uint8)
img = Image(
    header=Header(stamp=Time(sec=1, nanosec=0), frame_id="cam"),
    height=720, width=1280, encoding="rgb8",
    is_bigendian=0, step=1280 * 3,
    data=pixels,  # any contiguous buffer-protocol object
)

# Zero-copy view of the pixel data — `np.frombuffer` aliases the same
# bytes the message was constructed with, no copy.
arr = np.frombuffer(img.data, dtype=np.uint8).reshape(720, 1280, 3)

# Wire bytes for transport:
buf = img.to_bytes()
img2 = Image.from_cdr(buf)
assert img2.width == 1280
```

### Forwarding raw CDR bytes without a copy

For publish paths where you want to hand the wire bytes straight to a
transport (Zenoh, raw socket, mcap writer), `cdr_view()` exposes the
full CDR buffer (header + payload) as a zero-copy `BorrowedBuf`:

```python
transport.publish(memoryview(img.cdr_view()))   # zero-copy
```

`to_bytes()` is the explicit-copy alternative when the consumer wants
an owned `bytes` value.

### Zero-copy contract — `BorrowedBuf`

Every bulk byte payload (`Image.data`, `Mask.mask`, `RadarCube.cube`,
`PointCloud2.data`, `CompressedVideo.data`, `CompressedImage.data`)
returns a `BorrowedBuf` view that aliases the parent message's bytes —
not a copy. The `BorrowedBuf` holds a strong reference to the parent,
so it's safe to keep it (or a `memoryview` derived from it) live after
the original message reference is dropped.

| Method | Returns | Cost |
|---|---|---|
| `borrowed_buf` itself | wraps the bytes | zero-copy, O(1) |
| `np.frombuffer(borrowed_buf, dtype=...)` | numpy ndarray aliasing the bytes | zero-copy, O(1) (Py 3.11+) |
| `memoryview(borrowed_buf)` | parent-anchored memoryview | zero-copy, O(1) (Py 3.11+) |
| `borrowed_buf.tobytes()` | owned `bytes` | one memcpy |
| `borrowed_buf.view()` | memoryview (Py 3.11+) / bytes (abi3-py38) | zero-copy / one memcpy |

## Migrating from the pycdr2-backed `edgefirst.schemas`

This release replaces the pure-Python pycdr2 codec with a Rust-backed
pyo3 binding. Wire-format bytes are unchanged — anything encoded by the
previous pycdr2 module decodes through `from_cdr()` and vice-versa — but
the Python API surface is narrower and stricter.

### What changed at the call site

```python
# pycdr2-backed (old)                    # pyo3-backed (new)
img = Image()                            img = Image(
img.height = 720                             header=Header(stamp=Time(1, 0)),
img.width = 1280                             height=720, width=1280,
img.encoding = "rgb8"                        encoding="rgb8", is_bigendian=0,
img.data = pixels                            step=1280 * 3, data=pixels,
buf = img.serialize()                    )
img2 = Image.deserialize(buf)            buf = img.to_bytes()
                                         img2 = Image.from_cdr(buf)
```

| pycdr2 pattern | pyo3 replacement |
|---|---|
| `Foo()` then field assignment | `Foo(field=value, …)` constructor only — pyclasses are frozen |
| `.serialize()` | `.to_bytes()` |
| `Foo.deserialize(buf)` | `Foo.from_cdr(buf)` |
| `msg.data` returning `bytes` | `msg.data` returning `BorrowedBuf` (zero-copy view); use `.tobytes()` for the old shape |
| `from_schema()`, `decode_pcd()`, `colormap()`, registry helpers | **removed** — the legacy module survives only at `benches/python/legacy/` for benchmark parity |
| `std_msgs.{String, Int32, Float64, …}` primitive wrappers | **removed** — these were pycdr2-generated single-value wrappers; pass raw Python values instead |
| Mutable dataclass-style instances | Frozen pyclasses; rebuild instead of mutate |

### What stays the same

- All field names and types match the ROS 2 / Foxglove / EdgeFirst
  schemas verbatim.
- Wire-format CDR1 LE bytes are byte-equivalent across versions.
- The `edgefirst.schemas.<submodule>` import paths
  (`sensor_msgs.Image`, `std_msgs.Header`, etc.) are preserved.

### abi3-py38 builds — typed numpy caveats

The default published wheel on Python 3.11+ is `cp311-abi3`, where the
buffer protocol is in the limited API and typed numpy arrays
(`np.uint16`, `np.float32`, …) work zero-copy as constructor inputs.
On the `cp38-abi3` wheel (Python 3.8–3.10, or a source build with
`--features abi3-py38`) the buffer protocol isn't available; pass
`arr.tobytes()` for typed arrays, and `BorrowedBuf.view()` returns
`bytes` (one copy) instead of a parent-anchored memoryview. Plain
`bytes` / `bytearray` / `np.uint8` arrays work on either build.

```python
shape = np.array([2, 128, 12, 128], dtype=np.uint16)

# Default cp311-abi3 wheel — works directly:
RadarCube(..., shape=shape, ...)

# abi3-py38 build — pass bytes:
RadarCube(..., shape=shape.tobytes(), ...)
```

## Build from source

Default local develop (cp311-abi3, Python 3.11+ zero-copy):

```bash
maturin develop --release --manifest-path crates/python/Cargo.toml
```

cp38-abi3 wheel (Python 3.8+; copy-fallback on typed buffers):

```bash
maturin build --release \
  --manifest-path crates/python/Cargo.toml \
  --no-default-features --features abi3-py38
```

Cross-compile manylinux2014 wheels via zig:

```bash
maturin build --release --zig --compatibility manylinux2014 \
  --target aarch64-unknown-linux-gnu \
  --manifest-path crates/python/Cargo.toml
```

