Metadata-Version: 2.4
Name: tif1-native
Version: 0.2.1
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: Free Threading :: 2 - Beta
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Rust
Requires-Dist: numpy>=1.26
Summary: Optional native accelerators for tif1 (telemetry payload decoder)
License: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/TracingInsights/tif1
Project-URL: Repository, https://github.com/TracingInsights/tif1/tree/main/native

# tif1-native

Optional native accelerators for [tif1](https://github.com/TracingInsights/tif1).
`tif1` works without this package; when it is importable, `tif1` decodes
telemetry payloads with `tif1_native.decode_telemetry`, which parses a
`{driver}/{lap}_tel.json` payload straight into typed numpy arrays while
releasing the GIL, so the fetch and cache threads parse in parallel on
multi-core hosts. From 0.2 the `DriverAhead` channel is returned
dictionary-encoded (int32 codes into a short category list, built by the
`dict_factory` tif1 passes) instead of one Python `str` per sample; tif1
builds the string column from the codes. Frames are identical either way
(`tests/unit/test_native_decoder.py`).

From 0.2.1, `tif1_native.lz4_frame_decompress_into(src, dst) -> int` decodes
an lz4 frame into a caller-owned writable buffer with the GIL released.
On Linux, tif1 uses it to decompress the frame tier's driver rows into
huge-page `mmap` buffers (`tests/unit/test_elis_e4_huge_page_rows.py`).
The blocks are decoded by liblz4 (`LZ4_decompress_safe_usingDict`, the
decoder pyarrow's lz4 codec runs), vendored by the `lz4-sys` crate and
compiled into the extension; the frame header is parsed in Rust
(`src/lz4.rs`), which rejects checksummed frames and dictionary ids.

`TIF1_DISABLE_NATIVE=1` makes tif1 ignore an installed extension.

## Build locally

Needs a Rust toolchain and a C compiler (gcc or clang, MSVC on Windows) for
the vendored liblz4:

    uvx maturin develop --release -m native/Cargo.toml

or build a wheel with `uvx maturin build --release -m native/Cargo.toml`.

## Wheels

`.github/workflows/native.yml` builds and tests the distribution on every
change under `native/`, and uploads it to TestPyPI or PyPI from a manual
dispatch only:

- one `cp311-abi3` wheel per platform, used by every GIL CPython from 3.11
  on (3.15 included) without a rebuild;
- no free-threaded wheels for now: pyo3 0.29 builds only 3.14t and tif1 does
  not run on free-threaded CPython yet (orjson and polars ship no such
  wheels); the module declares `gil_used = false`, so a local build on a
  free-threaded interpreter keeps the GIL disabled;
- Linux manylinux2014 and musllinux_1_2 (x86_64, aarch64), macOS universal2,
  Windows x64 and arm64, plus an sdist for other platforms (needs Rust and a
  C compiler).

Pyodide is not built; tif1 uses orjson there.

