Metadata-Version: 2.4
Name: micro-reader-codecs
Version: 0.0.3
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Image Processing
License-File: LICENSE
License-File: THIRD_PARTY_LICENSES.txt
Summary: Native (Rust) image decoders for micro-reader: zstd, LZ4, LZW, deflate, PackBits, JPEG (incl. lossless), JPEG 2000, JPEG XL, PNG, WebP, packed bits, EER.
Keywords: microscopy,codecs,zstd,lz4,lzw,deflate,jpeg,jpeg2000,jpegxl,png,webp,eer
Author-email: Bugra Oezdemir <bugraa.ozdemir@gmail.com>
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/bugraoezdemir/micro-reader
Project-URL: Issues, https://github.com/bugraoezdemir/micro-reader/issues
Project-URL: Source, https://github.com/bugraoezdemir/micro-reader

# micro-reader-codecs

**Native image decoders for [micro-reader](https://github.com/bugraoezdemir/micro-reader).**

`micro-reader-codecs` holds the Rust decoders that micro-reader uses to read compressed microscopy data. It is a separate package so that micro-reader itself stays pure Python.

You normally don't need to install it yourself. It is installed automatically with:

```bash
pip install micro-reader
```

It has no Python dependencies.

## How it works

* Every decoder writes straight into a buffer you provide, such as a NumPy array.
* Decoders run without holding Python's lock, so micro-reader's thread pool can decode on all cores.
* Baseline JPEG, JPEG 2000 and JPEG XL decode on one thread each. micro-reader already decodes many tiles at once, and threads inside the codec would compete for the same cores.

## Decoders

| Function                    | Codec                                     | Rust crate                            |
| --------------------------- | ----------------------------------------- | ------------------------------------- |
| `zstd_decompress_into`      | zstd                                      | `zstd` (libzstd)                      |
| `lz4_block_decompress_into` | LZ4 block                                 | `lz4_flex`                            |
| `lzf_decompress_into`       | LZF (h5py's HDF5 filter)                  | own                                   |
| `lzw_decode_into`           | LZW, TIFF variant                         | `weezl`                               |
| `zlib_decompress_into`      | zlib / deflate                            | `libdeflater` (libdeflate)            |
| `packbits_decode_into`      | PackBits                                  | own                                   |
| `jpeg_decode_into`          | JPEG: baseline, extended, progressive, arithmetic; 8- and 12-bit; lossless (2-16 bit); NDPI's stripes | libjpeg-turbo 3.2.1 with its SIMD (C, vendored) |
|                             | JPEG libjpeg-turbo declines or warns about (4 components, damaged data) | own (following libjpeg-turbo), then `jpeg-decoder` |
| `jpegxr_decode_into`        | JPEG XR (Zeiss slide scans), colour as R, G, B; as imagecodecs decodes it | jxrlib (Microsoft, C, vendored) |
| `jxl_decode_into`           | JPEG XL                                   | `jxl-oxide`                           |
| `jpeg2k_decode_into`        | JPEG 2000 (J2K, JP2), also YCbCr with subsampled colour (e.g. Aperio slides) | `jpeg2k` (OpenJPEG) |
| `png_decode_into`           | PNG                                       | `png`                                 |
| `webp_decode_into`          | WebP, lossless and lossy                  | `image-webp`                          |
| `unpack_bits_into`          | packed samples, 1 to 32 bit               | own                                   |
| `eer_decode_into`           | EER (Falcon cryo-EM movies)               | own, ported from imagecodecs (BSD-3)  |

## Reading large blocks in parts

Some compressed blocks are too large to decode as a whole. These decoders read only part of a block:

* `jpeg2k_decode_area_into` and `jxl_decode_area_into` decode only the rectangle you ask for.
* `StreamDecoder` decodes one compressed block of a file as a stream and keeps only the bytes you ask for. It supports zlib, deflate, LZW, zstd, PackBits, PNG rows and uncompressed data. Each read continues where the last one stopped.
* `BlockIndex` keeps checkpoints of a deflate block. Several `StreamDecoder`s can share one, so jumping back in the block resumes from the nearest checkpoint instead of the start.

libjpeg-turbo and jxrlib are the libraries imagecodecs uses, called the same way, so the pixels are imagecodecs' (tests/test_c_decoders.py) and so is the speed: on six NDPI slides 0.89-1.42x tifffile + imagecodecs; on five Zeiss JPEG XR scans a tile decodes at 0.90-0.95x imagecodecs, whole reads within noise of it (2026-11-10).

## Not included yet

LERC, CCITT fax, float24 and a few rarer TIFF codecs still go through imagecodecs:

```bash
pip install "micro-reader[codecs]"
```

## Building

The Rust toolchain is pinned in `rust-toolchain.toml`. Building also needs CMake and, on x86, NASM, for libjpeg-turbo's SIMD (`build.rs`; ARM's NEON needs no assembler). To build and install into the active virtual environment:

```bash
maturin develop --release
```

Without NASM the build still works, with plain C JPEG decoding and a warning; `MICRO_READER_REQUIRE_SIMD=1` (the wheel builds) makes it an error. `NASM=<path>` names the assembler when it is not on `PATH`. `--no-default-features` leaves out both C libraries (features `libjpeg-turbo`, `jxrlib`): micro-reader then uses its own JPEG decoder and imagecodecs for JPEG XR.

## Tests

The tests live in the repository's `tests/` folder:

* `test_codecs.py`
* `test_czi_codecs.py`
* `test_tiff_codecs.py`
* `test_jpeg_exact.py`
* `test_c_decoders.py`
* `test_memory_limit.py`

They compare these decoders with independent reference decoders, bit for bit.

## License

MIT

The bundled crates and C libraries (libzstd, libdeflate, OpenJPEG, libjpeg-turbo, jxrlib) are MIT, Apache-2.0, BSD or Zlib licensed. libjpeg-turbo, and micro-reader's own JPEG decoder that follows it, carry the IJG and BSD licences: this software is based in part on the work of the Independent JPEG Group. All notices ship in every wheel as `THIRD_PARTY_LICENSES.txt`.

Whenever `Cargo.lock` changes, regenerate it with:

```bash
python scripts/third_party_licenses.py
```

