Metadata-Version: 2.4
Name: batchstats
Version: 0.7
Summary: Online statistics for batched, streaming, and out-of-core NumPy data.
Author: Cyril Joly
License-Expression: MIT
Project-URL: Documentation, https://batchstats.readthedocs.io
Project-URL: Issues, https://github.com/CyrilJl/BatchStats/issues
Project-URL: Repository, https://github.com/CyrilJl/BatchStats
Keywords: numpy,online-algorithms,statistics,streaming
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.22.4
Provides-Extra: test
Requires-Dist: pytest<10,>=8; extra == "test"
Requires-Dist: pytest-cov<8,>=5; extra == "test"
Provides-Extra: docs
Requires-Dist: sphinx<10,>=7; extra == "docs"
Requires-Dist: sphinx-favicon<2,>=1; extra == "docs"
Requires-Dist: sphinxawesome-theme<6,>=5; extra == "docs"
Provides-Extra: dev
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: pytest<10,>=8; extra == "dev"
Requires-Dist: pytest-cov<8,>=5; extra == "dev"
Requires-Dist: ruff<1,>=0.11; extra == "dev"
Requires-Dist: sphinx<10,>=7; extra == "dev"
Requires-Dist: sphinx-favicon<2,>=1; extra == "dev"
Requires-Dist: sphinxawesome-theme<6,>=5; extra == "dev"
Requires-Dist: twine<8,>=6; extra == "dev"
Dynamic: license-file

<div align="center">
  <img src="https://raw.githubusercontent.com/CyrilJl/BatchStats/main/docs/source/_static/logo_batchstats.svg" alt="Logo BatchStats" width="200">

[![PyPI Version](https://img.shields.io/pypi/v/batchstats.svg)](https://pypi.org/project/batchstats/)
[![Python Versions](https://img.shields.io/pypi/pyversions/batchstats.svg)](https://pypi.org/project/batchstats/)
[![conda Version](https://anaconda.org/conda-forge/batchstats/badges/version.svg)](https://anaconda.org/conda-forge/batchstats)
[![Documentation Status](https://img.shields.io/readthedocs/batchstats?logo=read-the-docs)](https://batchstats.readthedocs.io/en/latest/?badge=latest)
[![Unit tests](https://github.com/CyrilJl/BatchStats/actions/workflows/pytest.yml/badge.svg)](https://github.com/CyrilJl/BatchStats/actions/workflows/pytest.yml)

</div>

# BatchStats

BatchStats computes statistics on data that arrives in batches, so you can stream or process large datasets without loading everything into memory. Its incremental algorithms expose a small NumPy-friendly API and support merging independently computed accumulators.

BatchStats requires Python 3.10 or newer.

## Installation

```console
pip install batchstats
```

Or with `conda`/`mamba`:

```console
conda install -c conda-forge batchstats
```

## Quick Start

```python
import numpy as np
from batchstats import BatchMean, BatchVar

rng = np.random.default_rng(0)
data_stream = (rng.standard_normal((100, 10)) for _ in range(10))

batch_mean = BatchMean()
batch_var = BatchVar()

for batch in data_stream:
    batch_mean.update_batch(batch)
    batch_var.update_batch(batch)

mean = batch_mean()
variance = batch_var()

print(f"Mean shape: {mean.shape}")
print(f"Variance shape: {variance.shape}")
```

## Available Statistics

* `BatchSum` / `BatchNanSum`
* `BatchWeightedSum`
* `BatchMean` / `BatchNanMean`
* `BatchWeightedMean`
* `BatchMin` / `BatchNanMin`
* `BatchMax` / `BatchNanMax`
* `BatchPeakToPeak` / `BatchNanPeakToPeak`
* `BatchVar`
* `BatchStd`
* `BatchCov`
* `BatchCorr`
* `BatchTopK` / `BatchNanTopK` (exact extreme ranks and linear tail quantiles)

`BatchNanSum` and `BatchNanMean` support merging with `+`. These classes and
the top-k accumulators support versioned `to_state()` / `from_state()` and
NPZ checkpoints with `save()` / `load()`, without pickle.

```python
from batchstats import BatchNanTopK, required_k

extremes = BatchNanTopK(required_k(0.998, 8784), axis=0)
for block in data_stream:  # (time, *spatial_shape), same spatial positions
    extremes.update_batch(block)
p998 = extremes.quantile(0.998)  # raises if retained capacity is insufficient
extremes.save("extremes.npz")
```

Top-k outputs have a leading rank axis and mask unavailable ranks while preserving
the selected dtype. `BatchTopK` rejects NaNs; `BatchNanTopK` ignores them per cell.
See [the streaming and checkpoint guide](docs/source/extreme_statistics.rst).

Docs: https://batchstats.readthedocs.io

## Development

Install the development dependencies and run the local quality gates:

```console
python -m pip install -e ".[dev]"
python -m ruff check .
python -m ruff format --check .
python -m pytest --cov=batchstats
python -m build
python -m twine check dist/*
```
