Metadata-Version: 2.4
Name: relflow
Version: 0.2.3
Summary: Relationship Flow: schema-first PyTorch models for hierarchical, nested, and sequence data
Author: Grantham Taylor, Reid Taylor
License-Expression: Apache-2.0
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: beartype>=0.21.0
Requires-Dist: rich>=14.0.0
Requires-Dist: pydantic>=2.11.7
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: anytree>=2.13.0
Requires-Dist: pyarrow>=21.0.0
Requires-Dist: polars>=1.35.2
Requires-Dist: numpy>=2.2.6
Requires-Dist: lightning>=2.6.4
Requires-Dist: tensordict>=0.10.0
Requires-Dist: torch>=2.7.1
Requires-Dist: torchmetrics>=1.9.0
Requires-Dist: typing-extensions>=4.12
Provides-Extra: serving
Requires-Dist: fastapi>=0.124.0; extra == "serving"
Requires-Dist: orjson>=3.10.0; extra == "serving"
Requires-Dist: pydantic-settings>=2.10.1; extra == "serving"
Requires-Dist: uvicorn>=0.38.0; extra == "serving"
Provides-Extra: text
Requires-Dist: transformers>=4.55.0; extra == "text"
Provides-Extra: typing
Requires-Dist: pyarrow-stubs>=20.0.0.20260819; extra == "typing"
Requires-Dist: types-PyYAML>=6.0.12; extra == "typing"
Provides-Extra: duckdb
Requires-Dist: duckdb>=1.5.0; extra == "duckdb"
Dynamic: license-file

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="docs/assets/branding/banners/banner.dark.svg" />
    <source media="(prefers-color-scheme: light)" srcset="docs/assets/branding/banners/banner.light.svg" />
    <img alt="relflow" src="docs/assets/branding/banners/banner.light.svg" width="100%" />
  </picture>
</p>

<p align="center">
  <a href="https://pypi.org/project/relflow/"><img alt="PyPI version" src="https://img.shields.io/pypi/v/relflow?logo=pypi&amp;logoColor=white" /></a>
  <img alt="Python 3.12+" src="https://img.shields.io/badge/python-3.12%2B-3776AB?logo=python&amp;logoColor=white" />
  <a href="LICENSE"><img alt="Apache-2.0 license" src="https://img.shields.io/badge/license-Apache--2.0-2E8B57" /></a>
  <a href="https://relflow.github.io/relflow/"><img alt="Documentation" src="https://img.shields.io/badge/docs-Quarto-39729E?logo=quarto&amp;logoColor=white" /></a>
  <!-- discord-invite:start -->
  <a href="https://discord.gg/DVyZUkvTFA"><img alt="Discord channel invite" src="https://img.shields.io/badge/discord-join%20the%20channel-5865F2?logo=discord&amp;logoColor=white" /></a>
  <!-- discord-invite:end -->
</p>

relflow builds PyTorch/Lightning models from nested records streamed through
Apache Arrow. Typed fields represent values; branches combine them into local
contexts; decoders learn to predict selected fields from the available context.

## Install

Python 3.12 or newer:

```bash
uv add relflow
```

Add `relflow[text]` for Hugging Face text encoders or `relflow[serving]` for the
HTTP runtime.

## Describe One Record

```yaml
line_items:
  - sku: A12
    quantity: 2
    price: 19.99
  - sku: B07
    quantity: 1
    price: 45.50
returned: false
```

YAML illustrates one observation. Store these records as nested Arrow structs
and lists in Parquet datasets; relflow scans them in batches.

```python
import relflow as rf

model = rf.Model(
    d_model=64,
    n_layers=2,
    n_heads=4,
    batch_size=128,
    line_items=rf.Branch(
        length=32,
        sku=rf.Category,
        quantity=rf.Number,
        price=rf.Number,
    ),
    returned=rf.Boolean(mask=True),
)
```

Parent keywords name each field. `mask=True` makes `returned` a supervised
target whose value never enters the encoder. The branch builds line-item
context before its reduced representation reaches the order root.

## Stream Training And Prediction

The paths below are application-supplied Parquet datasets. Training and
validation records include `returned`; request records omit it.

```python
import lightning.pytorch as lit
import pyarrow.dataset as ds

train = ds.dataset("warehouse/train", format="parquet")
validation = ds.dataset("warehouse/validation", format="parquet")
requests = ds.dataset("warehouse/requests", format="parquet")

model.optimizer = rf.adamw(learning_rate=1e-3)
data = rf.ArrowDataModule(
    model=model, train=train, validate=validation, predict=requests
)
trainer = lit.Trainer(
    max_epochs=30, devices=1, callbacks=[rf.Writer("predictions")]
)
trainer.fit(model=model, datamodule=data)
trainer.predict(model=model, datamodule=data, return_predictions=False)

predictions = ds.dataset("predictions", format="parquet")
```

`ArrowDataModule` opens a fresh scan for each pass and prepares model batches
as records arrive. `rf.Writer` writes prediction batches to
`predictions/rank-0.parquet`; `return_predictions=False` avoids collecting
all outputs in memory. Open the output as an Arrow dataset for further batch
processing. Decoded values live under its `predictions` column at addresses
such as `/returned`.

## Documentation

- [Getting started](https://relflow.github.io/relflow/getting-started.html)
- [Model structure](https://relflow.github.io/relflow/core-concepts/model-tree.html)
- [Data types](https://relflow.github.io/relflow/core-concepts/data-types.html)
- [Arrow data loading](https://relflow.github.io/relflow/guides/data-modules.html)
- [Preprocessing](https://relflow.github.io/relflow/guides/preprocessors.html)
- [Training and checkpoints](https://relflow.github.io/relflow/guides/lightning.html)
- [Batch inference](https://relflow.github.io/relflow/guides/batch-inference.html)
- [Prediction output](https://relflow.github.io/relflow/guides/prediction-output.html)

Docs use static examples and Typst model diagrams. Build them with `make render`;
run `make check-docs` to validate the render in a temporary directory. Package
checks use `uv run pytest`; synthetic learning checks use `make proofs`.
See [CONTRIBUTING.md](CONTRIBUTING.md) for development conventions.

[Community](https://discord.gg/DVyZUkvTFA) · [Apache 2.0 license](LICENSE)
