Metadata-Version: 2.4
Name: naw
Version: 0.0.4
Summary: Log and manage metrics for machine learning projects
Author-email: Mathieu Lacage <mathieu.lacage@inria.fr>
Maintainer-email: Mathieu Lacage <mathieu.lacage@inria.fr>
License-Expression: MIT
Project-URL: Homepage, https://mlacage.gitlabpages.inria.fr/naw/latest/
Project-URL: Documentation, https://mlacage.gitlabpages.inria.fr/naw/latest/
Project-URL: Repository, https://gitlab.inria.fr/mlacage/naw
Project-URL: Changelog, https://mlacage.gitlabpages.inria.fr/naw/latest/changelog/
Project-URL: Issues, https://gitlab.inria.fr/mlacage/naw/-/issues
Keywords: metrics,logging,time-series,compression,machine-learning,wandb
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Logging
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: argcomplete>=3.7.0
Requires-Dist: pyyaml
Requires-Dist: tabulate
Provides-Extra: plot
Requires-Dist: uniplot>=0.23.2; extra == "plot"
Requires-Dist: matplotlib>=3.10.9; extra == "plot"
Dynamic: license-file

# naw

**naw** logs machine-learning time-series metrics to a compressed file on disk,
and gives you a CLI to read them back.

It is two things in one package:

- a **storage library** (`naw.rtsdb`) that writes typed, columnar, compressed
  rows to a single `.rtsdb` file, and reads them back — including while the file
  is still being written;
- a **wandb-compatible façade** (`naw.wandb`) so that code already instrumented
  with `wandb.init()` / `run.log()` keeps working with nothing but an import
  change.

Full documentation: <https://mlacage.gitlabpages.inria.fr/naw>

## Installation

```console
$ pip install naw          # or: uv add naw
```

naw requires **Python 3.10 or newer** and pulls in three small dependencies
(`argcomplete`, `pyyaml`, `tabulate`) — no NumPy. The `plot` and `heatmap`
subcommands render through [uniplot](https://pypi.org/project/uniplot/) and
[matplotlib](https://pypi.org/project/matplotlib/), which are not installed by
default; get them with the `plot` extra:

```console
$ pip install "naw[plot]"   # or: uv add "naw[plot]"
```

## Log a run

```python
import naw.wandb as wandb

run = wandb.init(project="demo", config={"lr": 3e-4})
for step in range(1000):
    run.log({"loss": loss})
run.finish()
```

`init()` creates `<dir>/wandb/<project>/<id>.rtsdb`. A run that is never
finished is still readable — you just lose the summary.

## Read it back

Every subcommand takes the run filename as its first argument:

| Command | Description |
| --- | --- |
| `naw metadata FILE` | Show the run's identity, configuration and final summary. |
| `naw metrics FILE` | List the metric columns present in the run and their storage types. |
| `naw watch FILE` | Print metric rows as they are written, then exit when the run closes. |
| `naw plot FILE -y METRIC` | Plot one metric against another (terminal, matplotlib, SVG, PNG or CSV). |
| `naw heatmap FILE -m METRIC` | Render a histogram metric as a 2-D density map over time. |

```console
$ naw plot wandb/demo/<run-id>.rtsdb -y loss --lines
```

## How it compresses

Values are grouped by column rather than by row, and each column is encoded with
a scheme suited to its type: integers as a delta against the previous row then
LEB128 varint, floats as an XOR against the previous row's value then a varint
over the resulting bit pattern, and `float16`/`bfloat16` in 2 bytes. A step
counter that increments by one costs a single byte per row.

The writer periodically emits a *sync point* carrying a schema index, so a
reader that opens the file mid-run can seek to the last sync point instead of
replaying the whole file. That is what makes `naw watch` a live tail rather than
a repeated full parse.

## Documentation

- [Get started](https://mlacage.gitlabpages.inria.fr/naw/latest/) — what naw is and
  when to reach for it.
- [Quickstart](https://mlacage.gitlabpages.inria.fr/naw/latest/quickstart/) — log a run
  and read it back in five minutes.
- [CLI](https://mlacage.gitlabpages.inria.fr/naw/latest/reference/cli/) — every
  subcommand and flag.
- [Python API](https://mlacage.gitlabpages.inria.fr/naw/latest/reference/api/) — the
  `naw.rtsdb` read and write path.
- [wandb compatibility](https://mlacage.gitlabpages.inria.fr/naw/latest/reference/wandb-compat/)
  — exactly which parts of the wandb API are covered.
- [File format](https://mlacage.gitlabpages.inria.fr/naw/latest/reference/file-format/)
  — what is actually in an `.rtsdb` file.
- [Contributing](https://mlacage.gitlabpages.inria.fr/naw/latest/contributing/) —
  build, test and release.

## References

The XOR-then-varint idea is the one introduced by Gorilla and refined by Chimp
and Elf; naw uses varint framing rather than these papers' bit-level framing,
which keeps decoding cheap and the format simple to reason about.

- [Gorilla](https://www.vldb.org/pvldb/vol8/p1816-teller.pdf): A fast, scalable, in-memory time series database
- [Chimp](https://www.vldb.org/pvldb/vol15/p3058-liakos.pdf): efficient lossless floating point compression for time series databases
- [Elf](https://www.vldb.org/pvldb/vol16/p1763-li.pdf): Erasing-Based Lossless Floating-Point Compression

## License

MIT — see [LICENSE](https://gitlab.inria.fr/mlacage/naw/-/raw/main/LICENSE?ref_type=heads).
