Metadata-Version: 2.5
Name: damicore-normalizer
Version: 0.1.0
Summary: Dataset normalization stage for the DAMICORE pipeline.
Project-URL: Homepage, https://github.com/Delbem-Research-and-Innovation/damicore
Project-URL: Repository, https://github.com/Delbem-Research-and-Innovation/damicore
Project-URL: Issues, https://github.com/Delbem-Research-and-Innovation/damicore/issues
Project-URL: Documentation, https://github.com/Delbem-Research-and-Innovation/damicore/blob/main/docs/quickstart.md
Project-URL: Changelog, https://github.com/Delbem-Research-and-Innovation/damicore/blob/main/CHANGELOG.md
Author-email: Ennio Politi Lopes <enniolopes@users.noreply.github.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: canonical-serialization,csv,damicore,normalization
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.11
Requires-Dist: pandas<4,>=2.2
Requires-Dist: pydantic<3,>=2.10
Description-Content-Type: text/markdown

# damicore-normalizer

damicore-normalizer deterministically serializes the columns or rows of a local
CSV as canonical JSONL objects, recording the size and SHA-256 of every object.
It is the first stage of the DAMICORE pipeline; most users install the aggregate
`damicore` distribution, which runs all four stages end to end. Install this
package alone to normalize datasets without the rest of the pipeline.

```bash
pip install damicore-normalizer
```

## Python

```python
from damicore_normalizer import NormalizationConfig, normalize_csv

result = normalize_csv(
    "dataset.csv",
    "normalization",
    config=NormalizationConfig(split="columns", chunk_rows=50_000),
)
```

The call streams through `pandas.read_csv`, bounds open column files with an
LRU pool, and writes the objects plus a `manifest.json` into the output
directory. That normalization manifest is the input the sibling
`damicore-distance` distribution consumes to compute the NCD matrix.

## Links

- Repository: <https://github.com/Delbem-Research-and-Innovation/damicore>
- Issues: <https://github.com/Delbem-Research-and-Innovation/damicore/issues>
- Documentation:
  <https://github.com/Delbem-Research-and-Innovation/damicore/blob/main/docs/quickstart.md>

Licensed under Apache-2.0.
