Metadata-Version: 2.5
Name: file2records
Version: 0.2.0
Summary: Turn scientific papers (PDF, JATS and Elsevier XML, HTML, Word) into a structured dataset: one model extracts, a second audits, you review against the source text.
Project-URL: Homepage, https://github.com/Enkhnyam/chemistry-data-extractor-toolkit
Project-URL: Repository, https://github.com/Enkhnyam/chemistry-data-extractor-toolkit
Project-URL: Issues, https://github.com/Enkhnyam/chemistry-data-extractor-toolkit/issues
Author: Enkhnyam Battulga
License-Expression: MIT
License-File: LICENSE
Keywords: chemistry,data extraction,elsevier,jats,llm,pdf,scientific papers
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Chemistry
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.115.0
Requires-Dist: httpx>=0.27
Requires-Dist: litellm>=1.91.0
Requires-Dist: lxml>=5.0
Requires-Dist: markdown-it-py>=3.0.0
Requires-Dist: pydantic>=2.0
Requires-Dist: python-docx>=1.1
Requires-Dist: python-dotenv>=1.0.1
Requires-Dist: python-multipart>=0.0.12
Requires-Dist: tenacity>=8.0
Requires-Dist: uvicorn[standard]>=0.32.0
Provides-Extra: pdf
Requires-Dist: docling>=2.110.0; extra == 'pdf'
Description-Content-Type: text/markdown

# file2records

file2records builds a dataset from a folder of scientific papers. You say which fields a
record has. A language model reads each paper and fills them in, a second model checks every
record against the paper, and you review the result with the source passage next to each
value. It reads PDF, JATS XML, Elsevier XML, HTML, and Word files, runs on your own machine,
and comes set up for RWTH's free KI:connect models.

Documentation: <https://enkhnyam.github.io/chemistry-data-extractor-toolkit/>

```bash
pip install "file2records[pdf]"        # or: pip install file2records   (no PDFs, ~250 MB)
file2records serve my-first-project    # opens a finished example in your browser
```

```python
import file2records as fr

project = fr.Project("my-review")
project.add("papers/")                                   # PDF, XML, HTML, Word, Markdown
project.extract(model=fr.rwth(), only=r"glycoly[sz]is")  # RWTH_API_KEY from the environment
project.judge(model=fr.rwth())
project.export("dataset.csv")                            # one row per record, with its DOI
```

Start with the [tutorial](https://enkhnyam.github.io/chemistry-data-extractor-toolkit/tutorial/):
a PET glycolysis dataset from three real papers in 15 minutes.

## Development

```bash
uv sync                                       # includes docling for PDF tests
uv run python -m unittest discover -s tests   # no network, no key
uvx zensical serve                            # the docs, at http://localhost:8000
```

Releasing: [RELEASING.md](RELEASING.md). Licence: the code is MIT; the demo paper is CC BY
([NOTICE](src/file2records/demo/NOTICE.md)). Please cite: [CITATION.cff](CITATION.cff).
