Metadata-Version: 2.5
Name: bibliography-io
Version: 0.4.0
Summary: Loading and writing bibliographic records across formats
Project-URL: Homepage, https://github.com/CoLRev-Environment/bibliography-io
Project-URL: Repository, https://github.com/CoLRev-Environment/bibliography-io
Project-URL: Issues, https://github.com/CoLRev-Environment/bibliography-io/issues
Project-URL: Changelog, https://github.com/CoLRev-Environment/bibliography-io/releases
Project-URL: Documentation, https://colrev-environment.github.io/bibliography-io/
Author: Gerit Wagner
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing
Requires-Python: >=3.10
Provides-Extra: all
Requires-Dist: grobid-tei>=0.3.0; extra == 'all'
Requires-Dist: openpyxl>=3.1.2; extra == 'all'
Requires-Dist: pandas<3,>=2.2; extra == 'all'
Requires-Dist: xlrd>=2.0.1; extra == 'all'
Provides-Extra: docs
Requires-Dist: sphinx-copybutton>=0.5.2; extra == 'docs'
Requires-Dist: sphinx-rtd-theme>=3.0; extra == 'docs'
Requires-Dist: sphinx<9,>=7.3; extra == 'docs'
Provides-Extra: excel
Requires-Dist: openpyxl>=3.1.2; extra == 'excel'
Requires-Dist: pandas<3,>=2.2; extra == 'excel'
Requires-Dist: xlrd>=2.0.1; extra == 'excel'
Provides-Extra: markdown
Requires-Dist: grobid-tei>=0.3.0; extra == 'markdown'
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == 'test'
Description-Content-Type: text/markdown

<div align="center">

<img src="https://raw.githubusercontent.com/CoLRev-Environment/bibliography-io/main/docs/_static/bibliography_io_logo.png" alt="bibliography-io logo" width="400">

# bibliography-io

[![PyPI - Version](https://img.shields.io/pypi/v/bibliography-io?color=blue)](https://pypi.org/project/bibliography-io/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/bibliography-io)
[![License](https://img.shields.io/github/license/CoLRev-Environment/bibliography-io)](https://github.com/CoLRev-Environment/bibliography-io/blob/main/LICENSE)
[![GitHub Actions](https://img.shields.io/github/actions/workflow/status/CoLRev-Environment/bibliography-io/ci.yml?label=tests)](https://github.com/CoLRev-Environment/bibliography-io/actions/workflows/ci.yml)
[![Documentation](https://img.shields.io/github/actions/workflow/status/CoLRev-Environment/bibliography-io/docs.yml?label=docs)](https://github.com/CoLRev-Environment/bibliography-io/actions/workflows/docs.yml)

</div>

`bibliography-io` addresses the distinct requirements of bibliographic data management throughout literature reviews: robust ingestion of heterogeneous source data and efficient, reproducible management of primary records.

These stages have different priorities:

- **Ingestion and normalization adapt source data for processing.** `load()` orchestrates parsing and normalization, applying source-specific rules to transform differing source structures into a common primary record representation.
- **Primary data management prioritizes efficiency and reproducibility.** Normalized records follow a consistent schema with unique IDs. Frequent reading and writing require efficient dictionary-based access, partial header loading, targeted updates, and deterministic serialization that avoids meaningless Git diffs.

Accordingly, `bibliography-io` distinguishes two interfaces:

- **`load()`** — Orchestrated ingestion and normalization that parses source-specific structures into a common primary record representation.
- **`load_primary()`** — Efficient I/O for consistently structured primary records.

Unlike format-specific libraries with differing internal representations, `bibliography-io` provides a common record structure across input formats, enabling source-independent processing in subsequent literature review steps.

The package also minimizes dependencies, requiring specialized packages (e.g., for Excel or GROBID-based extraction) only when needed, facilitating integration into diverse literature review workflows.

<!--
**Guiding principle:** Forgiving at the boundaries, strict in the core, and reproducible by design.
-->

## Installation

Install the base package with:

```shell
pip install bibliography-io
```

Install optional features as needed:

```shell
pip install "bibliography-io[markdown]"  # Markdown parsing through GROBID
pip install "bibliography-io[excel]"     # Excel file support
pip install "bibliography-io[all]"       # Both optional features
```

Markdown reference-list parsing uses the standalone `grobid-tei` client and
requires a GROBID service started by the caller. The loader is designed to
operate independently of the workflow application that uses it: it does not
start, stop, or otherwise manage GROBID. The service URL defaults to
`http://localhost:8070`; set `BIBLIOGRAPHY_IO_GROBID_URL` to provide another
URL. If the configured service is unavailable, Markdown loading raises
`BackendExecutionError`. Formats available in the base bibliography-io
installation do not require the Markdown extra.

## Usage

```python
from bibliography_io import load, load_primary, write

records = load("references.bib", source="colrev.web_of_science")
write(records, path="primary.bib", primary=True)
primary_records = load_primary("primary.bib")
```

File APIs infer the format from recognized suffixes; an explicit `format` can
override inference. String APIs always require `format`:

```python
from bibliography_io import to_string
from bibliography_io import loads_raw

records = loads_raw("@article{example, title={An example}}", format="bib")
text = to_string(records, format="bib")
```

File formats are inferred from their suffixes; pass `format` to `load()` when a
suffix is ambiguous or does not match the file's actual format.

### Format support

| Format | Read files | Write files | String APIs |
| --- | --- | --- | --- |
| BibTeX (`.bib`) | Yes | Yes | `loads_raw()` / `to_string()` |
| RIS (`.ris`) | Yes | Yes | `loads_raw()` / `to_string()` |
| ENL / EndNote (`.enl`) | Yes | *Planned* | `loads_raw()` (`to_string()` planned) |
| NBIB (`.nbib`) | Yes | *Planned* | `loads_raw()` (`to_string()` planned) |
| JSON (`.json`) | Yes | *Planned* | `loads_raw()` (`to_string()` planned) |
| CSV (`.csv`) | Yes | Yes | `loads_raw()` / `to_string()` |
| Excel (`.xls`, `.xlsx`) | Yes | Yes | File APIs only |
| Markdown (`.md`) | Yes (`markdown` extra) | Yes (tables) | `loads_raw()` / `to_string()` |

Markdown table writing is built into bibliography-io.
Markdown reference-list parsing uses GROBID through the standalone `grobid-tei`
client and requires a service started and managed by the caller. It defaults to
`http://localhost:8070`; set `BIBLIOGRAPHY_IO_GROBID_URL` to use another URL.
The loader is designed to operate independently of the workflow application
that uses it and never manages the service. PDF handling will be added later.

See the [documentation](https://colrev-environment.github.io/bibliography-io/)
for installation details, usage examples, and the generated API reference.

## Development

See the [development documentation](https://colrev-environment.github.io/bibliography-io/development.html)
for test, lint, and documentation build instructions.

## Related Python Libraries

The following libraries support additional bibliographic formats, conversion workflows, and reference extraction:

- [**BibtexParser**](https://github.com/sciunto-org/python-bibtexparser) — A lightweight, actively maintained library for reading, writing, and modifying `.bib` files. Supports customizable parsing, field transformations, and BibTeX writing. Best suited for general-purpose BibTeX processing.
- [**Pybtex**](https://pybtex.org/) — A bibliography processing library that supports BibTeX parsing, citation formatting, and conversion between bibliography formats. Provides extensible formatting styles and output formats such as LaTeX, HTML, and plain text. Best suited for generating formatted references.
- [**biblib**](https://pypi.org/project/biblib/) — A library for managing bibliographic databases, including merging entries, accessing metadata, and retrieving references using DOI or ISBN identifiers. Useful for building custom bibliography management workflows, although its last PyPI release was in 2018.
- [**rispy**](https://github.com/MrTango/rispy) — A lightweight Python library for reading and writing RIS files. Supports configurable tag mappings and custom parsers/writers. Best suited for standalone RIS processing.
- [**pybtexris**](https://pypi.org/project/pybtexris/) — A Pybtex plugin for reading RIS files and converting them into formats such as BibTeX. Also supports bibliography formatting through Pybtex.
- [**pybtexnbib**](https://pypi.org/project/pybtexnbib/) — A Pybtex plugin for parsing PubMed/MEDLINE (`.nbib`) files. Useful for integrating PubMed references into BibTeX and other bibliography workflows.
- [**endnote-utils**](https://pypi.org/project/endnote-utils/) — A utility for converting EndNote XML and RIS records to CSV, JSON, and Excel formats. Also provides command-line tools for bibliographic data processing.
- [**Biopython (Bio.Medline)**](https://biopython.org/) — Provides parsers for MEDLINE-formatted records, including PubMed exports. Best suited for biomedical literature processing.
- [**pubmed-parser**](https://github.com/titipata/pubmed_parser) — A Python library for extracting structured metadata from PubMed and PubMed Central XML datasets. Particularly useful for large-scale biomedical literature analysis.

## License

This project is distributed under the [MIT License](LICENSE).
