Metadata-Version: 2.4
Name: nerxiv
Version: 1.1.0
Summary: A Python package for extracting structured metadata information from arXiv papers.
Author-email: "Jose M. Pizarro" <jose.pizarro-blanco@bam.de>
Maintainer-email: "Jose M. Pizarro" <jose.pizarro-blanco@bam.de>
License: Copyright (c) 2025 Jose M. Pizarro
        
        This software is licensed under the PolyForm Noncommercial License 1.0.0.
        You may use this software for any noncommercial purpose, subject to the conditions
        set forth in the license. Commercial use is prohibited without a separate license
        from the copyright holder.
        
        
        # PolyForm Noncommercial License 1.0.0
        
        <https://polyformproject.org/licenses/noncommercial/1.0.0>
        
        ## Acceptance
        
        In order to get any license under these terms, you must agree
        to them as both strict obligations and conditions to all
        your licenses.
        
        ## Copyright License
        
        The licensor grants you a copyright license for the
        software to do everything you might do with the software
        that would otherwise infringe the licensor's copyright
        in it for any permitted purpose.  However, you may
        only distribute the software according to [Distribution
        License](#distribution-license) and make changes or new works
        based on the software according to [Changes and New Works
        License](#changes-and-new-works-license).
        
        ## Distribution License
        
        The licensor grants you an additional copyright license
        to distribute copies of the software.  Your license
        to distribute covers distributing the software with
        changes and new works permitted by [Changes and New Works
        License](#changes-and-new-works-license).
        
        ## Notices
        
        You must ensure that anyone who gets a copy of any part of
        the software from you also gets a copy of these terms or the
        URL for them above, as well as copies of any plain-text lines
        beginning with `Required Notice:` that the licensor provided
        with the software.  For example:
        
        > Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
        
        ## Changes and New Works License
        
        The licensor grants you an additional copyright license to
        make changes and new works based on the software for any
        permitted purpose.
        
        ## Patent License
        
        The licensor grants you a patent license for the software that
        covers patent claims the licensor can license, or becomes able
        to license, that you would infringe by using the software.
        
        ## Noncommercial Purposes
        
        Any noncommercial purpose is a permitted purpose.
        
        ## Personal Uses
        
        Personal use for research, experiment, and testing for
        the benefit of public knowledge, personal study, private
        entertainment, hobby projects, amateur pursuits, or religious
        observance, without any anticipated commercial application,
        is use for a permitted purpose.
        
        ## Noncommercial Organizations
        
        Use by any charitable organization, educational institution,
        public research organization, public safety or health
        organization, environmental protection organization,
        or government institution is use for a permitted purpose
        regardless of the source of funding or obligations resulting
        from the funding.
        
        ## Fair Use
        
        You may have "fair use" rights for the software under the
        law. These terms do not limit them.
        
        ## No Other Rights
        
        These terms do not allow you to sublicense or transfer any of
        your licenses to anyone else, or prevent the licensor from
        granting licenses to anyone else.  These terms do not imply
        any other licenses.
        
        ## Patent Defense
        
        If you make any written claim that the software infringes or
        contributes to infringement of any patent, your patent license
        for the software granted under these terms ends immediately. If
        your company makes such a claim, your patent license ends
        immediately for work on behalf of your company.
        
        ## Violations
        
        The first time you are notified in writing that you have
        violated any of these terms, or done anything with the software
        not covered by your licenses, your licenses can nonetheless
        continue if you come into full compliance with these terms,
        and take practical steps to correct past violations, within
        32 days of receiving notice.  Otherwise, all your licenses
        end immediately.
        
        ## No Liability
        
        ***As far as the law allows, the software comes as is, without
        any warranty or condition, and the licensor will not be liable
        to you for any damages arising out of these terms or the use
        or nature of the software, under any kind of legal claim.***
        
        ## Definitions
        
        The **licensor** is the individual or entity offering these
        terms, and the **software** is the software the licensor makes
        available under these terms.
        
        **You** refers to the individual or entity agreeing to these
        terms.
        
        **Your company** is any legal entity, sole proprietorship,
        or other kind of organization that you work for, plus all
        organizations that have control over, are under the control of,
        or are under common control with that organization.  **Control**
        means ownership of substantially all the assets of an entity,
        or the power to direct its management and policies by vote,
        contract, or otherwise.  Control can be direct or indirect.
        
        **Your licenses** are all the licenses granted to you for the
        software under these terms.
        
        **Use** means anything you do with the software requiring one
        of your licenses.
        
Project-URL: Homepage, https://github.com/JosePizarro3/NERxiv
Project-URL: Bug Tracker, https://github.com/JosePizarro3/NERxiv/issues
Classifier: Natural Language :: English
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.2.1
Requires-Dist: sentence-transformers
Requires-Dist: langchain-core>=0.3.20
Requires-Dist: langchain-ollama
Requires-Dist: langchain-huggingface
Requires-Dist: langchain-text-splitters
Requires-Dist: pyrxiv>=1.0.1
Requires-Dist: pymatgen
Requires-Dist: spacy
Requires-Dist: h5py~=3.12.1
Requires-Dist: structlog~=24.4.0
Provides-Extra: dev
Requires-Dist: mypy>=1.8.0; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-timeout; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: python-decouple; extra == "dev"
Provides-Extra: docu
Requires-Dist: marimo[recommended]; extra == "docu"
Requires-Dist: mkdocs-material; extra == "docu"
Requires-Dist: mkdocs-click; extra == "docu"
Requires-Dist: mkdocstrings; extra == "docu"
Requires-Dist: mkdocstrings-python; extra == "docu"
Requires-Dist: mike; extra == "docu"
Dynamic: license-file

<h1 align="center">
  <picture>
    <source srcset="https://github.com/JosePizarro3/NERxiv/raw/main/docs/assets/nerxiv_logo_name.png">
    <img src="https://github.com/JosePizarro3/NERxiv/raw/main/docs/assets/nerxiv_logo_name.png"
         alt="NERxiv logo"
         style="width: 25rem">
  </picture>
</h1>


<h4 align="center">

![CI](https://github.com/JosePizarro3/NERxiv/actions/workflows/actions.yml/badge.svg)
[![Coverage Status](https://coveralls.io/repos/github/JosePizarro3/NERxiv/badge.svg?branch=main&nocache=1)](https://coveralls.io/github/JosePizarro3/NERxiv?branch=main)
[![License: PolyForm NC 1.0.0](https://img.shields.io/badge/license-PolyForm_NC_1.0.0-orange.svg)](LICENSE)
[![PyPI version](https://img.shields.io/pypi/v/nerxiv.svg)](https://img.shields.io/pypi/v/nerxiv.svg)
[![Python versions](https://img.shields.io/pypi/pyversions/nerxiv.svg)](https://img.shields.io/pypi/pyversions/nerxiv.svg)
<!--[![Commercial License Available](https://img.shields.io/badge/commercial-license-green.svg)](COMMERCIAL-LICENSE.md)-->

</h4>

# NERxiv

**N**amed **E**ntity **R**ecognition for ar**xiv** papers (**NERxiv**) is a Python wrapper tool for extracting **structured metadata** from scientific papers on [arXiv](https://arxiv.org) using **LLMs** and modern **retrieval-augmented generation (RAG)** techniques.

Visit the [documentation page](https://JosePizarro3.github.io/NERxiv/) to learn how to use this tool.

## What It Does

* Uses [`pyrxiv`](https://pypi.org/project/pyrxiv/) to fetch, download, and extract text from arXiv papers
* Chunks and embeds text with SentenceTransformers or LangChain to categorize papers content using local LLMs (via Ollama)
* Includes CLI tools and notebook tutorials for reproducible workflows

---

## Installation

Install the core package:
```bash
pip install nerxiv
```

## Running LLMs Locally

We recommend running your own models locally using [Ollama](https://ollama.com/download):
```bash
# Install Ollama (follow instructions on their website)
ollama pull <model-name>   # e.g., llama3, deepseek-r1, qwen3:30b

# Start the local server
ollama serve
```


---

# Development

To contribute to `NERxiv` or run it locally, follow these steps:


## Clone the Repository

```bash
git clone https://github.com/JosePizarro3/NERxiv.git
cd NERxiv
```

## Set Up a Virtual Environment

We recommend Python ≥ 3.10:
```bash
python3 -m venv .venv
source .venv/bin/activate
```

## Install Dependencies

Use [`uv`](https://docs.astral.sh/uv/) (faster than pip) to install the package in editable mode with `dev` and `docu` extras:
```bash
pip install --upgrade pip
pip install uv
uv pip install -e .[dev,docu]
```

## Run tests

Use `pytest` with verbosity to run all tests:
```bash
python -m pytest -sv tests
```


To check code coverage:
```bash
python -m pytest --cov=nerxiv tests
```

### Code formatting and linting


We use [`Ruff`](https://docs.astral.sh/ruff/) for formatting and linting (configured via `pyproject.toml`).

Check linting issues:
```bash
ruff check .
```

Auto-format code:
```bash
ruff format . --check
```

Manually fix anything Ruff cannot handle automatically.

### Documentation writing

To view the documentation locally, make sure to have installed the extra `[docu]` packages:

```sh
uv pip install -e '[docu]'
```

**Note**: This command installs `mkdocs`, `mkdocs-material`, and other documentation-related dependencies.

The first time, build the server:

```sh
mkdocs build
```

Run the documentation server:

```sh
mkdocs serve
```

The output looks like:

```sh
INFO    -  Building documentation...
INFO    -  Cleaning site directory
INFO    -  [14:07:47] Watching paths for changes: 'docs', 'mkdocs.yml'
INFO    -  [14:07:47] Serving on http://127.0.0.1:8000/
```

Simply click on `http://127.0.0.1:8000/`. The changes in the `md` files of the documentation are immediately reflected when the files are saved (the local web will automatically refresh).
