Metadata-Version: 2.4
Name: ragone-ai
Version: 1.0.0
Summary: ModernBERT-based RAG evaluation model
Author: Hariharan Suthan
License-Expression: Apache-2.0
Keywords: RAG,retrieval-augmented-generation,evaluation,ModernBERT,NLP
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1
Requires-Dist: transformers>=4.45
Requires-Dist: huggingface_hub>=0.25
Requires-Dist: safetensors>=0.4
Dynamic: license-file

# RAGOne

**A lightweight, ModernBERT-based model for evaluating retrieved context in RAG pipelines.**

RAGOne evaluates retrieved context across four tasks: answerability,
evidence quality, contradiction detection, and relevance classification.

- **Hugging Face model:** [Hariharan05/ragone](https://huggingface.co/Hariharan05/ragone)
- **PyPI package:** `ragone-ai`
- **Python import:** `from ragone import RAGOne`

## Installation

```bash
pip install ragone-ai
```

## Quick start

```python
from ragone import RAGOne

model = RAGOne.from_pretrained("Hariharan05/ragone")

result = model.predict(
    query="What is the refund period?",
    context="Customers can request a refund within 30 days.",
    task="answerability",
    options=["yes", "no"],
)

print(result)
```

On first use, the model files are downloaded from Hugging Face.
The package includes the inference code; model weights are hosted
separately in the Hugging Face repository.

## Supported tasks

| Task | Purpose | Example options |
|---|---|---|
| `answerability` | Can the query be answered from the context? | `["yes", "no"]` |
| `evidence_quality` | How strong is the context as evidence? | `["strong", "medium", "weak"]` |
| `contradiction` | Does the statement contradict the context? | `["yes", "no"]` |
| `relevance` | Is the context relevant to the query? | `["relevant", "irrelevant"]` |

Use labels consistent with the model's training conventions.

## Example output

```python
{
    "prediction": "yes",
    "scores": [
        {"label": "yes", "score": 0.89},
        {"label": "no", "score": 0.11}
    ],
    "latency_ms": 85.0,
    "task": "answerability"
}
```

Scores and latency above are illustrative, not guaranteed results.
Scores are normalized across candidate options and are not necessarily
calibrated probabilities of correctness.

## Model and training

RAGOne uses a fine-tuned `answerdotai/ModernBERT-base` encoder and a
neural scoring head to rank candidate labels.

Reported training details:

- Dataset: 5,074 records
- Training: 5 epochs
- Best reported validation loss: 0.2697
- Reported final validation accuracy: 94.591%
- Separate evaluation: 37/40 correct (92.5%)

The separate evaluation used 10 examples per task. These results are
preliminary and do not establish performance on all real-world RAG
systems. The training and evaluation metrics come from different
evaluations and should not be compared as if they were the same test.

## Intended use

RAGOne is intended for research, experimentation, and development
of RAG evaluation workflows. It can help assess retrieved-context
relevance, answerability, evidence quality, and contradiction.

It does not perform document retrieval or answer generation and
should not be treated as a definitive source of truth.

## Limitations

- Performance may vary by domain, language, and retrieval strategy.
- Inputs are limited to the model's configured token length.
- Results depend on the candidate labels supplied.
- Softmax scores should not be interpreted as calibrated confidence
  without additional validation.
- Evaluate on independent, representative datasets before production use.

## License

RAGOne is licensed under the Apache License 2.0.

You may use, modify, and distribute this software under its terms.
See the [LICENSE](LICENSE) file or
https://www.apache.org/licenses/LICENSE-2.0 for the full license text.

