Metadata-Version: 2.4
Name: rupucontext
Version: 0.1.2
Summary: Local-first CLI to audit LLM context packs for duplicate chunks and wasted context — part of the Rupu family.
Project-URL: Homepage, https://github.com/EmanuelCorreaAR/rupucontext
Project-URL: Repository, https://github.com/EmanuelCorreaAR/rupucontext
Project-URL: Issues, https://github.com/EmanuelCorreaAR/rupucontext/issues
Project-URL: Changelog, https://github.com/EmanuelCorreaAR/rupucontext/blob/main/CHANGELOG.md
Author: Emanuel Correa
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: audit,cli,context,data-quality,deduplication,llm,overlap,rag
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.9
Requires-Dist: rich>=13.0.0
Requires-Dist: typer>=0.12.0
Provides-Extra: dev
Requires-Dist: build>=1.0.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Description-Content-Type: text/markdown

# RupuContext

**Lint the pack. Don't pay twice.**

Part of the **Rupu** family.

Local CLI. Deterministic JSON audit reports. Policy gates for CI. Technical signals — not certification, not token pricing, not another LLM call to "evaluate."


## Install

Requires Python 3.9+.

```bash
pip install rupucontext
rupucontext --help
```


## Quick start

```bash
rupucontext scan fixtures/dup-pack.jsonl
rupucontext scan fixtures/dup-pack.jsonl --fail-on-overlap
rupucontext compare fixtures/corpus.jsonl fixtures/questions.jsonl
```

Policy gates exit **2** when thresholds are exceeded. Exit **1** is reserved for errors. The JSON audit report (including gate) is written either way.


## Commands

| Command | Role |
|---------|------|
| `scan pack.jsonl` | Overlap inside the pack: retrieve∩retrieve, system∩retrieve, user∩retrieve, … |
| `compare corpus.jsonl questions.jsonl` | RAG leak signal (eval text in the KB) |

`scan` reports retrieve duplicates in `exact_duplicates` / `near_duplicates`, and cross-role overlap in `cross_segment` (e.g. policy pasted in system and retrieve). `compare` uses the same `segment_text` unit as `scan`.

A JSONL file with one or more `pack_id` values always emits `result.results[]` (one element per pack).

Gates are flags, not subcommands:

| Flag | Command | Meaning |
|------|---------|---------|
| `--fail-on-overlap` | scan, compare | Exit 2 if any overlap is found |
| `--max-duplicate-rate` | scan | Exit 2 if exact `duplicate_rate` exceeds threshold |
| `--max-near-duplicate-rate` | scan | Exit 2 if near-duplicate `record_rate` exceeds threshold |
| `--max-overlap-rate` | compare | Exit 2 if overlap rate exceeds threshold |

`duplicate_rate` and `record_rate` use **retrieve segments only** as the denominator (`rate_denominator: retrieve_segments`). `--near-duplicate-threshold` (default `0.85`) controls **detection** of near-duplicates. Policy gates use **rates**, not the Jaccard cutoff.


## Input format (JSONL)

One line per segment. Group by `pack_id`:

```jsonl
{"pack_id": "req-001", "role": "system", "text": "You are a helpful assistant..."}
{"pack_id": "req-001", "role": "retrieve", "chunk_id": "c42", "text": "Refund policy: items within 30 days..."}
{"pack_id": "req-001", "role": "retrieve", "chunk_id": "c17", "text": "Refund policy: items within 30 days..."}
{"pack_id": "req-001", "role": "user", "text": "Can I return this?"}
```

Roles: `system`, `retrieve`, `history`, `user` (extensible). `chunk_id` optional but recommended for retrieve segments.


## Audit report

Reports follow: `input → configuration → method → result → (optional) gate`.

```json
{
  "tool": "rupucontext",
  "version": "0.1.1",
  "command": "scan",
  "input": {
    "path": "fixtures/policy-twice.jsonl",
    "segments": 3,
    "packs": 1
  },
  "configuration": {
    "near_duplicate_threshold": 0.85,
    "fail_on_overlap": true
  },
  "method": {
    "unit": "segment_text",
    "exact": "text_exact_v1",
    "normalized": "text_normalized_v1",
    "near": "jaccard_char_shingles_v1"
  },
  "result": {
    "results": [
      {
        "pack_id": "req-policy",
        "segment_count": 3,
        "retrieve_segment_count": 1,
        "exact_duplicates": {
          "pairs": 0,
          "segments_flagged": 0,
          "duplicate_bytes": 0,
          "duplicate_rate": 0.0,
          "rate_denominator": "retrieve_segments",
          "evidence": []
        },
        "near_duplicates": {
          "pairs": 0,
          "segments_flagged": 0,
          "record_rate": 0.0,
          "rate_denominator": "retrieve_segments",
          "evidence": []
        },
        "cross_segment": [
          {"from": "retrieve", "to": "system", "method": "exact", "overlap": 1.0}
        ]
      }
    ]
  },
  "gate": {
    "passed": false,
    "rules": [
      {"metric": "overlap_pairs", "actual": 1, "threshold": 0, "passed": false}
    ]
  }
}
```

`fixtures/dup-pack.jsonl` (quick start) flags retrieve∩retrieve duplicates; `cross_segment` may be `[]` when no cross-role overlap is detected.

Byte counts and overlap ratios — not token estimates. Contract details: [docs/AUDIT.md](docs/AUDIT.md).


## Exit codes

| Code | Meaning |
|------|---------|
| `0` | Success (no policy failure) |
| `1` | Usage error — invalid file or schema |
| `2` | Policy gate failed |

```yaml
# .github/workflows/rupucontext.yml
- run: rupucontext scan fixtures/clean-pack.jsonl --fail-on-overlap
```


## What it is not

- Not token pricing (no tiktoken, no USD)
- Not prompt versioning or observability proxy
- Not semantic / paraphrase matching (on purpose)

If you can't export a pack or a trace, you're not the user yet.


## Development

```bash
git clone https://github.com/EmanuelCorreaAR/rupucontext.git
cd rupucontext
pip install -e ".[dev]"
pytest
```


## Status

**0.1.1** — `scan` + `compare`; policy gate flags; deterministic audit JSON.

**Next:** stabilize audit contract toward 1.0.


## License

Apache License 2.0
