Metadata-Version: 2.4
Name: llm-response-cache
Version: 1.0.0
Summary: SQLite-backed LLM response cache. Exact match + fuzzy match. Decorator API. Zero mandatory server dependencies.
Author: Linda Oraegbunam
License: MIT
Project-URL: Homepage, https://github.com/obielin/llm-cache
Keywords: llm,cache,anthropic,cost-reduction,sqlite,decorator,semantic-cache
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: black>=24.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Dynamic: license-file

# llm-cache

**SQLite-backed LLM response cache. Exact match + fuzzy match. Decorator API. Zero mandatory server dependencies.**

[![Tests](https://img.shields.io/badge/Tests-34%20passing-brightgreen?style=flat-square)](tests/)
[![Dependencies](https://img.shields.io/badge/Dependencies-zero%20mandatory-brightgreen?style=flat-square)](pyproject.toml)
[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?style=flat-square)](pyproject.toml)
[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](LICENSE)
[![LinkedIn](https://img.shields.io/badge/-Linda_Oraegbunam-blue?logo=linkedin&style=flat-square)](https://www.linkedin.com/in/linda-oraegbunam/)

---

## Why llm-cache?

Every LLM API call costs money and takes time. In development and testing, you hit the same prompts over and over. In production, users ask the same questions. `llm-cache` stores responses in a local SQLite database and serves them instantly — no Redis, no server, no external service.

Two match layers:
- **Exact match** — SHA-256 hash of the normalised prompt. O(1) lookup.
- **Fuzzy match** — token overlap similarity for near-identical prompts. Catches rephrasings without embeddings.

---

## Install

```bash
pip install llm-cache
```

---

## Quick start

```python
from llm_cache import LLMCache

cache = LLMCache()  # stores in .llm_cache.db

# Decorator — wraps any LLM call function
@cache.cached(model="claude-opus-4-5")
def ask(prompt: str) -> str:
    return client.messages.create(
        model="claude-opus-4-5",
        messages=[{"role": "user", "content": prompt}],
    ).content[0].text

ask("What is RAG?")   # hits API first time
ask("What is RAG?")   # served from cache instantly — no API call
ask("What is rag?")   # fuzzy match — also served from cache
```

---

## Direct API

```python
from llm_cache import LLMCache, MatchType

cache = LLMCache("myapp.db", fuzzy_threshold=0.85, ttl_seconds=86400)

# Get
response, match = cache.get("What is RAG?")
if match == MatchType.MISS:
    response = call_api("What is RAG?")
    cache.set("What is RAG?", response, model="claude-opus-4-5", tags=["qa"])

# Match types
# MatchType.EXACT  — identical prompt
# MatchType.FUZZY  — near-match above threshold
# MatchType.MISS   — not cached

# Stats
print(cache.stats())
# CacheStats(entries=142, hit_rate=73%, tokens_saved≈48,200)

# Management
cache.delete(tags=["qa"])              # delete by tag
cache.delete(older_than_seconds=3600)  # expire old entries
cache.delete(model="claude-opus-4-5") # delete by model
cache.clear()                          # wipe everything
```

---

## Configuration

```python
cache = LLMCache(
    db_path=".llm_cache.db",    # SQLite file path
    fuzzy_threshold=0.85,        # 0-1, how similar prompts must be for fuzzy hit
    ttl_seconds=86400,           # auto-expire entries after N seconds (None = never)
    max_entries=10000,           # evict oldest when cache exceeds this size
)
```

---

## Cost savings estimate

```python
stats = cache.stats()
print(f"Tokens saved:  {stats.estimated_tokens_saved:,}")
print(f"Estimated saving: ~£{stats.cost_savings_estimate:.2f}")
print(f"Hit rate: {stats.hit_rate:.0%}")
```

---

## Context manager

```python
with LLMCache("session.db") as cache:
    result, _ = cache.get("my prompt")
```

---

**Linda Oraegbunam** | [LinkedIn](https://www.linkedin.com/in/linda-oraegbunam/) | [Twitter](https://twitter.com/Obie_Linda) | [GitHub](https://github.com/obielin)
