Metadata-Version: 2.4
Name: semantic-redis-cache
Version: 0.1.5
Summary: Semantic caching wrapper for Redis — stop paying for duplicate LLM calls
Project-URL: Homepage, https://github.com/ausaf/semcache
Project-URL: Repository, https://github.com/ausaf/semcache
Project-URL: Issues, https://github.com/ausaf/semcache/issues
Author-email: Ausaf <kausaf141@gmail.com>
License: MIT License
        
        Copyright (c) 2024 Ausaf
        
        Permission is hereby granted, free of charge,
        to any person obtaining a copy of this software
        to deal in the Software without restriction,
        including the rights to use, copy, modify, merge,
        publish, distribute, sublicense, and/or sell copies.
        
        THE SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY
        OF ANY KIND.
License-File: LICENSE
Keywords: ai,cache,embeddings,llm,nlp,redis,semantic,vector
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.8
Requires-Dist: numpy>=1.21.0
Requires-Dist: redis>=4.0.0
Requires-Dist: sentence-transformers>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# semantic-redis-cache

Semantic caching for Redis.
Stop paying for duplicate LLM calls.

[![PyPI version](https://badge.fury.io/py/semantic-redis-cache.svg)](https://badge.fury.io/py/semantic-redis-cache)
[![Python](https://img.shields.io/pypi/pyversions/semantic-redis-cache)](https://pypi.org/project/semantic-redis-cache/)
[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

---

## The Problem

Traditional caching misses similar queries:

```
"what is monthly revenue?"  → cache miss ❌
"show me monthly revenue"   → cache miss ❌
"monthly revenue figures?"  → cache miss ❌
```

All three mean the same thing.
You pay for 3 LLM calls instead of 1.

---

## The Solution

`semantic-redis-cache` uses sentence embeddings to find semantically similar cached responses. Zero cost embeddings — runs locally on your machine.

```
"what is monthly revenue?"  → cache miss  → calls LLM → stores result
"show me monthly revenue"   → cache HIT ✅ → returns cached result
"monthly revenue figures?"  → cache HIT ✅ → returns cached result
```

---

## Install

```bash
pip install semantic-redis-cache
```

---

## Requirements

- Redis running locally or remotely
- Python 3.8+

```bash
# Run Redis locally with Docker
docker run -d -p 6379:6379 redis
```

---

## Quick Start

```python
from semcache import SemanticRedisCache

# initialize cache
cache = SemanticRedisCache(threshold=0.85)

# define your LLM function
def call_llm(question: str) -> str:
    # replace with your actual LLM call
    # works with OpenAI, Claude, Groq, Ollama — anything
    return "Your LLM response here"

# get_or_set — checks cache first, calls LLM only on miss
result = cache.get_or_set(
    query="what is monthly revenue?",
    func=call_llm,
    question="what is monthly revenue?"
)

print(result.value)      # LLM response
print(result.hit)        # False (first call — cache miss)
print(result.similarity) # 0.0

# second call — similar question
result = cache.get_or_set(
    query="show me monthly revenue",
    func=call_llm,
    question="show me monthly revenue"
)

print(result.value)      # same cached response
print(result.hit)        # True  ✅ cache hit
print(result.similarity) # 0.91  ← similarity score
```

---

## Full Example — With OpenAI

```python
from semcache import SemanticRedisCache
from openai import OpenAI

client = OpenAI()
cache = SemanticRedisCache(threshold=0.85)

def call_openai(question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": question}]
    )
    return response.choices[0].message.content

questions = [
    "what is monthly revenue?",
    "show me the monthly revenue",   # similar → cache hit
    "monthly revenue figures?",      # similar → cache hit
    "what is the weather today?",    # different → cache miss
]

for question in questions:
    result = cache.get_or_set(
        query=question,
        func=call_openai,
        question=question
    )
    status = "HIT" if result.hit else "MISS"
    print(f"[{status}] ({result.similarity:.2f}) {question}")

print(cache.metrics)
```

Output:

```
[MISS] (0.00) what is monthly revenue?
[HIT]  (0.94) show me the monthly revenue
[HIT]  (0.91) monthly revenue figures?
[MISS] (0.12) what is the weather today?

{
  'hits': 2,
  'misses': 2,
  'hit_rate': '50.0%',
  'total_cached': 2
}
```

---

## Full Example — With Claude

```python
from semcache import SemanticRedisCache
import anthropic

client = anthropic.Anthropic()
cache = SemanticRedisCache(threshold=0.85)

def call_claude(question: str) -> str:
    message = client.messages.create(
        model="claude-opus-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": question}]
    )
    return message.content[0].text

result = cache.get_or_set(
    query="explain semantic caching",
    func=call_claude,
    question="explain semantic caching"
)

print(result.value)
print(result.hit)
```

---

## Full Example — With Ollama (Free, Local)

```python
from semcache import SemanticRedisCache
import ollama

cache = SemanticRedisCache(threshold=0.85)

def call_ollama(question: str) -> str:
    response = ollama.chat(
        model="llama3",
        messages=[{"role": "user", "content": question}]
    )
    return response["message"]["content"]

result = cache.get_or_set(
    query="what is machine learning?",
    func=call_ollama,
    question="what is machine learning?"
)

print(result.value)
print(result.hit)
```

---

## Test Script — No LLM Required

Run this to verify everything is working:

```python
from semcache import SemanticRedisCache

# initialize
cache = SemanticRedisCache(
    host="localhost",
    port=6379,
    threshold=0.85,
    namespace="test"
)

# clear any existing test data
cache.flush()

# mock LLM function — no API key needed
call_count = 0
def mock_llm(question: str) -> str:
    global call_count
    call_count += 1
    return f"Answer to: {question}"

print("=" * 50)
print("semantic-redis-cache test")
print("=" * 50)

# test 1 — cache miss (first call)
result = cache.get_or_set(
    "what is monthly revenue?",
    mock_llm,
    question="what is monthly revenue?"
)
print(f"\nTest 1 — First call:")
print(f"  Hit:        {result.hit}")        # False
print(f"  LLM calls:  {call_count}")        # 1
print(f"  Value:      {result.value}")

# test 2 — semantic hit (similar question)
result = cache.get_or_set(
    "show me the monthly revenue",
    mock_llm,
    question="show me the monthly revenue"
)
print(f"\nTest 2 — Similar question:")
print(f"  Hit:        {result.hit}")        # True
print(f"  Similarity: {result.similarity}") # ~0.90
print(f"  LLM calls:  {call_count}")        # still 1 — not called again

# test 3 — cache miss (unrelated question)
result = cache.get_or_set(
    "what is the weather today?",
    mock_llm,
    question="what is the weather today?"
)
print(f"\nTest 3 — Unrelated question:")
print(f"  Hit:        {result.hit}")        # False
print(f"  LLM calls:  {call_count}")        # 2

# test 4 — metrics
print(f"\nMetrics:")
print(f"  {cache.metrics}")

# test 5 — manual get and set
cache.set("capital of France?", "Paris")
result = cache.get("what is the capital of France?")
print(f"\nTest 4 — Manual get/set:")
print(f"  Hit:        {result.hit}")        # True
print(f"  Value:      {result.value}")      # Paris
print(f"  Similarity: {result.similarity}")

print("\n" + "=" * 50)
print("All tests passed ✅")
print("=" * 50)
```

Expected output:

```
==================================================
semantic-redis-cache test
==================================================

Test 1 — First call:
  Hit:        False
  LLM calls:  1
  Value:      Answer to: what is monthly revenue?

Test 2 — Similar question:
  Hit:        True
  Similarity: 0.9034
  LLM calls:  1

Test 3 — Unrelated question:
  Hit:        False
  LLM calls:  2

Metrics:
  {'hits': 1, 'misses': 2, 'hit_rate': '33.33%', 'total_cached': 2}

Test 4 — Manual get/set:
  Hit:        True
  Value:      Paris
  Similarity: 0.9187

==================================================
All tests passed ✅
==================================================
```

---

## Configuration

```python
cache = SemanticRedisCache(
    host="localhost",                # Redis host
    port=6379,                       # Redis port
    threshold=0.85,                  # similarity threshold (0.0 - 1.0)
    ttl=86400,                       # cache TTL in seconds (24 hours)
    model_name="all-MiniLM-L6-v2",  # sentence transformer model
    namespace="semcache"             # Redis key namespace
)
```

### Threshold Guide

| Threshold | Behaviour |
|-----------|-----------|
| 0.70 | Aggressive — high hit rate, risk of wrong answers |
| 0.85 | Balanced — recommended starting point ✅ |
| 0.90 | Conservative — safe, fewer hits |
| 0.95 | Strict — almost like exact matching |

---

## API Reference

### `SemanticRedisCache`

```python
cache = SemanticRedisCache(...)
```

| Method | Description |
|--------|-------------|
| `cache.get(query)` | Search cache for similar query |
| `cache.set(query, response)` | Store query and response |
| `cache.get_or_set(query, func, *args, **kwargs)` | Check cache, call func on miss |
| `cache.delete(query)` | Delete a cached entry |
| `cache.flush()` | Clear all cached entries |
| `cache.metrics` | Get hit rate and counts |

### `CacheResult`

```python
result.value       # cached or fresh response
result.hit         # True if cache hit
result.similarity  # cosine similarity score
result.cached_at   # when it was cached
```

---

## How It Works

```
1. Query arrives
         ↓
2. Convert to embedding (local model — free)
         ↓
3. Search Redis for similar embeddings
         ↓
4. Similarity score > threshold?
         ↓ yes              ↓ no
5. Return cached      Call your LLM
   response           Store result
                      Return response
```

Embeddings are generated locally using `sentence-transformers` — zero API cost.

---

## Results

- 65-70% cache hit rate on real LLM workloads
- Zero cost embeddings — runs on your machine
- Works with any LLM — OpenAI, Claude, Groq, Ollama
- Drop-in Redis wrapper — no infrastructure changes

---

## Contributing

PRs welcome. Open an issue first for major changes.

```bash
git clone https://github.com/YOUR_USERNAME/semcache
cd semcache
pip install -e ".[dev]"
pytest tests/
```

---

## License

MIT