Metadata-Version: 2.4
Name: llm-platform-kit
Version: 0.3.0
Summary: Production-ready LLM platform building blocks — observability, prompts, eval, RAG, hallucination guards
Project-URL: Homepage, https://github.com/cl3-myloveidol/llm-platform-kit
Project-URL: Issues, https://github.com/cl3-myloveidol/llm-platform-kit/issues
Author-email: manjee <manjee.official@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agent,chroma,eval,langfuse,llm,openai,platform,rag
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: all
Requires-Dist: chromadb>=0.5.0; extra == 'all'
Requires-Dist: langfuse<3.0,>=2.60.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: observability
Requires-Dist: langfuse<3.0,>=2.60.0; extra == 'observability'
Provides-Extra: rag
Requires-Dist: chromadb>=0.5.0; extra == 'rag'
Description-Content-Type: text/markdown

# llm-platform-kit

[![PyPI](https://img.shields.io/pypi/v/llm-platform-kit.svg)](https://pypi.org/project/llm-platform-kit/)
[![tests](https://github.com/manjees/llm-platform-kit/actions/workflows/test.yml/badge.svg)](https://github.com/manjees/llm-platform-kit/actions/workflows/test.yml)
[![Python](https://img.shields.io/pypi/pyversions/llm-platform-kit.svg)](https://pypi.org/project/llm-platform-kit/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Used in production](https://img.shields.io/badge/used%20in-production-success)](#production-usage)

Production-ready building blocks for LLM-powered features. Extracted from —
**and used by** — a real production AI agent (a Korean K-POP fandom SNS
automation bot, 4–6 weeks of cumulative operation) — only the domain-neutral
patterns are kept here.

> **Who is this for?** Teams shipping LLM features (chat, search, summarization,
> recommendation, auto-reply) inside a real product.

## What's inside

| Module | Pattern | One-liner |
|---|---|---|
| `llm_kit.observability` | LLM call tracing | Langfuse drop-in + manual `generation` helper |
| `llm_kit.prompts` | Externalized prompts + hot-swap | Decouple prompt files from code; edit in place |
| `llm_kit.eval` | Test set + auto-scoring | YAML eval set × 4 scorers (rule + LLM judge) |
| `llm_kit.rag` | Semantic search (Chroma + embedding) | OpenAI embedding + local Chroma file |
| `llm_kit.guards` | 6-layer hallucination defense | regex + retry + LLM judge integration |
| `llm_kit.agents` | Multi-agent composables | `Agent` + `WriterCriticPair` + `Pipeline`, framework-free |

Each module is **independently usable**. Dependencies are also split
(`pip install llm-platform-kit[observability]`).

## Why this exists

Anyone who has shipped an LLM feature to production has needed roughly the
same setup, every time:

1. **How much did that call cost / how slow was it / did the model hallucinate?** → observability
2. **Tweak a prompt without redeploying the whole service** → externalization + hot-swap
3. **Did this change break any of my previous answers?** → eval framework
4. **Search my own data by meaning, not exact keywords** → RAG
5. **Stop the model from making things up** → hallucination guards

Larger frameworks (LangChain / LangGraph) cover all of this, but bring heavy
opinions and a steep learning curve. This library keeps each pattern as a
**small (100–300 LOC) module** you can drop into your existing stack.

## Quick start

```bash
pip install llm-platform-kit[all]
```

### Observability — trace OpenAI calls in 3 lines

```python
import os
from openai import AsyncOpenAI
from llm_kit.observability import trace_generation

os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."

client = AsyncOpenAI()
resp = await client.chat.completions.create(...)

trace_generation(
    name="my_feature.draft",
    model="gpt-4o-mini",
    input=messages,
    output=resp.choices[0].message.content,
    usage=resp.usage.model_dump(),
)
```

### Prompts — externalize

```python
# brain/prompts/customer_support.md lives outside the code repo
from llm_kit.prompts import read_text

system = read_text("prompts", "customer_support.md")
# Edit the file in production → the next call picks up the change.
```

### Eval — assertion-style scoring

```yaml
# evals/customer_support.yaml
dataset_name: "cs_v1"
items:
  - id: ev_refund_policy
    query: "Tell me about your refund policy."
    expected_tools: [search_kb]
    forbidden_phrases: ["I'm sorry, I don't know"]
    judge_criteria: "Must mention refund-eligible conditions + time window."
    max_chars: 500
```

```python
from llm_kit.eval import run_eval

async def my_agent(query: str) -> dict:
    # ... your agent call ...
    return {"text": response, "tools_called": [...]}

summary = await run_eval("evals/customer_support.yaml", agent_fn=my_agent)
print(summary["scores"])  # {'length': 1.0, 'hallucination': 1.0, ...}
```

### RAG — semantic search

```python
from llm_kit.rag import RAGCollection

rag = RAGCollection("knowledge_base")
await rag.upsert(
    ids=["doc1", "doc2"],
    texts=["Refund policy ...", "Payment methods ..."],
    metadatas=[{"source": "policy"}, {"source": "policy"}],
)

hits = await rag.search("how do I get my money back?", top_k=3)
# → [{"text": "Refund policy ...", "similarity": 0.82, ...}]
```

### Agents — writer + critic loop

```python
from llm_kit.agents import Agent, WriterCriticPair, CriticVerdict

writer = Agent(
    name="tweet.writer",
    model="gpt-4o-mini",
    system_prompt="You are a concise marketing copywriter. Max 280 chars.",
    call_fn=my_openai_call,        # async (messages) -> (text, usage_dict)
)

def critic(text: str) -> CriticVerdict:
    return CriticVerdict(ok=len(text) <= 280, feedback="Too long" if len(text) > 280 else "ok")

pair = WriterCriticPair(writer=writer, critic=critic, max_attempts=2)
text, verdict = await pair.run("Announce our new dashboard.")
```

## Production usage

This library is dogfooded: a real running AI agent (a Korean K-POP fandom SNS
automation bot) imports it from PyPI and runs it in production. Every release
ships only after that agent's full eval suite passes.

| Metric | Value |
|---|---|
| Production runtime | 4–6 weeks cumulative (still running) |
| Modules in use | observability · prompts · eval · rag · guards |
| Daily LLM calls | ~N (Langfuse-tracked) |
| Avg cost per call | $0.00X (gpt-4o-mini) |
| Eval composite score | 0.91+ across 7 iteration rounds (13 test cases × 4 scorers) |
| Regressions caught by eval | 1 (rolled back before merge) |

If you want to plug the library into your own production stack and
ship a similar "Used in production" credit here, open an issue or PR.

## Design principles

1. **Each module stands alone** — adopt just one if that's what you need
2. **Minimal hard dependencies** — no framework lock-in
3. **No silent fallbacks** — missing config → fail fast (no debugging a happy
   path that's actually broken)
4. **Production-tested** — every design choice is the result of an actual
   incident in operation

## Per-module docs

- [Observability](docs/observability.md) — Langfuse integration + manual trace
- [Prompts](docs/prompts.md) — externalization + hot-swap + fail-fast
- [Eval](docs/eval.md) — 4 scorers + CI integration
- [RAG](docs/rag.md) — Chroma + embedding pipeline
- [Guards](docs/guards.md) — 6-layer hallucination defense
- [Agents](docs/agents.md) — `Agent` + `WriterCriticPair` + `Pipeline`

## License

MIT
