Metadata-Version: 2.4
Name: genruntime
Version: 0.4.0
Summary: LLM-as-a-Service runtime: reliable, observable, schema-validated inference
Author: GenRuntime
License: MIT
Project-URL: Repository, https://github.com/tejaspawar12/GenRuntime
Keywords: llm,inference,runtime,anthropic,claude
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: anthropic>=0.18.0
Requires-Dist: openai>=1.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn[standard]>=0.22.0
Requires-Dist: prometheus_client>=0.19.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: httpx>=0.24.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Dynamic: license-file

# GenRuntime

**LLM-as-a-Service platform for reliable, observable, schema-validated inference.**

[![CI](https://github.com/tejaspawar12/GenRuntime/actions/workflows/ci.yml/badge.svg)](https://github.com/tejaspawar12/GenRuntime/actions/workflows/ci.yml)

---

## Why GenRuntime

**Problem:** Applications that call LLM provider APIs directly (e.g. Anthropic/OpenAI SDK) are fragile — no retries, no output validation, no observability. Failures and debugging are harder in production.

**Solution:** GenRuntime is a backend layer between your app and LLM providers. It adds **retries** (with backoff), **schema-validated outputs** (Pydantic), **structured logging** (JSON lines), and **Prometheus metrics**. Use it in code with `from genruntime import generate` or via HTTP `POST /generate`. One code path serves both (single source of truth).

---

## Features

| Feature | Description |
|--------|-------------|
| Retries | Exponential backoff on transient failures (network, 5xx, rate limit); configurable max retries |
| Schema validation | `generate_structured(prompt, schema=YourModel)` — parse JSON and validate with Pydantic; retry once on validation failure |
| Structured logging | Every request logged as one JSON line (request_id, model, latency_ms, tokens, status); no PII (prompt/response length only) |
| request_id | Unique ID per request in response and logs for tracing |
| Health & readiness | `GET /health` (liveness), `GET /ready` (config and env check; no provider ping) |
| Prometheus metrics | `GET /metrics` — request count, latency histogram, error count (by model/status) |
| Docker | Dockerfile and docker-compose; run API with env-based config |
| Multi-provider | **Mock** (no key), **Anthropic Claude**, and **OpenAI**; config-driven via `GENRUNTIME_PROVIDER` and `GENRUNTIME_MODEL` |

---

## Architecture

Request flow: **API → runtime → provider adapter → provider**. The runtime handles retry, validate, and log. One code path serves both the Python library and the HTTP API (single source of truth).

```
Applications (Python or HTTP)
         │
         ▼
┌─────────────────────────────────────┐
│  GenRuntime                          │
│  ┌─────────┐  ┌─────────┐  ┌───────┐ │
│  │   API   │→ │ Runtime │→ │ Logger│ │
│  └─────────┘  └────┬────┘  └───────┘ │
│                    │                 │
│              ┌─────▼─────┐           │
│              │ Providers │           │
│              └─────┬─────┘           │
└────────────────────┼─────────────────┘
                     ▼
              LLM providers (Claude, …)
```

---

## Requirements

- Python 3.10+

---

## Development setup

Use a **virtual environment** so project dependencies stay isolated from your system Python and other projects.

**1. Create and activate a virtual environment**

```bash
# Create (run from project root)
python -m venv .venv

# Activate — Windows (PowerShell)
.venv\Scripts\Activate.ps1

# Activate — Windows (CMD)
.venv\Scripts\activate.bat

# Activate — macOS / Linux
source .venv/bin/activate
```

**2. Install the package in editable mode (with dev dependencies)**

```bash
pip install -e ".[dev]"
```

This installs `genruntime`, `anthropic`, `pydantic`, `fastapi`, `uvicorn`, plus `pytest`, `httpx`, `ruff` for development.

**3. Verify**

```bash
python -c "import genruntime; print(genruntime.__all__)"
pytest tests/unit -v
```

The `.gitignore` already excludes `.venv` and `venv/`, so the environment is not committed.

---

## Quick start

### Option 1: Python library

```bash
pip install -e .
```

Set your API key (or use the mock provider without a key):

```bash
# Optional: use real Claude (required for GENRUNTIME_PROVIDER=anthropic)
set ANTHROPIC_API_KEY=your_key_here

# Or use OpenAI: set GENRUNTIME_PROVIDER=openai and OPENAI_API_KEY=...
# Or use mock provider (no key needed)
set GENRUNTIME_PROVIDER=mock
```

Then in Python:

```python
from genruntime import generate

response = generate("Hello, world!")
print(response.text)
print(response.meta.request_id)
```

**Basic usage (with defaults):**

```python
from genruntime import generate, configure

configure(model="claude-3-haiku-20240307")

response = generate("Explain neural networks simply")
print(response.text)
print(response.meta.latency_ms)
print(response.meta.request_id)
```

**Structured output:**

```python
from pydantic import BaseModel
from genruntime import generate_structured

class Summary(BaseModel):
    topic: str
    summary: str

result = generate_structured("Summarize neural networks", schema=Summary)
print(result.parsed.summary)
print(result.meta.request_id)
```

### Option 2: HTTP API with Docker

Build and run (pass your API key, or use mock):

```bash
docker build -t genruntime .
docker run -p 8000:8000 -e ANTHROPIC_API_KEY=your_key_here genruntime
```

Or with docker-compose (create a `.env` file with `ANTHROPIC_API_KEY=...` or set `GENRUNTIME_PROVIDER=mock`):

```bash
docker-compose up --build
```

Then call the API:

```bash
curl -X POST http://localhost:8000/generate ^
  -H "Content-Type: application/json" ^
  -d "{\"prompt\": \"Hello\"}"
```

On macOS/Linux use `\` instead of `^` and single quotes:

```bash
curl -X POST http://localhost:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Hello"}'
```

Health and readiness:

```bash
curl http://localhost:8000/health
curl http://localhost:8000/ready
```

Optional Prometheus metrics:

```bash
curl http://localhost:8000/metrics
```

---

## Configuration

All runtime and API behaviour can be configured via environment variables (12-factor style; no secrets in code).

| Variable | Description | Default |
|----------|-------------|---------|
| `ANTHROPIC_API_KEY` | API key for Anthropic Claude (required if provider is `anthropic`) | — |
| `OPENAI_API_KEY` | API key for OpenAI (required if provider is `openai`) | — |
| `GENRUNTIME_PROVIDER` | Provider: `anthropic`, `openai`, or `mock` | `anthropic` if ANTHROPIC_API_KEY set, else `openai` if OPENAI_API_KEY set, else `mock` |
| `GENRUNTIME_MODEL` | Default model (e.g. `claude-3-haiku-20240307`, `gpt-4o-mini`) | `claude-3-haiku-20240307` |
| `GENRUNTIME_TIMEOUT` | Request timeout in seconds | `60` |
| `GENRUNTIME_MAX_RETRIES` | Max retries on transient failure | `2` |

You can also set defaults in code with `configure(model=..., timeout=..., max_retries=...)` (see Public API below).

---

## Public API (library)

The stable surface is **8 items**. Everything else is internal.

| Item | Description |
|------|-------------|
| `generate(prompt, **options)` | Run inference; returns `GenRuntimeResponse` (`.text`, `.meta`). Options: `model`, `timeout`, `max_retries`, `provider`. |
| `generate_structured(prompt, schema, **options)` | Same as above; parses response as JSON and validates with Pydantic `schema`. Returns `GenRuntimeStructuredResponse` (`.parsed`, `.meta`). Retries once on validation failure. |
| `configure(**kwargs)` | Set default `model`, `timeout`, `max_retries` for function-based usage. |
| `GenRuntime` | Class with optional defaults; `.generate()` and `.generate_structured()` delegate to the same runtime. |
| `GenRuntimeResponse` | Response type: `.text` (str), `.meta` (GenRuntimeMeta). |
| `GenRuntimeStructuredResponse` | Response type: `.parsed` (Pydantic model), `.meta` (GenRuntimeMeta). |
| `GenRuntimeError` | Base exception for all runtime failures. |
| `ValidationError` | Raised when structured output fails schema validation after retries. |

---

## Development

Run tests and lint:

```bash
pytest tests/unit tests/integration -v
ruff check .
```
