Metadata-Version: 2.4
Name: routesmith-llm
Version: 0.9.2
Summary: Adaptive LLM execution engine - intelligent routing, cascading, caching, and budget management
Project-URL: Homepage, https://github.com/routesmith/routesmith
Project-URL: Documentation, https://github.com/routesmith/routesmith#readme
Project-URL: Repository, https://github.com/routesmith/routesmith
Author: routesmith contributors
License-Expression: MIT
License-File: LICENSE
Keywords: ai,caching,cascading,cost-optimization,llm,optimization,routing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: litellm>=1.40.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: simpleeval>=0.9.13
Provides-Extra: all
Requires-Dist: routesmith[anthropic,autogen,cache,crewai,docs,dspy,langchain,predictor,proxy,tui]; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.25.0; extra == 'anthropic'
Provides-Extra: autogen
Requires-Dist: pyautogen>=0.2.0; extra == 'autogen'
Provides-Extra: cache
Requires-Dist: faiss-cpu>=1.7.0; extra == 'cache'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'cache'
Provides-Extra: crewai
Requires-Dist: crewai>=0.80.0; extra == 'crewai'
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pip-audit>=2.0.0; extra == 'dev'
Requires-Dist: prometheus-client>=0.16.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: scikit-learn>=1.3.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25.0; extra == 'docs'
Requires-Dist: pymdown-extensions>=10.0.0; extra == 'docs'
Provides-Extra: dspy
Requires-Dist: dspy-ai>=2.0.0; extra == 'dspy'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3.0; extra == 'langchain'
Provides-Extra: langchain-agents
Requires-Dist: langchain-core>=0.3.0; extra == 'langchain-agents'
Requires-Dist: langgraph-supervisor>=0.0.20; extra == 'langchain-agents'
Requires-Dist: langgraph-swarm>=0.1.0; extra == 'langchain-agents'
Requires-Dist: langgraph>=1.0.0; extra == 'langchain-agents'
Provides-Extra: predictor
Requires-Dist: scikit-learn>=1.3.0; extra == 'predictor'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'predictor'
Provides-Extra: proxy
Requires-Dist: certifi>=2024.0.0; extra == 'proxy'
Requires-Dist: httpx>=0.24.0; extra == 'proxy'
Requires-Dist: prometheus-client>=0.16.0; extra == 'proxy'
Requires-Dist: pyyaml>=6.0.0; extra == 'proxy'
Requires-Dist: questionary>=2.0.0; extra == 'proxy'
Provides-Extra: tui
Requires-Dist: textual>=0.40.0; extra == 'tui'
Description-Content-Type: text/markdown

# RouteSmith

[![PyPI](https://img.shields.io/pypi/v/routesmith-llm)](https://pypi.org/project/routesmith-llm/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://pypi.org/project/routesmith-llm/)
[![License](https://img.shields.io/github/license/yunpengl9071/routesmith)](LICENSE)

**The smart router for AI coding tools.** Backed by contextual bandit research — see [Research](#research) for measured results.

```bash
pip install "routesmith-llm[proxy]"
routesmith quickstart        # detects your provider keys, builds a matching model pool
routesmith run claude        # or: routesmith run codex / routesmith run opencode
```

That's it — Claude Code (or Codex, or OpenCode) now runs exactly as normal, except every
request is silently routed to the best model for it. No tool config to edit, no model
picker, no server terminal to babysit.

RouteSmith sits between your AI coding tool and the LLM. It routes every request
to the best model for that specific task — cheap models for simple edits, frontier
models for complex refactors. You never think about model IDs again.

| | OpenRouter Auto | RouteSmith |
|---|---|---|
| Learns from your traffic | ❌ | ✅ Online bandit learning |
| Self-hosted / no data leak | ❌ | ✅ Open source, self-hosted |
| Custom model pool | ❌ | ✅ Any model, any provider |
| Custom rewards & policies | ❌ | ✅ Per-role, configurable reward fns |
| Decision audit log | ❌ | ✅ Full routing decision trace |
| Budget caps (daily/hourly) | ❌ | ✅ Monthly, per-request, per-project |
| Conversation stickiness | ✅ | ✅ 

**v0.9.0**: `routesmith run <tool>` one-command integration, provider-aware model catalogs, `routesmith models refresh`, persistent conversation stickiness, `routesmith connect --verify`. See [CHANGELOG.md](CHANGELOG.md).

## Who it's for

**💰 You pay for API access. Cut your bill.**
Your Claude Code or Codex session burns through tokens on every turn, including
subagent calls. RouteSmith sends typos and formatting to gpt-4o-mini, saves Claude
Opus for architecture decisions. Paper experiments measured 45% cost savings at 71%
accuracy on 5-model routing — see [Research](#research).

**🆓 You use free models. Get better answers.**
Free models are good individually — but none is great at everything. RouteSmith
orchestrates them: hard problems get the strongest free model, easy ones get the
fastest, and weak answers cascade to second opinions.

```python
from routesmith import RouteSmith

# Free models — zero cost, smart routing
rs = RouteSmith.with_free_models()

# Or bring your own models
rs = RouteSmith()
rs.register_model("gpt-4o-mini", cost_per_1k_input=0.15, cost_per_1k_output=0.60, quality_score=0.85)
rs.register_model("claude-sonnet-4", cost_per_1k_input=3.0, cost_per_1k_output=15.0, quality_score=0.92)

response = rs.completion(messages=[{"role": "user", "content": "Explain recursion"}])
```

## AI coding tools

The fastest path for any tool is `routesmith run <tool>` (see [Quick start](#quick-start)).
For manual setup or CI, `routesmith connect <tool>` prints the exact config, and
`routesmith connect <tool> --verify` confirms requests are actually being routed
(not just reachable):

| Tool | `routesmith connect <tool>` |
|------|------------------------------|
| Claude Code | `claude-code` — sets `ANTHROPIC_BASE_URL` (native `/v1/messages`, full tool-use support) |
| Codex | `codex` — sets `OPENAI_BASE_URL` + `wire_api: chat` |
| OpenCode | `opencode` |
| OpenClaw | `openclaw` |
| pi | `pi` |
| Hermes | `hermes` |
| OpenAI SDK | `openai-sdk` |
| Anthropic SDK | `anthropic-sdk` |

[Integration guides →](docs/integrations/)

## Why RouteSmith

| | Raw OpenRouter | Manual routing | RouteSmith |
|---|---|---|---|
| Picks model per query | ❌ | 😓 You do it | ✅ Automatic |
| Cascades when answer is weak¹ | ❌ | ❌ | ✅ |
| Caches repetitive queries | ❌ | ❌ | ✅ |
| Enforces budget limits | ❌ | ❌ | ✅ |
| Tracks costs per model | ✅ | ❌ | ✅ |
| Works with 100+ models | ✅ | ❌ | ✅ |
| Zero-config start | ✅ | ❌ | ✅ |
| Learns from feedback | ❌ | ❌ | ✅ |

¹ Today all strategies select a single model; cascade execution lands in Phase 2 (see [ROADMAP.md](ROADMAP.md)).

## Features

### Intelligent Routing
- **7 predictor types**: LinUCB, LinTS, NeuralUCB, REINFORCE, WarmStart LinUCB, Adaptive (random forest), Embedding
- **35-dimensional feature space**: query type classification, difficulty estimation, model metadata
- **Online learning**: bandits improve from the first query onward — no pretraining labels needed
- **Multi-model routing**: scales to $K$ arms naturally (validated on 5-model deployments)

### Enterprise
- **Provisioned throughput support**: prioritize pre-paid capacity, overflow to on-demand
- **Compliance routing**: tag-based filtering (HIPAA, SOC2, PCI)
- **Budget enforcement**: FAIL, FALLBACK, QUEUE behaviors
- **Multi-project isolation**: per-project cost allocation and stats

### Production
- **Semantic cache**: embedding-based dedup, configurable similarity
- **Framework adapters**: LangChain, DSPy, CrewAI, AutoGen, Anthropic, OpenClaw
- **OpenAI-compatible proxy**: works with any tool, zero code changes
- **Observability**: Prometheus metrics, structured logging, cost tracking, dashboard TUI
- **Resilience**: circuit breakers, retry with backoff, health checks, Docker

## Research

RouteSmith is backed by a research paper evaluating contextual bandit routing:

- **LinTS-35d** achieves 46% cost reduction with APGR=0.593 on MMLU
- **LinUCB-35d** achieves APGR=1.126 by selective strong-arm routing
- **5-arm routing**: 45% cost savings across GPT-4o, Claude-Sonnet-4.5, Qwen-Plus, MiniMax-M1, DeepSeek-V3
- **Zero pretraining labels** — learns from ~100 queries vs. 55K+ required by supervised routers
- Sub-millisecond routing latency (<0.5ms P99)

Paper: [`paper/main.pdf`](paper/main.pdf) | Compile with: `cd paper && tectonic main.tex`

## Framework integrations

```python
# Anthropic SDK
from routesmith.integrations.anthropic import RouteSmithAnthropic
client = RouteSmithAnthropic.with_openrouter_models()

# LangChain
from routesmith.integrations.langchain import ChatRouteSmith
llm = ChatRouteSmith.with_openai_models()

# DSPy
from routesmith.integrations.dspy import RouteSmithLM
lm = RouteSmithLM()

# CrewAI
from routesmith.integrations.crewai import routesmith_crewai_chat_model
llm = routesmith_crewai_chat_model()

# AutoGen
from routesmith.integrations.autogen import routesmith_autogen_agents
assistant, user = routesmith_autogen_agents()
```

## Quick start

```bash
# Interactive setup: browse OpenRouter catalog, pick models
routesmith init

# Start the proxy
routesmith serve

# Check stats
routesmith stats

# View routing decisions
routesmith audit

# Manage per-role routing policies
routesmith roles list
routesmith roles set --role coder --model-pool gpt-4o-mini gpt-4o
routesmith roles unset --role coder
```

```python
# Python API
from routesmith import RouteSmith

rs = RouteSmith.with_free_models()
response = rs.completion(messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)

# Learn from user feedback
rs.record_outcome(response._routesmith_request_id, score=0.9)
```

## Examples

| File | Requires | Description |
|------|----------|-------------|
| `examples/quickstart_python.py` | — | Register models, completion, stats, feedback |
| `examples/quickstart_proxy.sh` | `routesmith[proxy]` | Proxy via CLI: init → serve → curl |
| `examples/multi_agent_roles.py` | — | Per-role routing (planner/coder/summarizer) |
| `examples/langgraph_agents.py` | `langchain_core` | 2-node LangGraph with per-role `ChatRouteSmith` |
| `examples/crewai_crew.py` | `crewai` | 2-agent CrewAI crew with shared RouteSmith |
| `examples/autogen_pair.py` | `autogen` | AutoGen agent pair via proxy |
| `examples/dspy_pipeline.py` | `dspy` | DSPy `Predict` with `RouteSmithLM` |
| `examples/openai_agents_sdk.py` | `openai` | OpenAI SDK pointed at proxy |
| `examples/pydantic_ai_agent.py` | `pydantic_ai` | Pydantic AI agent via proxy |
| `examples/llamaindex_engine.py` | `llama_index` | LlamaIndex `OpenAILike` via proxy |

## Documentation

- [CHANGELOG.md](CHANGELOG.md)
- [CONTRIBUTING.md](CONTRIBUTING.md)
- [Integration guides](docs/integrations/)
- [CLI reference](docs/cli.md)
- [Concepts](docs/concepts/)

## Installation

```bash
# Proxy + interactive setup (recommended)
pip install "routesmith-llm[proxy]"

# Core Python API only
pip install routesmith-llm

# With specific integrations
pip install "routesmith-llm[langchain]"
pip install "routesmith-llm[anthropic]"
pip install "routesmith-llm[cache]"
pip install "routesmith-llm[all]"
```

Requires Python 3.10+. Set `OPENROUTER_API_KEY` to use OpenRouter models.

## License

MIT — see [LICENSE](LICENSE)