Metadata-Version: 2.4
Name: fastapi-llm-gateway
Version: 0.1.1
Summary: A composable FastAPI LLM gateway with unified multi-provider support
Author-email: LLM Gateway Team <twork097@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/young-tim/fastapi-llm-gateway
Project-URL: Documentation, https://github.com/young-tim/fastapi-llm-gateway#readme
Project-URL: Repository, https://github.com/young-tim/fastapi-llm-gateway
Project-URL: Issues, https://github.com/young-tim/fastapi-llm-gateway/issues
Keywords: fastapi,llm,gateway,openai,anthropic,ai,volcengine
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: litellm>=1.0.0
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn[standard]>=0.23.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: aiohttp>=3.8.0
Requires-Dist: httpx>=0.24.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: httpx>=0.24.0; extra == "dev"
Dynamic: license-file

# fastapi-llm-gateway

A composable FastAPI LLM gateway with unified multi-provider support.

**Features:**
- 🚀 **Multiple Usage Patterns**: Use as SDK, integrate into FastAPI apps, or run as standalone service
- 🔌 **Unified API**: Single interface for OpenAI, Anthropic, Cohere, VolcEngine, and 100+ providers
- 🔄 **Smart Routing**: Built-in load balancing, failover, and model aliasing
- 💰 **Cost Tracking**: Automatic cost calculation per request
- 🌍 **Zero Config**: Works out-of-the-box with environment variables
- 📦 **PyPI Ready**: Install with a single pip command

## Installation

```bash
pip install fastapi-llm-gateway
```

## Quick Start

### Pattern 1: SDK - Programmatic Access

```python
from fastapi_llm_gateway.sdk import GatewayClient

# Option A: Pass API keys directly
client = GatewayClient(api_keys={"openai": "sk-..."})

# Option B: Use environment variables (recommended)
export OPENAI_API_KEY="sk-..."
client = GatewayClient()  # Automatically reads from env vars

# Make requests
response = client.chat(
    messages=[{"role": "user", "content": "Hello!"}],
    provider="openai",
    model="gpt-4o"
)
print(response.content)
print(f"Cost: ${response.cost}")
```

### Pattern 2: FastAPI Integration (Recommended)

```python
from fastapi import FastAPI
from fastapi_llm_gateway.fastapi import GatewayRouter

app = FastAPI()
app.include_router(GatewayRouter(prefix="/api/llm"))

# Your app now has LLM gateway endpoints:
# POST /api/llm/v1/chat/completions
# GET  /api/llm/models
# GET  /api/llm/health
```

### Pattern 3: Standalone Service

```bash
# Set your API keys as environment variables
export OPENAI_API_KEY="sk-..."

# Start the server
fastapi-llm-gateway-serve

# Or with custom options
fastapi-llm-gateway-serve --host 0.0.0.0 --port 8080 --workers 4
```

## Configuration

### Environment Variables

| Variable | Description | Required |
|----------|-------------|----------|
| `OPENAI_API_KEY` | OpenAI API key | No |
| `ANTHROPIC_API_KEY` | Anthropic API key | No |
| `COHERE_API_KEY` | Cohere API key | No |
| `VOLCENGINE_API_KEY` | VolcEngine API key | No |
| `LLM_GATEWAY_CONFIG` | Path to custom YAML config | No |
| `LLM_GATEWAY_PORT` | Server port (default: 8001) | No |
| `LLM_GATEWAY_HOST` | Server host (default: 0.0.0.0) | No |

## Built-in Models

The package includes built-in definitions for common models:

- **OpenAI**: gpt-4o, gpt-4o-mini, gpt-3.5-turbo
- **Anthropic**: claude-3-5-sonnet, claude-3-opus
- **Cohere**: command-r, command-r-plus
- **VolcEngine**: doubao-pro-32k (requires endpoint_id config)

## License

MIT License - see [LICENSE](LICENSE) for details.
