Metadata-Version: 2.5
Name: powergrid-ai
Version: 1.3.3
Summary: Local-first AI routing runtime — one AI, many engines
Project-URL: Homepage, https://github.com/micymike/powergrid
Project-URL: Repository, https://github.com/micymike/powergrid
Project-URL: Issues, https://github.com/micymike/powergrid/issues
Project-URL: Changelog, https://github.com/micymike/powergrid/blob/main/CHANGELOG.md
Author-email: Michael Moses <micymike@example.com>
License-Expression: MIT
License-File: LICENSE
Keywords: ai,failover,llm,local-first,multi-provider,openai,routing
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: aiosqlite>=0.20.0
Requires-Dist: click>=8.1.0
Requires-Dist: fastapi>=0.115.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: keyring>=25.0.0
Requires-Dist: pydantic-settings>=2.5.0
Requires-Dist: pydantic>=2.9.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: rich>=13.9.0
Requires-Dist: sqlalchemy>=2.0.0
Requires-Dist: sse-starlette>=2.1.0
Requires-Dist: typer>=0.12.0
Requires-Dist: uvicorn[standard]>=0.30.0
Provides-Extra: dev
Requires-Dist: mypy>=1.11.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
Requires-Dist: pytest>=8.3.0; extra == 'dev'
Requires-Dist: ruff>=0.6.0; extra == 'dev'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://img.shields.io/badge/Local-First-blueviolet?style=for-the-badge" alt="Local-First">
  <img src="https://img.shields.io/badge/OpenAI-Compatible-black?style=for-the-badge" alt="OpenAI-Compatible">
  <img src="https://img.shields.io/badge/Routing-Intelligent-brightgreen?style=for-the-badge" alt="Routing">
  <img src="https://img.shields.io/badge/License-MIT-yellow?style=for-the-badge" alt="License">
</p>

<h1 align="center">PowerGrid</h1>

<p align="center">
  <strong>One AI. Many engines. Zero vendor lock-in.</strong><br>
  Local-first AI routing runtime that turns multiple AI providers into a single OpenAI-compatible endpoint.
</p>

<p align="center">
  <a href="https://pypi.org/project/powergrid-ai"><img src="https://img.shields.io/pypi/v/powergrid?style=flat-square" alt="PyPI"></a>
  <a href="https://www.npmjs.com/package/powergrid-ai"><img src="https://img.shields.io/npm/v/powergrid-ai?style=flat-square" alt="npm"></a>
  <a href="https://github.com/micymike/powergrid/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/micymike/powergrid/ci.yml?style=flat-square" alt="CI"></a>
  <a href="https://github.com/micymike/powergrid"><img src="https://img.shields.io/github/stars/micymike/powergrid?style=flat-square" alt="Stars"></a>
</p>

---

## What is PowerGrid?

PowerGrid runs on your machine and makes multiple AI providers appear as **one endpoint**. Your apps talk to `localhost:8787`, and PowerGrid handles routing, failover, caching, and rate limit prediction — automatically.

```
Your App ──→ localhost:8787/v1 ──→ PowerGrid ──→ Gemini
                                          ├─→ Groq
                                          ├─→ Cerebras
                                          ├─→ OpenRouter
                                          └─→ Ollama (local)
```

**No vendor lock-in. No cloud dependency. Your keys never leave your machine.**

---

## Quick Start

### Install

```bash
# Python (recommended)
pip install powergrid-ai

# Node.js (wraps Python)
npm install -g powergrid-ai
```

> **npm users:** If setup doesn't run automatically, use `npm install -g powergrid-ai --allow-scripts=powergrid-ai`

### Setup

```bash
powergrid setup
```

Walk through providers, enter your API keys, and you're done:

```
──────── Core (free, no card, recommended) ────────

  1. Google Gemini — Strong models + generous free tier
     Add Google Gemini? [y/n]: y
     API key: ••••••••••••••••
     ✓ Google Gemini connected!

  2. Groq — Extremely fast inference, great for coding
     Add Groq? [y/n]: y
     API key: ••••••••••••••••
     ✓ Groq connected!

  3. Cerebras — Fast inference, large context
     Add Cerebras? [y/n]: n
     skipped
```

### Start

```bash
powergrid start
```

### Use it

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8787/v1", api_key="powergrid")

response = client.chat.completions.create(
    model="powergrid-auto",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

---

## Supported Providers

### Core (free, no credit card required)

| Provider | Free Tier | Speed | Best For |
|----------|-----------|-------|----------|
| Google Gemini | Generous | Fast | General, reasoning, large context |
| Groq | 30 RPM | Blazing | Fast response, coding, tool calling |
| Cerebras | Generous | Very Fast | Large context, reasoning |
| OpenRouter | Rotating free models | Varies | Model variety, fallback |
| NVIDIA NIM | Generous | Fast | Large model catalog |
| Ollama | Unlimited (local) | Local | Privacy, offline, zero cost |

### Extended

| Provider | Notes |
|----------|-------|
| Mistral AI | Strong European models, Codestral |
| Hugging Face | Open-source ecosystem |
| SambaNova | High-performance inference |
| Together AI | Open-source model hosting |
| Cloudflare Workers AI | Edge inference |
| Cohere | RAG + embeddings specialist |
| GitHub Models | Great selection via GitHub |

### Community / Experimental

| Provider | Notes |
|----------|-------|
| SiliconFlow | Chinese/open-source models |
| Chutes AI | Decentralized inference |
| **Any OpenAI-compatible API** | Just provide a base URL |

---

## How It Works

### Intelligent Routing (default)

Every request is analyzed and routed to the best provider based on:

```
Request → Task Classification → Provider Scoring → Best Match
                │                      │
                ├─ coding               ├─ health (uptime %)
                ├─ reasoning            ├─ quota (RPM remaining)
                ├─ creative             ├─ capability match
                ├─ fast_response        ├─ priority weight
                └─ long_context         └─ latency score
```

### Automatic Failover

```
Request → Provider A
            │
            ├─ 200 OK → return response
            │
            ├─ 429 Rate Limited → cooldown → retry Provider B
            │
            ├─ 5xx Error → backoff → retry Provider C
            │
            └─ Timeout → retry Provider B
```

### Predictive Rate Limit Protection

PowerGrid tracks usage patterns and shifts traffic **before** you hit rate limits:

```
Groq RPM = 30
18:01 → 21 requests
18:02 → 26 requests  ← PowerGrid sees trend
18:03 → traffic shifted to Cerebras before 429
```

### Response Caching

Exact-match requests are cached with configurable TTL. Same prompt = instant response, zero API cost.

---

## Architecture

```
┌─────────────────────────────────────────────────┐
│                  APPLICATION                      │
└──────────────────────┬──────────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────────┐
│                  POWERGRID                        │
│                                                   │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────┐ │
│  │   Request    │  │   Routing    │  │  Cache  │ │
│  │  Analyzer    │──│    Engine    │──│  Layer  │ │
│  └─────────────┘  └──────┬───────┘  └─────────┘ │
│                          │                        │
│  ┌─────────────┐  ┌──────┴───────┐  ┌─────────┐ │
│  │   Health     │  │   Quota     │  │  Retry  │ │
│  │  Tracker     │──│   Engine    │──│ Handler │ │
│  └─────────────┘  └──────────────┘  └─────────┘ │
│                                                   │
└──────────────────────┬──────────────────────────┘
                       │
         ┌─────────────┼─────────────┐
         │             │             │
         ▼             ▼             ▼
     ┌────────┐   ┌────────┐   ┌────────┐
     │ Gemini │   │  Groq  │   │Cerebras│
     └────────┘   └────────┘   └────────┘
```

### Core Components

| Component | Purpose |
|-----------|---------|
| **Provider Adapters** | Normalize provider APIs into a common format |
| **Routing Engine** | Score and select the best provider per request |
| **Health Tracker** | Monitor availability, apply cooldowns on failures |
| **Quota Engine** | Track RPM/TPM usage, predict rate limit exhaustion |
| **Retry Handler** | Manage retries, failover, and exponential backoff |
| **Response Cache** | TTL-based exact-match caching |
| **Request Analyzer** | Classify task types with heuristics |

---

## Agent Integrations

PowerGrid works with any tool that speaks the OpenAI API format.

### OpenCode

```json
{
  "provider": {
    "name": "powergrid",
    "model": "powergrid-auto",
    "api_key": "powergrid",
    "base_url": "http://localhost:8787/v1"
  }
}
```

### Claude Code / Codex / Aider

```bash
export OPENAI_BASE_URL=http://localhost:8787/v1
export OPENAI_API_KEY=powergrid
```

### Cline (VS Code)

```json
{
  "cline.apiProvider": "openai-compatible",
  "cline.openaiCompatibleBaseUrl": "http://localhost:8787/v1",
  "cline.openaiCompatibleApiKey": "powergrid",
  "cline.openaiCompatibleModelId": "powergrid-auto"
}
```

### Python

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8787/v1", api_key="powergrid")
response = client.chat.completions.create(
    model="powergrid-auto",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

### Node.js

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8787/v1",
  apiKey: "powergrid",
});

const response = await client.chat.completions.create({
  model: "powergrid-auto",
  messages: [{ role: "user", content: "Hello!" }],
});
```

### curl

```bash
curl http://localhost:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "powergrid-auto",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

---

## CLI Reference

| Command | Description |
|---------|-------------|
| `powergrid setup` | Interactive setup wizard |
| `powergrid init` | Initialize configuration |
| `powergrid start` | Start the runtime |
| `powergrid start --background` | Start as background process |
| `powergrid stop` | Stop background process |
| `powergrid status` | Show runtime status |
| `powergrid providers` | List configured providers |
| `powergrid provider` | Add provider interactively |
| `powergrid provider-remove NAME` | Remove a provider |
| `powergrid models` | List available models |
| `powergrid logs` | Show recent routing decisions |
| `powergrid test` | Test provider connectivity |
| `powergrid config` | Show config (redacted) |
| `powergrid cache-clear` | Clear response cache |
| `powergrid --version` | Show version |

---

## API Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/v1/models` | GET | List available models |
| `/v1/chat/completions` | POST | Chat completions (OpenAI-compatible) |
| `/health` | GET | Health check with provider status |
| `/status` | GET | PowerGrid status and stats |
| `/providers` | GET | Provider details (keys redacted) |
| `/routing/logs` | GET | Recent routing decisions |
| `/routing/stats` | GET | Aggregate routing statistics |

---

## Configuration

### YAML Config

Located at `~/.config/powergrid/config.yaml`:

```yaml
providers:
  - name: groq
    type: openai_compatible
    base_url: https://api.groq.com/openai/v1
    api_key_env: GROQ_API_KEY
    models:
      - llama-3.3-70b-versatile
    priority: 90
    rpm_limit: 30
    timeout: 30
    capabilities:
      - general
      - coding
      - fast_response

  - name: gemini
    type: gemini
    api_key_env: GEMINI_API_KEY
    models:
      - gemini-2.0-flash
    priority: 85
    rpm_limit: 15
    timeout: 60
    capabilities:
      - general
      - reasoning
      - coding
```

### Custom Provider (any OpenAI-compatible API)

```yaml
providers:
  - name: my-ai
    type: openai_compatible
    base_url: https://api.mycompany.com/v1
    models:
      - my-model-v1
    priority: 50
    capabilities:
      - general
```

### Routing Strategies

Set in config or environment:

```yaml
routing:
  strategy: intelligent  # intelligent | round_robin | priority
```

---

## Security & Privacy

### Local-First by Default

- Binds to `127.0.0.1` — not accessible from the network
- Warns explicitly if you bind to `0.0.0.0`

### API Key Protection

- Keys stored in OS credential store (Windows Credential Manager, macOS Keychain, Linux SecretService)
- Never written to config files, logs, or API responses
- Optional `auth_token` to protect your endpoint

### No Telemetry

- Zero data sent anywhere
- No prompts, responses, or usage metrics collected
- All state in `~/.config/powergrid/`

---

## Development

```bash
git clone https://github.com/micymike/powergrid.git
cd powergrid
pip install -e ".[dev]"
pytest
```

### Running Tests

```bash
# Unit tests (no API keys needed)
pytest

# With mock provider
powergrid start  # uses mock provider if configured

# Real E2E tests (needs running server + API keys)
pytest tests/test_real_e2e.py -v
```

### Custom Provider Adapter

```python
from powergrid.providers.base import ProviderAdapter, ProviderStatus

class MyProvider(ProviderAdapter):
    name = "my_provider"
    display_name = "My Provider"

    async def chat_completion(self, request, model_id):
        # Call your provider API
        ...

    async def chat_completion_stream(self, request, model_id):
        # Stream SSE chunks
        ...

    async def health_check(self):
        return ProviderStatus.HEALTHY
```

Register it:

```python
from powergrid.providers.registry import registry
registry.register(MyProvider(name="my_provider", api_key="..."))
```

---

## Roadmap

- [ ] Provider packs — `powergrid provider install free-tier-pack`
- [ ] Dashboard UI — web interface for monitoring
- [ ] Plugin system — custom routing strategies
- [ ] Model loading — load local models via PowerGrid
- [ ] Team mode — shared configs with key rotation

---

## License

MIT

---

<p align="center">
  Built with ❤️ by <a href="https://github.com/micymike">Michael Moses</a>
</p>
