Metadata-Version: 2.4
Name: peerai
Version: 0.2.0
Summary: PeerAI — P2P GPU sharing for AI inference. Contribute your GPU, earn revenue. Use spot inference at a fraction of cloud cost.
Author-email: WorkWeek <askqai@mitns.com>
License: MIT
Project-URL: Homepage, https://askvai.com/peerai
Project-URL: Repository, https://github.com/rchow93/peerAI
Project-URL: Documentation, https://askvai.com/peerai/docs
Keywords: gpu,inference,llm,p2p,ai,marketplace
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn[standard]>=0.27
Requires-Dist: websockets>=12.0
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.0
Requires-Dist: typer[all]>=0.9
Requires-Dist: sse-starlette>=2.0
Requires-Dist: aiosqlite>=0.20
Requires-Dist: asyncpg>=0.29
Requires-Dist: redis>=5.0
Requires-Dist: pydantic-settings>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: httpx; extra == "dev"
Requires-Dist: respx>=0.22; extra == "dev"

# PeerAI

**Peer-to-peer GPU sharing for AI inference — OpenAI-compatible, zero friction.**

PeerAI turns idle GPUs into a shared AI inference network. Contributors earn credits by sharing compute; consumers get cheap inference through a standard OpenAI-compatible API. No code changes required — just point your existing OpenAI SDK at PeerAI.

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="your-key")

response = client.chat.completions.create(
    model="llama3:8b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

# Streaming works too
for chunk in client.chat.completions.create(
    model="llama3:8b",
    messages=[{"role": "user", "content": "Tell me a joke"}],
    stream=True,
):
    print(chunk.choices[0].delta.content or "", end="")
```

## How It Works

```
Your App (OpenAI SDK / curl / any HTTP client)
    → PeerAI Gateway (OpenAI-compatible API)
        → Smart Router (model, latency, load, reputation, price-aware)
            → Peer Node A (RTX 4090, Ollama)     ← direct P2P
            → Peer Node B (M3 Max, LM Studio)    ← WebSocket relay
            → Peer Node C (A100, vLLM)            ← gateway proxy
```

**Contribute** idle GPU time → earn credits.
**Consume** inference → spend credits or pay market price.

## Quick Start

### 1. Install

```bash
pip install -e ".[dev]"
```

### 2. Start the Gateway

```bash
peerai gateway --port 8000
```

The gateway starts with an in-memory SQLite backend by default. Visit `http://localhost:8000/dashboard` for the web UI.

### 3. Start a Node (contributor)

In a separate terminal, with Ollama running locally:

```bash
peerai serve --backend-url http://localhost:11434/v1 --gateway-url ws://localhost:8000/ws/node
```

The node agent auto-discovers your models and registers with the gateway.

### 4. Send Requests

```bash
# Chat completion
peerai chat llama3:8b "What is the meaning of life?"

# Streaming
peerai chat llama3:8b "Write a haiku" --stream

# Or use curl
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3:8b", "messages": [{"role": "user", "content": "Hello"}]}'

# List models
peerai models list
```

## Features

| Feature | Description |
|---------|-------------|
| **OpenAI-compatible API** | Drop-in replacement — chat completions, completions, embeddings, model listing |
| **Multi-node routing** | 7 strategies: round_robin, least_connections, least_latency, best_reputation, random, cheapest, best_value |
| **Auto failover** | Retries on up to 2 other nodes if one fails, with mid-stream resume |
| **Credit metering** | Token-based billing, contributor earnings (70% split), SQLite or PostgreSQL ledger |
| **Authentication** | API keys (SHA-256 hashed), node tokens, admin roles |
| **Rate limiting** | Global RPM limits + per-user per-model RPM/TPM/TPH/TPD budgets |
| **Node reputation** | Composite scoring: success rate, latency, uptime, disconnect history |
| **GPU marketplace** | Dynamic pricing, capacity listings, bidding, price-aware routing |
| **Batch inference** | Priority queuing, semaphore throttling, webhooks, pause/resume/retry |
| **Model management** | Auto-pull on demand, remote delete, pull progress tracking (Ollama) |
| **Direct P2P** | HMAC-signed tokens, connectivity probing, three modes: direct/relay/gateway |
| **Multi-gateway federation** | Redis pub/sub sync across gateway instances |
| **Job checkpointing** | Mid-stream failover with resume, preemption protocol, spot instance model |
| **Model catalog** | Rich metadata, load state tracking, per-model analytics, hot/idle discovery |
| **Security hardening** | Audit logging, login throttling, key rotation/expiration, security headers, CORS |
| **Observability** | Metrics endpoint, latency percentiles, per-node/model analytics |
| **Web dashboard** | Tabbed UI: Overview, Catalog, Marketplace, Nodes, Metrics — auto-refreshing |
| **Python SDK** | Sync and async clients mirroring OpenAI SDK structure |
| **CLI** | 28 subcommands across 6 groups (models, batch, nodes, credits, market, chat) |

## Architecture

```
┌─────────────────────────────────────────────────┐
│                  PeerAI Gateway                  │
│                                                  │
│  ┌──────────┐ ┌────────┐ ┌──────────────────┐  │
│  │  Routes   │ │ Router │ │   Marketplace    │  │
│  │(OpenAI)  │ │(7 strats)│ │(pricing, bids)  │  │
│  └─────┬────┘ └────┬───┘ └────────┬─────────┘  │
│        │           │              │              │
│  ┌─────┴───────────┴──────────────┴──────────┐  │
│  │              Request Broker                │  │
│  │      (asyncio.Queue / Redis pub/sub)       │  │
│  └────────────────────┬───────────────────────┘  │
│                       │                          │
│  ┌────────────────────┴───────────────────────┐  │
│  │            Node Registry                   │  │
│  │   (reputation, connectivity, federation)   │  │
│  └────────────────────┬───────────────────────┘  │
└───────────────────────┼──────────────────────────┘
                        │ WebSocket (persistent)
        ┌───────────────┼───────────────┐
        ▼               ▼               ▼
   ┌─────────┐    ┌─────────┐    ┌─────────┐
   │  Node A  │    │  Node B  │    │  Node C  │
   │ (Ollama) │    │(LM Studio)│   │ (vLLM)  │
   └─────────┘    └─────────┘    └─────────┘
```

See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed design documentation.

## Documentation

| Document | Description |
|----------|-------------|
| [API Reference](docs/API.md) | Complete HTTP API documentation (42 endpoints) |
| [Python SDK](docs/SDK.md) | PeerAI Python SDK — sync and async clients |
| [CLI Reference](docs/CLI.md) | All CLI commands and options |
| [Deployment Guide](docs/DEPLOYMENT.md) | Local dev, Docker, production setup |
| [Architecture](docs/ARCHITECTURE.md) | System design, data flow, protocols |
| [Project Plan](docs/PROJECT_PLAN.md) | Roadmap and competitive analysis |

## Configuration

All settings are configurable via environment variables (`PEERAI_` prefix) or a `.env` file:

| Variable | Default | Description |
|----------|---------|-------------|
| `PEERAI_DATABASE_URL` | `sqlite:///peerai.db` | Database URL (SQLite or PostgreSQL) |
| `PEERAI_REDIS_URL` | `""` | Redis URL (enables Redis broker + federation) |
| `PEERAI_PORT` | `8000` | Gateway port |
| `PEERAI_ROUTING_STRATEGY` | `least_connections` | Routing strategy |
| `PEERAI_ENABLE_CREDITS` | `true` | Enable credit metering |
| `PEERAI_ENABLE_AUTH` | `true` | Enable API key authentication |
| `PEERAI_ENABLE_RATE_LIMIT` | `true` | Enable global rate limiting |
| `PEERAI_ENABLE_BATCH` | `true` | Enable batch inference |
| `PEERAI_ENABLE_AUTO_PULL` | `false` | Auto-pull models when no node has them |
| `PEERAI_ENABLE_DIRECT_CONNECT` | `false` | Enable direct P2P connections |
| `PEERAI_ENABLE_FEDERATION` | `true` | Enable multi-gateway federation |
| `PEERAI_CORS_ORIGINS` | `""` | Comma-separated allowed CORS origins |
| `PEERAI_API_KEY_MAX_AGE_DAYS` | `0` | API key expiration (0 = never) |
| `PEERAI_LOGIN_THROTTLE_ENABLED` | `true` | IP-based login throttling |

See [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) for the full configuration reference.

## Development

```bash
# Install with dev deps
pip install -e ".[dev]"

# Run tests
pytest tests/ -v

# Start gateway (dev mode, no auth)
peerai gateway --no-auth --no-credits

# Start node
peerai serve --backend-url http://localhost:11434/v1
```

## Project Structure

```
src/peerai/
├── cli/             # CLI (Typer) — 28 subcommands
├── config.py        # Settings (pydantic-settings, PEERAI_* env vars)
├── gateway/         # FastAPI gateway
│   ├── app.py           # App factory
│   ├── routes.py        # OpenAI-compatible HTTP API
│   ├── ws_handler.py    # WebSocket node handler
│   ├── registry.py      # Node registry + reputation
│   ├── broker.py        # Request broker (asyncio.Queue)
│   ├── redis_broker.py  # Request broker (Redis pub/sub)
│   ├── federation.py    # Multi-gateway sync
│   ├── marketplace.py   # GPU marketplace engine
│   ├── model_catalog.py # Model metadata + analytics
│   ├── model_rate_limit.py  # Per-model rate limiting
│   ├── batch_store.py   # Batch job store
│   ├── batch_scheduler.py   # Background batch processor
│   ├── dispatch.py      # Shared inference dispatcher
│   ├── checkpoint.py    # Job checkpointing for failover
│   ├── pull_tracker.py  # Model pull job tracker
│   ├── direct_auth.py   # HMAC token signing for P2P
│   ├── connectivity.py  # Node reachability probing
│   ├── auth.py          # API key auth (SQLite)
│   ├── pg_auth.py       # API key auth (PostgreSQL)
│   ├── audit.py         # Security audit logging + login throttler
│   ├── security.py      # Security headers middleware + key expiration
│   └── static/          # Web dashboard
├── node/            # Node agent
│   ├── agent.py         # Single-gateway agent
│   ├── federated_agent.py   # Multi-gateway agent
│   ├── model_ops.py     # Model pull/delete mixin
│   └── direct_server.py # Direct P2P HTTP server
├── relay/           # Relay service for NAT traversal
│   ├── server.py        # Relay HTTP/WS server
│   └── tunnel.py        # Node-side tunnel client
├── router/          # Request routing
│   └── selector.py      # 7 routing strategies + reputation
├── models/          # Pydantic models
│   ├── api.py           # OpenAI-compatible schemas
│   ├── node.py          # Node models
│   ├── protocol.py      # WebSocket message protocol
│   ├── marketplace.py   # Marketplace models
│   ├── batch.py         # Batch job models
│   ├── pull.py          # Model pull/delete models
│   └── direct_connect.py    # P2P connection models
├── sdk/             # Python SDK
│   ├── __init__.py      # PeerAI, AsyncPeerAI exports
│   └── client.py        # Sync + async clients
└── db/              # Database layer
    ├── ledger.py        # Credit ledger (SQLite)
    └── pg_ledger.py     # Credit ledger (PostgreSQL)
```

## License

TBD
