Metadata-Version: 2.4
Name: fewertokens
Version: 0.3.2
Summary: FewerTokens Python SDK — control-plane routing with local LiteLLM dispatch
License-Expression: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.25.0
Requires-Dist: pydantic>=2.7.0
Requires-Dist: litellm<1.104,>=1.97.0
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
Requires-Dist: respx>=0.21.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Dynamic: license-file

# fewertokens

Python SDK for **FewerTokens** — a control-plane client for cross-family LLM routing.

The SDK asks FewerTokens which provider/model to use, then calls that model **locally** via [LiteLLM](https://github.com/BerriAI/litellm) with your provider keys. Usage is reported back asynchronously.

## Install

```bash
pip install fewertokens
```

## Quick start

```python
from fewertokens import FewerTokens

ft = FewerTokens(
    api_key="tr_live_...",                         # your FewerTokens key from the dashboard
    openai_api_key="sk-proj-...",
    openai_default_model="gpt-4o",                 # your current default — savings are measured against this
    anthropic_api_key="sk-ant-...",
    anthropic_default_model="claude-sonnet-4-6",   # priced default so routing can save money
    gemini_api_key="AIzaSy...",
    gemini_default_model="gemini-3.1-pro-preview", # priced default so routing can save money
    base_url="https://api.fewertokens.in",
)

response = ft.chat.completions.create(
    messages=[{"role": "user", "content": "What is Python?"}],
)

# The answer
print(response.content)

# Which model was actually used (may be cheaper than your default)
print(f"Routed to: {response.routing.provider}/{response.routing.model_used}")
print(f"Baseline was: {response.routing.baseline_model}")

# Token usage
print(f"Tokens — prompt: {response.usage.prompt_tokens}, completion: {response.usage.completion_tokens}")

# Estimated savings for this request (USD)
print(f"Estimated saving this call: ${response.routing.estimated_savings_usd:.6f}")
```

## Response object

`ft.chat.completions.create(...)` returns a `ChatResponse` — **not** the OpenAI SDK object. Key attributes:

| Attribute | Type | Description |
|---|---|---|
| `response.content` | `str` | The model's reply text |
| `response.model` | `str` | Exact model string returned by the provider |
| `response.finish_reason` | `str \| None` | e.g. `"stop"` |
| `response.usage.prompt_tokens` | `int` | Input tokens billed by the provider |
| `response.usage.completion_tokens` | `int` | Output tokens billed by the provider |
| `response.routing.provider` | `str` | Provider family used (e.g. `"gemini"`) |
| `response.routing.model_used` | `str` | Model dispatched (e.g. `"gemini-3.1-flash-lite"`) |
| `response.routing.baseline_model` | `str \| None` | Your configured default (the savings baseline) |
| `response.routing.estimated_savings_usd` | `float` | Estimated USD saved vs your baseline for this call, from the published catalog's USD input rates (0 for unlisted or unpriced models and fallbacks) |
| `response.routing.fallback_reason` | `str \| None` | `None`, `"route_unavailable"` or `"routed_model_error"` (see "How it works") |
| `response.raw` | `dict` | Full raw provider response |

---

## How it works

1. **Route handshake** — `POST /v1/route` with the routing prompt (your `system` messages + the last `user` message only; the rest of the conversation is never sent, see "What FewerTokens receives" below) + `provider_models` (each provider key's default model). The whole handshake (DNS, connect, send, receive) has one **800 ms** deadline.
2. **Local dispatch** — `litellm.completion` / `acompletion` with the routed provider key
3. **Telemetry** — background `POST /v1/telemetry` with `route_id` + token usage. The completion call does not wait, and a failed report does not raise. `close()`, the context manager (`with` and `async with`), and process exit wait up to 2 seconds for a report still in flight. That wait does not raise.

If the control plane is unreachable, slow, or rejects the request (including an invalid FewerTokens API key), the SDK calls your first configured `*_default_model` locally and skips telemetry, so there is no fee. FewerTokens must not be the reason a prompt never reaches the LLM provider. `response.routing.fallback_reason` is `"route_unavailable"` in that case.

When an invoice hasn't been paid, the response comes back as HTTP 402 with error code `routing_paused`. The SDK logs one warning per client on the `fewertokens.chat` logger: "FewerTokens routing is paused because an invoice hasn't been paid. Pay it using the link in your invoice email to turn routing back on." That call goes to your default model and sends no usage telemetry, so no fee is charged.

If the **routed** model fails on the provider side (5xx, overloaded, 429 rate limit, timeout, connection error), the SDK retries **once** on your first configured `*_default_model`. Telemetry marks the call as a fallback, so no fee is charged, and `response.routing.fallback_reason` is `"routed_model_error"`. Errors caused by the request or your account (400/422, 401/403, 404, content policy, context length) are raised without a retry. For streams, the retry only happens if the routed stream fails before its first chunk; once tokens have reached your code, a mid-stream error is raised.

## Privacy & Security

🔒 **Privacy & Security First:**

- **Zero Key Exposure:** Your provider keys stay strictly in your client runtime's RAM memory and are never sent to FewerTokens servers.
- **Direct Local Execution:** Prompt content and response streams flow directly between your machine and LLM provider endpoints (OpenAI, Anthropic, Gemini, etc.). FewerTokens only receives the routing prompt (your `system` messages + the last `user` message, see "What FewerTokens receives" below) for the routing decision (~20ms) and never sees or logs completion payloads.

## Streaming

```python
for chunk in ft.chat.completions.create(
    messages=[{"role": "user", "content": "Explain Docker"}],
    stream=True,
):
    print(chunk.content, end="", flush=True)
```

## Async

```python
response = await ft.chat.completions.acreate(
    messages=[{"role": "user", "content": "Hello!"}],
)
```

## Notes

- At least one of `openai_api_key`, `anthropic_api_key`, or `gemini_api_key` is required, and each key must have a matching `*_default_model`.
- String shorthand: `prompt="Hello"` is accepted as an alternative to `messages=`.
- `model=` on `.create()` is **ignored** — the control plane decides the model.
- Your `*_default_model` is the **savings baseline**: set it to the model you are currently calling. If you set it to an already-cheap model, there is nothing to save and the fee is `$0`.
- FewerTokens **never sees your provider API keys or completion payloads**. Keys stay in your process RAM.

### What FewerTokens receives

Since 0.3.0, `POST /v1/route` carries exactly three fields: `prompt`, `available_providers` and `provider_models`.

- `prompt` is built locally from your messages:
  - all of your `system` messages, joined;
  - plus the **last** `user` message only;
  - only text parts are used; images and other non-text parts are dropped;
  - if there is no `user` message at all, the last message of any role is used instead.
- Earlier user turns, assistant replies and tool results are never sent. The full conversation goes only to your LLM provider, through LiteLLM on your machine. (0.2.x sent the whole `messages` list as well.)
- The server's classifier reads at most the first 256 tokens of `prompt`.
- The SDK sends at most the first 8,192 characters of `prompt`. A `prompt` longer than 32,000 characters is rejected by the API. The SDK then calls your default model directly: there is no routing and no fee.
- `available_providers` and `provider_models` carry your provider names and each provider's `*_default_model` id.
- `POST /v1/telemetry` carries the `route_id` and token counts. For a retried call it also carries `fallback`, `fallback_provider` and `fallback_model`. It never carries prompt or completion text.

## Default model has no verified price

You may see this warning once per process, on the `fewertokens.pricing` logger:

```
Your default model has no verified price, so routing and fees are off. Pick a priced default to turn them on. See https://pypi.org/project/fewertokens/#user-content-default-model-has-no-verified-price
```

**What it means.** Your default model is the `*_default_model` of the first provider you configure (OpenAI, then Anthropic, then Gemini, then `extra_providers` in order). FewerTokens has no verified price for it, so it can't measure savings against it. Your calls still work: every call goes straight to your default model, with no routing and no fee. The SDK never switches your default model for you.

The SDK logs the warning when you create the client and your default model isn't in its price table, or when `POST /v1/route` reports `routing_disabled_reason: "default_model_unpriced"`.

**How to fix it.** Your default model is set in your SDK config, not on the dashboard. Pass a priced model as the `*_default_model` of your first provider:

```python
from fewertokens import FewerTokens

ft = FewerTokens(
    api_key="tr_live_...",
    anthropic_api_key="sk-ant-...",
    anthropic_default_model="claude-sonnet-4-6",  # was "claude-3-5-sonnet", which has no verified price
)
```

For providers in `extra_providers`, set `"default_model"` in that provider's entry.

**How to check whether a model is priced.** The SDK's price table holds the models in the published FewerTokens catalog (1.2.1). Priced defaults include `gpt-4o`, `gpt-4.1` and `gpt-5.5` (OpenAI), `claude-sonnet-4-6` and `claude-opus-4-7` (Anthropic), and `gemini-3.1-pro-preview` (Gemini). Models the catalog dropped, such as `claude-3-5-sonnet`, `o1-mini`, `gpt-5.6`, `gemini-2.0-flash`, `grok-4`, and (in 1.2.1) `gemini-2.5-pro`, `gemini-2.5-flash`, `o1` and `o4-mini`, have no verified price, and neither do fine-tunes or other models outside the catalog.

To check your own config, create the client with logging on. If the warning doesn't appear when you create the client or on your first call, your default model is priced:

```python
import logging

logging.basicConfig(level=logging.WARNING)
ft = FewerTokens(api_key="tr_live_...", openai_api_key="sk-proj-...", openai_default_model="gpt-4o")
```

**How to silence it or route it.** The SDK adds no log handlers. The warning goes through Python's standard `logging`, so your logging config decides where it ends up. Without any config, Python prints it to stderr.

```python
import logging

# Silence it:
logging.getLogger("fewertokens.pricing").setLevel(logging.ERROR)

# Or send it to your own handler instead of the root logger:
pricing_log = logging.getLogger("fewertokens.pricing")
pricing_log.addHandler(logging.FileHandler("fewertokens.log"))
pricing_log.propagate = False
```

Silencing the warning doesn't turn routing back on. Only a priced default model does that.
