Metadata-Version: 2.4
Name: brevitas-systems
Version: 0.9.12
Summary: Lossless token savings + per-pipeline/agent tracking between your agents and the model.
License: Proprietary
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: cryptography>=42.0.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: requests>=2.31.0
Requires-Dist: tiktoken>=0.7.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: fastembed>=0.3.0
Requires-Dist: fastapi>=0.111.0
Requires-Dist: uvicorn[standard]>=0.29.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: supabase>=2.0.0
Requires-Dist: click>=8.1.0
Requires-Dist: rich>=13.0.0
Requires-Dist: google-cloud-kms==3.15.0
Requires-Dist: google-crc32c==1.8.0
Requires-Dist: agentmap-scan<0.2,>=0.1.1
Provides-Extra: retrieval
Provides-Extra: semanticcache
Requires-Dist: sentence-transformers>=2.2.0; extra == "semanticcache"
Provides-Extra: promptopt
Requires-Dist: llmlingua>=0.2.1; extra == "promptopt"
Provides-Extra: pdf
Requires-Dist: pymupdf>=1.23.0; extra == "pdf"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.28.0; extra == "anthropic"
Provides-Extra: openai
Requires-Dist: openai>=1.30.0; extra == "openai"
Provides-Extra: all
Requires-Dist: sentence-transformers>=2.2.0; extra == "all"
Requires-Dist: llmlingua>=0.2.1; extra == "all"
Requires-Dist: anthropic>=0.28.0; extra == "all"
Requires-Dist: openai>=1.30.0; extra == "all"
Requires-Dist: pymupdf>=1.23.0; extra == "all"

# Brevitas — provider-cache optimization and metering for LLM agents

Brevitas is middleware that sits between your code and the model providers
(Anthropic, OpenAI, DeepSeek, Groq). Its default request path preserves prompt
content while measuring provider-native cache reads and writes. Optional retrieval,
compression, message reordering, and fuzzy response reuse can reduce provider work,
but can affect behavior and are disabled until explicitly enabled.

- **Content-preserving default.** Requests pass through unchanged except for explicitly
  enabled provider cache metadata. Provider caching can lower cost without lowering the
  provider's token count.
- **Quality-affecting levers fail closed.** Retrieval, LLMLingua, reordering, and fuzzy
  semantic response reuse require explicit operator opt-in and an untripped tenant gate.
- **Mechanism-separated evidence.** Reports distinguish provider input tokens avoided,
  native-cache discount, model calls avoided, transport bytes avoided, and measured
  Brevitas lift from an isolated control arm.
- **Two ways in.** The **hosted gateway** is a base-URL change and is the path that
  produces metered, billable savings. The **local proxy** is a zero-code install that
  keeps every byte on your machine — and, by design, cannot be billed on
  percentage-of-savings.

Site: https://brevitassystems.com

## Install

```bash
pip install brevitas-systems            # core
pip install "brevitas-systems[all]"     # + retrieval embeddings, llmlingua, provider SDKs
```

## Quick start — hosted gateway (recommended)

One command. It opens your browser, you approve as a workspace owner or admin, and
it hands back an organization service key scoped to your workspace.

```bash
brevitas connect
```

Then three lines in your app — no install, no background service, no code changes
beyond the client constructor:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.brevitassystems.com/v1",
    api_key=os.environ["OPENAI_API_KEY"],
    default_headers={
        # BOTH are required. X-Brevitas-Key is the only header the gateway
        # authenticates on (api/server.py:1731); `api_key=` becomes an
        # Authorization bearer, which nothing maps to it, so leaving this out
        # is `401 Missing X-Brevitas-Key header` on the first call.
        "X-Brevitas-Key": os.environ["BREVITAS_API_KEY"],
        "X-Brevitas-Customer-ID": "acme",
    },
)
```

Claude direct through Anthropic — same two headers, different SDK:

```python
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.brevitassystems.com/v1",
    api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={
        "X-Brevitas-Key": os.environ["BREVITAS_API_KEY"],
        "X-Brevitas-Customer-ID": "acme",
    },
)
```

```bash
export BREVITAS_API_KEY=bvt_...
export OPENAI_BASE_URL=https://api.brevitassystems.com/v1
export BREVITAS_CUSTOMER_ID=acme
```

Claude on **Amazon Bedrock** — the same two headers, plus your own Bedrock API key
as the bearer. Brevitas never holds an AWS credential: Bedrock API keys authenticate
`bedrock-runtime` with `Authorization: Bearer` and no SigV4 signing, so the gateway
forwards the key you send and stores nothing.

```bash
curl https://api.brevitassystems.com/bedrock/model/us.anthropic.claude-opus-5-v1:0/invoke \
  -H "X-Brevitas-Key: $BREVITAS_API_KEY" \
  -H "X-Brevitas-Customer-ID: acme" \
  -H "X-Brevitas-Bedrock-Region: us-east-1" \
  -H "Authorization: Bearer $AWS_BEARER_TOKEN_BEDROCK" \
  -H "Content-Type: application/json" \
  -d '{"anthropic_version":"bedrock-2023-05-31","max_tokens":1024,
       "messages":[{"role":"user","content":"hello"}]}'
```

`/invoke-with-response-stream` works the same way and returns Bedrock's
`application/vnd.amazon.eventstream` frames unchanged. What this lane does **not**
accept, on purpose:

- **The Converse API** (`/converse`, `/converse-stream`) — refused with a `400`.
  Converse reports cache reads and writes under field names the receipt parser does
  not read, and its content blocks are not recognised by the token counter, so every
  Converse call would be recorded as zero measured savings while AWS still charged
  for it. Silent zero is worse than a refusal.
- **Non-Anthropic InvokeModel bodies** (Amazon/Meta/Mistral) — refused for the same
  reason. The body must carry `anthropic_version` and a non-empty `messages` array.
- **SigV4-signed requests** — a signature covers bytes a proxy cannot preserve. Use
  a Bedrock API key.
- **Regions outside `us-east-1/us-east-2/us-west-1/us-west-2`** — these route and
  work normally, but are recorded **unpriced** and bill nothing, because we have not
  verified their Claude rates against the ones in `brevitas/receipts.py`. Under-
  billing is the only direction this codebase rounds.

**Azure OpenAI** — point `azure_endpoint` at the gateway with your resource name in
the path and the rest of your code is unchanged. The Azure SDK builds
`{azure_endpoint}/openai/deployments/{deployment}/chat/completions`, so the resource
name that used to be a hostname becomes the first path segment; Brevitas validates it
down to a DNS label and rebuilds `https://<resource>.openai.azure.com` itself, so no
caller-supplied hostname is ever dialled.

```python
from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://api.brevitassystems.com/azure/contoso-openai",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],   # forwarded; never stored
    api_version="2024-10-21",                     # required, and forwarded
    default_headers={
        "X-Brevitas-Key": os.environ["BREVITAS_API_KEY"],
        "X-Brevitas-Customer-ID": "acme",
        "X-Brevitas-Azure-SKU": "GlobalStandard",   # see below
    },
)
client.chat.completions.create(model="gpt-4.1", messages=[...])   # deployment name
```

`Authorization: Bearer` works too, for Entra ID and managed identity. Streaming works
and returns Azure's SSE bytes unchanged.

Two things about this lane are worth reading before you rely on the numbers:

- **Pricing follows the RESPONSE, not the path.** On Azure the `model=` you pass is
  your *deployment* name, which you chose — it is not evidence of anything. Brevitas
  prices from the dated snapshot Azure returns in the response body
  (`gpt-4.1-2025-04-14`). If the response names no model we recognise, the row is
  recorded **unpriced** and bills nothing rather than being charged at the rate of
  whatever model your deployment happens to be named after.
- **The deployment type has to be declared.** Global Standard, DataZone Standard,
  regional Standard and Provisioned/PTU are the same model at different per-token
  prices, and Azure returns the SKU in no response field and no documented header —
  it is genuinely unreadable. So `X-Brevitas-Azure-SKU` (or `BREVITAS_AZURE_SKU`)
  declares it, and **only `GlobalStandard` prices today**. Everything else, including
  an undeclared SKU, is recorded unpriced. PTU capacity is bought by the hour and has
  no marginal per-token price at all, so a **PTU customer correctly bills $0** instead
  of paying a percentage of a token cost they never incurred.

What this lane does **not** accept, on purpose:

- **The Responses API** (`/openai/responses`, `/openai/v1/responses`) — refused with a
  `400`. Microsoft documents that `response.model` there returns the *deployment* name
  rather than the model, so every Responses row would either bill nothing or bill
  against a label you invented. Use Chat Completions, whose response carries the real
  snapshot.
- **Query parameters other than `api-version`** — refused rather than silently
  dropped, so the request Azure sees is always the request you made.
- **Hosts other than `<resource>.openai.azure.com`** — sovereign clouds and Foundry
  aliases are separate price lists, and routing to one while pricing from another is
  the mistake the SKU rule above exists to prevent.

Confirm your traffic is actually being metered. `brevitas billing-check` and its
`GET /v1/billing/readiness` endpoint are **designed but not yet shipped**; until they
are, ask us and we will read it out of the usage log for you — the query is in
`docs/ONBOARD_HOSTED_CUSTOMER.md` §3.2.

### `X-Brevitas-Customer-ID` is required on every hosted request

An organization service key **rejects every proxy call without it**:

```
400  {"detail": "Organization service proxy calls require X-Brevitas-Customer-ID"}
```

This is the single most common reason a first request fails. It is deliberate: one
organization key can route traffic for many end customers, and the header is what
says which one. Identity assignment is exact and stable — never semantic, never
fuzzy.

- **You are the tenant** (most integrations): use one stable id such as your company
  slug. `brevitas connect` creates that customer record up front and pins it to the
  key it mints, so a header-less call resolves to it rather than 400ing. The header
  still wins whenever it is present, and we still recommend always sending it.
- **You resell to your own customers**: send each end customer's stable id from your
  own database. `brevitas connect --multi-tenant` leaves the key unpinned so a
  missing header stays a hard 400 — attribution is never guessed from "this account
  only has one customer".

Existing customers can be bulk-imported by stable id (`POST /v1/customers/import`) or
created automatically on first traffic. End customers do not install anything and do
not receive Brevitas keys.

## Local proxy — privacy-first, not on savings-based pricing

Everything stays on your machine. Your provider keys stay in **your** environment or
`.env`; Brevitas never receives them in this flow.

Be aware of the tradeoff: receipts from the local proxy arrive over `POST /v1/usage`
and are recorded **non-authoritative**, because a client-side proxy cannot certify
its own savings. They give you dashboards and accounting. They are **not** eligible
for percentage-of-savings billing — that requires the hosted gateway above.

### 1. See where you'd save (no changes made)

```bash
brevitas init            # scans your workspace, finds every LLM call site,
                         # checks which provider keys you have, shows next steps
brevitas init --ai       # add an LLM pass for tricky/dynamic call sites
```

### 2a. Zero-code proxy — no code changes

```bash
brevitas start                         # starts the local proxy on :4242
export ANTHROPIC_BASE_URL=http://localhost:4242
export OPENAI_BASE_URL=http://localhost:4242/openai   # also routes DeepSeek/Groq by model
```

Your existing SDK code now runs through Brevitas unchanged.

### 2b. One-line wrap — per client

```python
import openai, brevitas
client = brevitas.wrap(openai.OpenAI())      # or anthropic.Anthropic()
# use `client` exactly as before — requests are metered and safe cache routing is applied
```

`brevitas apply --write` can insert that wrap for you (shows a diff and asks first).

## What it does per request

A router measures provider prefix-cache behavior and preserves stable prompt prefixes.
OpenAI-compatible providers normally cache those prefixes automatically. For GPT-5.6,
Brevitas can add a tenant-scoped `prompt_cache_key`; billable explicit breakpoints require
`BREVITAS_OPENAI_CACHE_BREAKPOINTS=1`. Brevitas-owned Anthropic cache writes require
`BREVITAS_ANTHROPIC_CACHE=1`, because a write has a premium and no online router can prove
that a future read will occur. Caller-owned cache policy is always preserved.

Quality-affecting features are separately opt-in:

- `BREVITAS_RETRIEVAL_ENABLED=1` can omit context.
- `BREVITAS_COMPRESS_LOSSY=1` can rewrite context.
- `BREVITAS_MESSAGE_REORDER=1` can change conversational ordering.
- `BREVITAS_SEMANTIC_CACHE=1` can reuse a response for a non-identical prompt.

The byte-identical exact response cache is separate and remains available by default;
it skips a model call by replaying a prior complete response. That is reported as a
**call avoided**, not as prompt compression or a blanket losslessness claim.

## Evidence and benchmarks

Historical benchmark percentages in this repository are not product claims. Provider
cache discounts are not Brevitas-incremental savings unless an isolated control arm proves
the difference. New benchmark output must report randomized paired control/treatment runs,
isolated cache namespaces, fixed transcripts, cold and warm results, repeated trials, and
confidence intervals. Without that control evidence, the dashboard shows the provider's
native cache discount but leaves “Brevitas vs control” unmeasured.

## Billing (hosted gateway only)

Brevitas bills a percentage of **verified** savings only. Savings are checked by an
always-valid sequential quality gate (mSPRT) on an audited sample; if a lever's quality
drops, billing for it stops automatically. Every call is logged with the provider's
usage receipt and an idempotency key.

Three things are true and worth knowing before you pick a path:

- **Only hosted-gateway traffic is billable.** Receipts posted by the local proxy are
  recorded non-authoritative by design, because a client-side proxy cannot certify its
  own savings. Local-proxy usage produces analytics, never an invoice.
- **Billing is off until a human at Brevitas attests your commercial arrangement.**
  The database refuses that write from the application entirely, so no bug and no
  leaked key can turn savings into a charge. Ask us for your attestation state at any
  time — it is not an internal detail, and a self-service view of it is planned.
- **Savings that come only from cache replays currently settle at $0.** Today's
  halting conditions stop any period where zero-spend rows dominate the savings, and a
  cache replay is a zero-spend row by construction. The redesign is pending. We would
  rather say this here than have you find it on an invoice.

Operator-side detail: [Onboarding a hosted (billable) customer](docs/ONBOARD_HOSTED_CUSTOMER.md).

## Cloud usage tracking

See [Account and company onboarding](docs/ONBOARDING.md) for the individual,
employee-invitation, workspace-switching, and enterprise-customer flows.

For a SaaS integration, the SaaS company holds one Brevitas service key per environment
(`brevitas connect` mints one; the dashboard's **Company Administration → service
accounts** is the manual equivalent). Each request from its backend includes an exact,
stable `X-Brevitas-Customer-ID` from its own database — see
[the header rules above](#x-brevitas-customer-id-is-required-on-every-hosted-request),
which are the most common cause of a failed first request. End customers do not install
BVX and do not receive Brevitas keys.

AgentMap-discovered backend services, workers, Claude Code, Codex, and custom clients all
write the same content-free receipt:

`account → project → environment → source/agent → provider → model → operation`

```bash
export BREVITAS_API_KEY=bvt_...
export BREVITAS_PROJECT=billing-app
export BREVITAS_ENVIRONMENT=production
export BREVITAS_SOURCE=api-worker
```

`BREVITAS_BASE_URL` selects the control plane receipts are sent to and defaults to
`https://api.brevitassystems.com`. Set it **only** if you run your own API (self-hosted or
local development) — otherwise a self-hosted deployment reports its usage to the hosted
service. Do not give it a `/v1` suffix: the SDK appends `/v1` itself, so a `/v1` base
produces `/v1/v1` and silently 404s.

A `/v1` suffix is correct for **gateway** base URLs (`ANTHROPIC_BASE_URL`,
`OPENAI_BASE_URL`, the OpenAI SDK's `base_url`) and wrong for `BREVITAS_BASE_URL`. Both
`https://api.brevitassystems.com/v1` and `https://brevitassystems.com/v1` reach the
gateway — the marketing origin rewrites `/v1/*` to the API host — but prefer the direct
`api.` host, which is what `brevitas connect` prints and one fewer hop.

When `BREVITAS_PROJECT` is unset the SDK falls back to your local Git-root folder name so
the dashboard has a project dimension. That folder name is your own material, so
`BREVITAS_PROJECT_AUTO=0` suppresses the fallback and sends nothing.

The hosted gateway accepts `X-Brevitas-Key` plus the equivalent `X-Brevitas-*` metadata
headers. Provider keys use their normal `Authorization` or `X-Api-Key` header. Unknown
models retain token totals and are shown as **Unpriced** rather than receiving a guessed
price.

The gateway natively proxies Anthropic Messages plus OpenAI Responses, Chat Completions,
Completions, and Embeddings (including compatible providers). Gemini is **not** currently
a native wrapper or proxy integration. `report_receipt()` can normalize Gemini SDK
`usage_metadata` objects for accounting—including cached, candidate, and thinking tokens—
but it does not optimize Gemini requests or establish Brevitas-attributable savings:

```python
import brevitas

brevitas.report_receipt(
    "google_gemini", "your-model", baseline_tokens=1200,
    usage=response.usage_metadata,
    operation="generate_content",
)
```

For Codex, export `OPENAI_API_KEY` (the customer's provider key), `BREVITAS_API_KEY`,
`BREVITAS_REPO`, and `BREVITAS_CLIENT=codex`, then add this to `~/.codex/config.toml`:

```toml
model_provider = "brevitas"
model = "YOUR_OPENAI_MODEL"

[model_providers.brevitas]
name = "Brevitas"
base_url = "https://brevitassystems.com/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"
env_http_headers = { "X-Brevitas-Key" = "BREVITAS_API_KEY", "X-Brevitas-Repo" = "BREVITAS_REPO", "X-Brevitas-Client" = "BREVITAS_CLIENT" }
```

For Claude Code:

```bash
export ANTHROPIC_BASE_URL="https://brevitassystems.com"
export BREVITAS_CLIENT="claude-code"
export ANTHROPIC_CUSTOM_HEADERS="X-Brevitas-Key: ${BREVITAS_API_KEY}
X-Brevitas-Repo: ${BREVITAS_REPO}
X-Brevitas-Client: ${BREVITAS_CLIENT}"
```

These follow the supported [Codex custom-provider configuration](https://developers.openai.com/codex/config-advanced/)
and [Claude Code environment variables](https://code.claude.com/docs/en/env-vars).

The Supabase `usage_log` stores numeric categories and labels only—never prompts, responses,
code, absolute paths, Git remotes, or raw provider receipts. A hosted proxy necessarily sees
request and response bytes in transit; use the SDK/direct receipt path when that is not acceptable.

## Status

Active development on `main`. The maintained test suites cover the provider proxy,
tenant isolation, receipt accounting, cache safety, and quality gates. Provider support
is described above; no unsupported provider or benchmark percentage is implied.
