Metadata-Version: 2.5
Name: mellontoken
Version: 0.5.0
Summary: Client for the mellontoken gateway: ask(prompt), and it tells you what it cost.
Requires-Python: >=3.12
Requires-Dist: httpx>=0.27
Provides-Extra: schema
Requires-Dist: pydantic>=2.0; extra == 'schema'
Description-Content-Type: text/markdown

# mellontoken

Client for the mellontoken gateway.

```python
import mellontoken

answer = await mellontoken.ask("Summarise this tender in one line.")
parsed = await mellontoken.ask(document_text, schema=DocExtraction)
full   = await mellontoken.ask_full(document_text, schema=DocExtraction)
full.cost_usd     # "0.00059550"
full.model        # the model that actually served it
```

## Letting the gateway pick the model

```python
answer = await mellontoken.ask(prompt, model="auto")
answer = await mellontoken.ask(prompt, model="auto", tier="high")   # bound the spend

full = await mellontoken.ask_full(prompt, model="auto")
full.route.model_slug    # what it picked
full.route.task          # the grid row it read
full.route.explanation   # why, in a paragraph you can argue with
```

One command, and one request. `model="auto"` goes to the gateway as the model name;
the gateway classifies the prompt on three axes (task, effort, capability), reads the
resulting grid cell, and serves the answer from whatever that cell names — all inside
the same call. `ask_full` reports the decision on `Answer.route`, so routing is
legible after the fact without a second call to ask about it.

**Two of those three axes are read locally and cost nothing.** A distilled model on
the gateway reads the task and the effort in one forward pass. Only the capability
gate goes to a provider — the distilled model has no capability head, and an unread
gate routes "draw me a diagram of this schema" to a model that cannot draw, then
bills for the failure. So a default `ask` is **one** classifier call, around $0.0002.
`llm=True` puts all three axes on the LLM classifier instead: three calls, roughly
$0.005.

Either way they land on the ledger as `kind=classifier` rows, so the spend is visible
rather than hidden. **A service sending the same shape of prompt a thousand times
should read `full.route.model_slug` once and then name that slug**: the decision is a
property of the workload, not of each call.

`tier=` bounds what the router may spend and outranks the effort classifier, because
a tier is a statement about your budget rather than a property of the text. Set it
per call, or once via `MELLONTOKEN_TIER`. It is only read on the `auto` path — with a
named model it is logged as ignored rather than silently dropped.

**A router that resolves nothing is not an error.** When the capability gate rejects
every candidate in the cell the gateway answers 422 with the reason; `ask` logs it and
retries against `MELLONTOKEN_MODEL`, which is what that error advises — keep your own
model, so `auto` is never *less* reliable than naming a slug. The one case that raises
is `MELLONTOKEN_MODEL=auto` as well, where there is no own model to keep.

## Upgrading to 0.5.0

`ask(..., llm=True)` is unchanged and still does what it says. What changed is
underneath it: the client now sends `classifier="llm"`, where it used to send
`local=False`, because the gateway replaced a boolean with a named choice — a
boolean never said what would read the prompt instead.

**If you passed `local=` directly, it is gone.** It was never documented here,
but it was reachable: `local=True` becomes the default (send nothing), and
`local=False` becomes `llm=True`.

**Upgrade the client if you use `llm=True`.** Against a current gateway, 0.4.0's
`llm=True` puts nothing on the wire, and the gateway now reads that as its own
default — the local classifier. It will not fail; it will quietly give you the
cheaper read. 0.5.0 says what it means.

**`web_search=True` no longer narrows which model answers.** The gateway runs the
search itself rather than asking the provider, so the request needs a model that
can call a function instead of one with a search index of its own — which is
nearly all of them. A research question is now routed on the work it is, not on
who happens to be able to look things up.

On `model="auto"` the flag is an override rather than the trigger: the router
reads the need off the prompt, so a question that plainly wants current
information searches without being told to. Pass `web_search=False` to forbid it.

`Answer.web_search_calls` says how many searches ran. Read it next to
`input_tokens`: the fee is a tenth of a cent, and what searching costs is the
results arriving as input.

## Configuration

From the environment:

| variable | default |
|---|---|
| `MELLONTOKEN_BASE_URL` | `https://dev.mellontoken.internal.techmellon.com/api/v1` |
| `MELLONTOKEN_API_KEY` | — required |
| `MELLONTOKEN_MODEL` | `auto` (or a catalogue slug, to pin every call) |
| `MELLONTOKEN_TIER` | — (the effort classifier judges it) |
| `MELLONTOKEN_MAX_TOKENS` | `32000` |
| `MELLONTOKEN_TIMEOUT` | `600` |

or in code:

```python
mellontoken.configure(base_url="http://gateway:8000/api/v1", api_key=key, model="gpt-5.6-luna")
```

**`MELLONTOKEN_API_KEY` is the only variable most callers set.** The default base
URL is the dev gateway, which resolves on the internal network only — from outside
it, set `MELLONTOKEN_BASE_URL` yourself. Running the stack locally, that is
`http://localhost:8000/api/v1`, or `http://host.docker.internal:8000/api/v1` from
inside a container, where `localhost` is the container rather than the host.

## What `ask` does that a bare HTTP call does not

- **Sends the schema twice**, as `response_format` *and* as prompt text. The request
  field is only honoured where the provider can enforce it; Gemini accepts a schema
  it will then ignore, and the field names in the prompt are what stop it inventing
  `col_idx` for `col`.
- **Reads JSON out of a fenced reply**, and wraps a bare list under the schema's
  single array field when that is unambiguous.
- **Retries the shape, not the wire.** A 401/402/403/404 is final — a key, a budget,
  a permission, a missing model — and raises immediately. A reply that did not parse
  is retried three times.
- **Logs the cost of every call**, which is the reason to route through the gateway
  at all: `mellon.cost_usd` is the only per-call price any provider path reports.
- **Sends `model="auto"` straight through**, and lets the gateway route it. This
  client used to resolve it first against `/api/auto/preview` and then send the slug
  that named — two round trips for one logical request. That endpoint still exists,
  for callers that want a decision *without* a completion; this path no longer uses
  it. Routing happens once per `ask`: the first reply names the model it chose and
  the shape-retries are pinned to it, so a reply that did not parse is a reason to
  ask the same model again rather than to pay three classifiers a second time.

## Install

Built from this directory, which lives in the gateway's own repo so the client and
the endpoint version together:

    uv build            # -> dist/mellontoken-0.1.0-py3-none-any.whl
