Metadata-Version: 2.4
Name: suspense
Version: 0.3.0
Summary: The fuse box for AI agents: freeze, inspect, resume or stop a runaway agent run without losing its state.
License: Apache-2.0
Keywords: llm,agents,proxy,circuit-breaker,observability
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=6
Provides-Extra: fast
Requires-Dist: httpx>=0.27; extra == "fast"
Dynamic: license-file

# Suspense — the fuse box for AI agents

Agents run unattended. When one goes wrong (loops forever, burns money, does something
you didn't expect) your only options today are *watch it* or *kill it*. Killing it loses
everything it was doing.

Suspense gives you a third option: **freeze it, look at it, then resume or stop it.**

It's a tiny HTTP proxy that sits between your agent and its model provider. Because every
model call carries the agent's full working state (the message array), Suspense can meter
every step, trip a breaker when a run crosses a limit, and hold the agent simply by not
answering yet. The agent blocks. Nothing is lost. An operator decides what happens next.

No framework integration. No code changes. Point `base_url` at Suspense and you're done.

## What it fixes

Agents are the first software that spends money and takes actions on its own, at a pace
set by a model rather than a person. The engineering symptom is "it loops". The business
problems are what the loop does to a budget, to work already done, to the systems the
agent can touch, and to whoever answers for it afterwards. Suspense is aimed at four of them.

| The pain | Who feels it | What changes with Suspense |
|---|---|---|
| **Unbounded spend.** A run has no natural ceiling, and each step re-reads the whole conversation so cost per step grows. The provider bill arrives the next day with no idea which run caused it. | Finance, platform lead | A hard ceiling per run in dollars, steps, minutes and burn rate, enforced *before* the next call. Spend attributed to a run, a tag, a team. |
| **Killing loses the work.** Today the only way to stop a bad run is to kill it. Hours of tool calls and paid tokens are gone, and the re-run often repeats the mistake. | Agent team, operations | Hold instead of kill. The agent keeps its full state; resume continues from that exact call. Nothing is redone or re-bought, and you see the state before deciding. |
| **Irreversible actions with no checkpoint.** An agent that can delete, send, pay or deploy will eventually be told to by its own model. After-the-fact review doesn't prevent it. | Security, compliance | The tool-call gate puts a human between the model's decision and its execution, for exactly the tools you name. Every step is a snapshot, exportable as one file for the incident review. |
| **Nobody can see the fleet.** Past a handful of agents, "what is running, what is it costing per minute, can I stop the one that matters?" has no answer short of grepping logs. | Engineering manager, on-call | One page, every run, live, sorted by dollars per minute, with hold, resume and stop per row. Freeze everything tagged `prod` in one command. Slack alert with the reason. |

The impact is measurable per team with three numbers: runaway runs per month, what each
cost before someone noticed, and the engineer hours spent noticing, killing and re-running.
The breakers remove the first, hold-and-resume most of the second, the fleet view and alerts
shrink the third. The number that decides the security conversation, the cost of one
irreversible action, is the one no team can quote in advance.

**Who it's for.** The platform lead who owns the provider bill and wants a ceiling and
attribution. The security lead asked to sign off on agents touching production, who needs a
checkpoint before actions and evidence after, and who can read the whole thing in one file.
The team shipping the agent, who wants to leave it running overnight and deal with a trip in
the morning without losing the night's work.

**What it is not.** It only sees model calls that go through it, and a hold lands at the
next call, not mid-tool. It is one proxy and one SQLite file, not a control plane. It stores
prompts and responses locally so you can inspect and replay them; that is the feature, and
retention is yours to set.

## Not a gateway

Gateways route, hold API keys, split traffic, filter content, and reject requests that
exceed a budget. They are the right place for all of that, and Suspense sits in front of
one happily: set `upstream` to the gateway and keep everything else. What a gateway
can't do is the reason Suspense exists: it doesn't know which requests belong to one
agent's run, so it can't cap that run; when a limit hits, it can only refuse, which kills
the run instead of parking it; and it never withholds a model's answer so a human can
approve the tool call inside it. Suspense is the intervention layer. It acts on the run,
not the endpoint, and it holds instead of rejecting.

## Quick start

```bash
pipx install suspense
suspense init                      # writes suspense.yaml with a token, prints the one-line change for your SDK
suspense serve                     # :4141; /v1/messages -> Anthropic, everything else -> OpenAI
```

From a checkout, `python3 suspense.py serve` works the same. As a sidecar:

```bash
docker build -t suspense . && docker run -p 4141:4141 \
  -v $PWD/suspense.yaml:/etc/suspense.yaml -v suspense-data:/data suspense
```

In your agent, change one line:

```python
client = OpenAI(base_url="http://localhost:4141/v1")        # OpenAI SDK (Chat Completions and Responses)
client = Anthropic(base_url="http://localhost:4141")        # Anthropic SDK
# LangChain / LangGraph / OpenAI Agents SDK: set the same base URL in its config
```

OpenClaw: add Suspense as a custom provider. Its `openai-completions`, `openai-responses`
and `anthropic-messages` adapters are all shapes Suspense speaks, and the static `headers`
field is how each agent gets its own tag for freeze-by-selector (OpenClaw sends no
identifiers of its own on proxy routes):

```json5
{ models: { providers: { suspense: {
    baseUrl: "http://localhost:4141/v1", apiKey: "${OPENAI_API_KEY}", api: "openai-completions",
    headers: { "X-Suspense-Tag": "inbox-agent" },
    models: [{ id: "gpt-4o", name: "gpt-4o via Suspense", input: ["text"], contextWindow: 128000, maxTokens: 16384,
               cost: { input: 2.5, output: 10, cacheRead: 0, cacheWrite: 0 } }] } } },
  agents: { defaults: { model: { primary: "suspense/gpt-4o" } } } }
```

A held agent is a *waiting* agent, and Suspense keeps it waiting past its own timeout.
Streaming clients get SSE keepalive comments, which every SDK ignores, so a streamed call
can be held indefinitely. Non-streaming clients get 1xx frames chosen per client from the
SDK's own headers: Python stacks skip `100 Continue`, Node's `fetch` skips `103 Early Hints`,
each rejects the other, and unknown clients get none. Verified with the real Python SDKs at
a 3-second timeout held for longer (`tests/sdk_compat.py`) and with the Node SDKs
(`tests/node_sdk_compat.mjs`). Two caveats: the Node SDKs' `timeout` is a hard deadline on
the whole request (default 10 minutes) that no keepalive can extend, so JavaScript agents
should stream or raise it; and Go's `net/http` gives up after five 1xx frames, so set
`held_keepalive_1xx: false` and a long timeout there. Holding is cheap on the proxy side: a
held request is a parked coroutine, not a thread. If an agent gives up and disconnects while
held, Suspense notices and drops it.

When you stop a run, the agent's next call gets a 409 with `type: suspended`. Both SDKs
retry a 409 twice before raising `ConflictError`, so expect a couple of seconds' delay.

Then operate:

```bash
python3 suspense.py ls                       # every run, status, steps, cost
python3 suspense.py show <run> 14            # the exact request at step 14
python3 suspense.py hold <run>               # freeze manually
python3 suspense.py resume <run>             # continue with a fresh budget
python3 suspense.py resume <run> --steps 20 --cost 1.50   # ...with per-run limits for this window
python3 suspense.py stop <run>               # agent's next call gets a 409
python3 suspense.py replay <run> 14 --model gpt-4o-mini   # re-run a saved step
python3 suspense.py export <run> -o run.json # everything about a run: the forensic bundle
```

Every step is a snapshot, so you can branch from one:

```bash
python3 suspense.py fork <run> 14 --set messages.0.content="new system prompt"   # new run from step 14, edited
python3 suspense.py fork <run> 14 --edit request.json --as experiment-7           # or a whole edited request
python3 suspense.py diff <run> experiment-7 14        # the two answers side by side: model, tokens, cost, tool calls, text
python3 suspense.py compare <run> 14 --model gpt-4o-mini   # re-run one step on a cheaper model and diff it, one command
```

Freeze by selector instead of one run at a time. `hold`, `resume` and `stop` all take:

```bash
python3 suspense.py hold --tag prod            # every run tagged prod (X-Suspense-Tag / metadata.tag)
python3 suspense.py hold --model gpt-4o        # model prefix
python3 suspense.py hold --run 'agent-*'       # run id glob
python3 suspense.py hold --prompt 3f9a1c2b7d   # same conversation prefix (see `show <run>`)
python3 suspense.py stop --all --status held   # selectors combine
```

## Fleet view

Open `http://localhost:4141/suspense/` in a browser: every run, live, sorted by spend rate,
with hold / resume / stop buttons. It asks for the control token once and keeps it in the
browser. No build step, no extra dependency: the page is served by the proxy itself.

## Evals from interventions

Every operator decision is a labelled example at the exact state it was made. Suspense
turns them into test cases you can replay against another model or prompt:

| You did | Case it becomes |
|---|---|
| **Deny** a gated tool call | `must_not_call` that tool here (with the rule's argument regex, if any) |
| **Approve** a gated tool call | `must_call` that tool here |
| A **loop hold** fired | `must_not_call` the repeated call with those exact arguments |
| Nothing, just a recorded run | `calls_equal`: the same tool calls the recorded model made, per step |

```bash
suspense evals export -o cases.jsonl               # one case per Deny / Approve / loop hold
suspense evals export --regression <run> -o r.jsonl   # every step of a run, label-free
suspense evals run cases.jsonl --model gpt-4o-mini    # replay, check, tally cost; exit 1 on any failure
```

A case is one JSON line: the full request at that step (redacted, `stream` removed), where
it came from, and one assertion:

```json
{"id": "inbox-run/7/denied",
 "source": {"run_id": "inbox-run", "step": 7, "event": "denied", "ts": 1789600000.0, "tag": "prod", "model": "gpt-4o"},
 "path": "/v1/chat/completions",
 "request": {"model": "gpt-4o", "messages": ["...the exact conversation at step 7..."], "tools": ["..."]},
 "assert": {"must_not_call": {"tool": "delete_mailbox", "args_regex": "@(?!example\\.com)"}}}
```

Replays go through the proxy, so they are metered and snapshotted under an `eval` tag, but
they carry the control token and are never held, gated or budgeted. The same is true of
`replay`, `fork` and `compare`. Decisions are recorded in an `events` table and included in
`export`.

## Export and retention

`export <run>` writes one JSON bundle with the run row and every step's request and
response, for the security team or a bug report. `retention_days: 30` purges runs not
seen for that long, once an hour. Held runs are never purged: they are evidence nobody
has looked at yet. Default is to keep everything.

## Tool-call gate

The same hold primitive, triggered from the response side. When the model asks for a tool
that matches a deny rule, Suspense meters and records the step, then holds the run *before
the agent sees the answer*. The operator inspects what the model wanted to do (`show <run>
<step>`), then `resume` delivers the answer and the agent runs the tool, or `stop` refuses it.

```yaml
tool_gate:
  on_match: hold                   # or: stop
  deny:
    - "delete_*"                   # glob on the tool name
    - "*"                          # ...or every tool: human approval for each action
    - {name: send_email, args: "@(?!example\\.com)"}   # name glob + regex on the JSON arguments
```

Works for OpenAI and Anthropic shapes, streaming or not. On a stream, chunks pass through
as they arrive and the hold happens just before the terminating frame, so the SDK is still
waiting when you decide. Rules match the arguments as sent, so a regex can gate on a
recipient domain, a path, a shell command, or an amount. In the fleet view a gated run shows
what the model wants to call, with its arguments, and Approve / Deny buttons; the Slack
alert carries the same plus the commands to paste.

## Config (`suspense.yaml`)

```yaml
upstream: auto                     # route by shape; or one URL (your vLLM box); or {openai: ..., anthropic: ...}
limits:
  max_cost_usd: 5.00
  max_steps: 200
  max_minutes: 60
  max_tokens_per_minute: 200000
limits_by_tag:                     # per-environment or per-team ceilings
  prod: {max_cost_usd: 50}
  dev: {max_steps: 30}
budgets:                           # ceilings ACROSS runs, per UTC day / month, per tag ("*" = all)
  prod: {daily_usd: 100, monthly_usd: 2000}
  "*": {daily_usd: 500}
max_repeated_tool_calls: 5         # loop detection: same tool, same arguments, this many times in a row
on_trip: hold                      # or: stop
alert_webhook: https://hooks.slack.com/services/...   # message carries the resume/stop command; for a gate, the tool and its arguments
event_webhook: https://ops.example.com/suspense       # JSON {event, run_id, reason, gate_call, overrides, tag} on every hold/gate/resume/stop
control_token: change-me           # required on /suspense/* routes; or env SUSPENSE_TOKEN
readonly_token: dashboards-only    # may GET but never hold/resume/stop
bind: 127.0.0.1                    # loopback by default; 0.0.0.0 to serve a network or container
redact: ["[\\w.+-]+@[\\w-]+\\.[\\w.]+"]   # scrubbed from stored snapshots, never from traffic
http_client: auto                  # httpx if installed (async, no per-call thread), else urllib pool
```

Env vars `SUSPENSE_CONFIG`, `SUSPENSE_UPSTREAM`, `SUSPENSE_PORT`, `SUSPENSE_DB` and
`SUSPENSE_TOKEN` override the file (handy in Docker).

**Control API auth.** With `control_token` set, every `/suspense/*` route (except
`/suspense/health`) requires `Authorization: Bearer <token>`; the CLI sends it
automatically from the same config or env var. A `readonly_token` may read runs, steps and
exports but never change anything: give that one to dashboards. The proxy binds to
loopback by default, so without a token the control API is reachable only from the same
machine; if you bind elsewhere without a token the server warns at startup.

**Per-run limits.** `resume --steps N --cost X --minutes M` sets limits for that run's
next budget window only. A flag you leave out falls back to the global limit, and a plain
`resume` clears all overrides.

**Budgets across runs.** Per-run limits cap the worst run; `budgets` cap the bill. A budget
is per tag (or `"*"` for everything) per UTC day or month, summed over every step recorded
in that window. When it's exhausted, every run under that tag is held as it next calls,
before spending anything. A run you resume with explicit overrides is exempt for that
window: the operator has spoken.

**Loop detection.** `max_repeated_tool_calls: 5` holds a run the fifth time in a row the
model asks for the same tool with the same arguments, before the agent sees the answer.
It catches the classic runaway long before a step cap would. Resume restarts the count.

**Metrics.** `GET /suspense/metrics` is a Prometheus exposition of runs by status, spend,
steps and tokens by tag, and dollars per minute by tag. Scrape it with the read-only token.

**Storage.** Every step is a snapshot, but a conversation only grows, so each step is
stored as a delta on the previous one and rebuilt on read; storage per run is linear in
its length, not quadratic. `redact:` is a list of regexes scrubbed from stored requests and
responses (never from the traffic itself), for emails, card numbers, or your own tokens.

**Upstream client.** With `httpx` installed, provider calls run on the event loop with no
thread per call and no concurrency ceiling; without it, a bounded thread pool of urllib
calls (`upstream_workers`). `pip install suspense[fast]` pulls httpx in.

**Trace ids.** A W3C `traceparent` header, which OpenTelemetry-instrumented agents already
send, is accepted as the run id: the whole trace becomes the run.

**Metering streams.** OpenAI only reports usage on a stream if the client asks. Suspense
asks on the client's behalf (`meter_openai_streams: true`) and hides the extra usage-only
chunk unless the client requested it, so the cost breaker works on plain streams too.

**Prices.** On start, Suspense fetches LiteLLM's maintained price list in the background
(about 1,800 chat models), caches it as `suspense_prices.json` next to the database, and
refreshes it daily. If the fetch fails it keeps whatever it has: cache, then the built-in
table. Anything under `prices:` in your config wins over the fetched values. Set
`prices_url: false` to stay fully offline.

## How runs are identified

First match wins, all configurable under `run_id:` in the config:

1. A request header: `X-Suspense-Run`, `X-Run-Id`, `X-Session-Id` or `X-Conversation-Id`.
2. A body field: `metadata.run_id`, `.session_id`, `.conversation_id`, `.thread_id` or
   `.trace_id`. Anthropic's `metadata.user_id` and OpenAI's `user` are deliberately not
   defaults, since they identify an end-user, not a run; add them if one budget per
   end-user is what you want.
3. On the Responses API, the run that produced `previous_response_id`.
4. Otherwise a fingerprint of the model, the system prompt and the first message after it,
   which stays constant across an agent's loop.

Frameworks add nothing usable on their own: LangChain, LangGraph and the OpenAI Agents SDK
all send bare requests (`tests/framework_compat.py` checks this in CI), so most agents are
tracked by the fingerprint with zero setup. Fingerprinted runs are split into generations:
within one loop the message count only grows, so a request with fewer messages than the
last one is a new invocation of the same agent (a cron job, a re-run), as is one after
`auto_run.idle_minutes` of silence. That keeps a recurring agent from accumulating into one
run forever, and lets it run again after you stopped a previous invocation.

Most SDKs let you set default headers in one line, e.g. `OpenAI(default_headers={"X-Suspense-Run": run_id})`.

## Try it offline

`examples/demo.sh` runs a fake model and a deliberately runaway agent through Suspense
with `max_steps: 5`, then walks through trip → hold → inspect → resume → stop.

To poke at it by hand, `examples/suspense.fake.yaml` points at the fake model:

```bash
python3 examples/fake_llm.py &
SUSPENSE_CONFIG=examples/suspense.fake.yaml python3 suspense.py serve
```

## Tests

```bash
python3 -m unittest discover -s tests -v    # end-to-end against the fake model, stdlib only
```

CI runs the suite on Python 3.9 and 3.13, the offline demo, the real-SDK compatibility
script (`pip install openai anthropic && python3 tests/sdk_compat.py`), the framework
script (`tests/framework_compat.py`: LangChain, LangGraph, OpenAI Agents SDK), a pipx
install of the built wheel, and a Docker build that must answer `/suspense/health`. A `v*` tag publishes to
PyPI via trusted publishing (see `.github/workflows/publish.yml` for the one-time setup).

## What's here and what isn't

Works today: OpenAI Chat Completions, OpenAI Responses and Anthropic Messages request
shapes, streaming and non-streaming (streaming
hold/resume verified against the real OpenAI endpoint; Anthropic streaming is written but
not yet tested live), cost metering with a fetched price table, four breakers,
hold/resume/stop with per-run overrides, freeze-by-selector (tag, model, run glob, prompt
hash), a tool-call gate on the response side, bearer auth on the control API, a live fleet
view, snapshots, export bundles, retention, replay, Slack alerts, SQLite persistence, pipx and Docker packaging, an
asyncio server tested with 2,000 concurrently held connections, budgets across runs,
loop detection, a metrics endpoint, delta snapshots with redaction. Standard library plus
PyYAML; httpx optional.

Not yet: multi-instance deployment, process-level freezing for
agents that don't go through an HTTP model call. See ROADMAP.md.

## License and what stays free

Suspense is Apache-2.0. Everything in this repo is and stays free: the proxy, the breakers,
hold/resume/stop, the tool-call gate, selectors, the fleet view, export, replay. A
single proxy should be something a security team can read end to end and run anywhere.

What will cost money, later and in a separate product: the things one proxy can't do.
A control plane across many proxies and teams, SSO and roles, an audit log of who
resumed what, approvals from Slack, hosted retention and cost reporting over time.
