Metadata-Version: 2.5
Name: runback-sdk
Version: 0.1.1
Summary: Capture, replay, and audit every AI agent decision — native LangGraph capture, plus a standalone collector/redactor usable from any Python agent framework.
Project-URL: Homepage, https://runback.dev/docs
Project-URL: Repository, https://github.com/letsRunback/runback-proofs
Project-URL: Issues, https://runback.dev/support
Author-email: Runback <contact@runback.dev>
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agents,audit,compliance,eu-ai-act,governance,langchain,langgraph,llm,observability,policy-enforcement,replay,runback,tracing
Requires-Python: >=3.9
Requires-Dist: python-ulid>=2.2
Requires-Dist: requests>=2.28
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
Provides-Extra: capture
Requires-Dist: anthropic==1.12.1; extra == 'capture'
Requires-Dist: crewai==1.15.26; extra == 'capture'
Requires-Dist: google-adk==2.11.0; extra == 'capture'
Requires-Dist: jsonschema>=4.0; extra == 'capture'
Requires-Dist: langchain-core==1.6.9; extra == 'capture'
Requires-Dist: langchain-openai==1.7.0; extra == 'capture'
Requires-Dist: langgraph==1.2.14; extra == 'capture'
Requires-Dist: litellm==1.104.2; extra == 'capture'
Requires-Dist: llama-index-core==0.14.25; extra == 'capture'
Requires-Dist: llama-index-llms-openai==0.8.2; extra == 'capture'
Requires-Dist: openai-agents==0.20.0; extra == 'capture'
Requires-Dist: openai==2.54.0; extra == 'capture'
Requires-Dist: openinference-instrumentation-crewai==1.1.20; extra == 'capture'
Requires-Dist: openinference-instrumentation-google-adk==1.0.3; extra == 'capture'
Requires-Dist: openinference-instrumentation-llama-index==4.5.4; extra == 'capture'
Requires-Dist: openinference-instrumentation-openai==0.1.64; extra == 'capture'
Requires-Dist: opentelemetry-exporter-otlp-proto-common==1.42.1; extra == 'capture'
Requires-Dist: opentelemetry-sdk==1.42.1; extra == 'capture'
Requires-Dist: pydantic-ai-slim[openai]==2.31.1; extra == 'capture'
Requires-Dist: pytest>=8.0; extra == 'capture'
Requires-Dist: responses>=0.25; extra == 'capture'
Provides-Extra: dev
Requires-Dist: jsonschema>=4.0; extra == 'dev'
Requires-Dist: langchain-core>=0.3; extra == 'dev'
Requires-Dist: langgraph>=0.2; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: responses>=0.25; extra == 'dev'
Provides-Extra: langgraph
Requires-Dist: langchain-core>=0.3; extra == 'langgraph'
Requires-Dist: langgraph>=0.2; extra == 'langgraph'
Provides-Extra: openai
Requires-Dist: openai>=1.40; extra == 'openai'
Provides-Extra: openai-agents
Requires-Dist: openai-agents>=0.2; extra == 'openai-agents'
Provides-Extra: otel
Requires-Dist: opentelemetry-exporter-otlp-proto-common>=1.25; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.25; extra == 'otel'
Description-Content-Type: text/markdown

# runback-sdk (Python)

Capture for [Runback](https://runback.dev) — the AI-agent audit/observability platform. First-party capture for the OpenAI and Anthropic clients, LangChain/LangGraph and the OpenAI Agents SDK, plus OpenTelemetry recipes for frameworks that already emit OTel spans.

## Install

```bash
pip install "runback-sdk[openai]"     # or [anthropic], [langgraph], [openai-agents], [otel]
```

Configure with the `RUNBACK_API_KEY` environment variable, or pass `api_key=` explicitly. A runback.dev key (`rb_cloud_…`) sends to runback.dev with nothing else set. Any other key (self-hosted `rb_live_…`) defaults to `http://localhost:3000`, so a self-hosted deployment's data never leaves its network by accident: set `RUNBACK_INGEST_URL` (or `ingest_url=`) to your own origin. Redaction (standard tier) is **on by default**, in-process, before anything is sent.

To have Runback decide each tool call before it runs (server-side policy enforcement), pass `decide=` to `Collector`, for example `Collector(run_name="support-bot", decide={"fail_mode": "closed"})`, then call `run.decide_tool_call(tool, args)` before the tool executes.

## What is tested

Every row below has a test in `tests/capture/` that drives the real, pinned library against a local scripted model provider (no API keys, no network). What each integration sends is then stored through Runback's real ingest path (`storeEvents`) and checked: the right llm and tool steps, the tokens, redaction of a secret placed in the input, and a stored digest. The pins are in `capture-requirements.txt`; CI (`.github/workflows/capture-matrix.yml`) runs them on every push and the newest releases weekly.

| Integration | Module | Tested against |
|---|---|---|
| OpenAI client: `chat.completions`, `responses`; streaming and not; sync and async | `runback.openai.wrap_openai` | openai 2.54.0 |
| Anthropic client: `messages.create`, `messages.stream`; sync and async | `runback.anthropic.wrap_anthropic` | anthropic 1.12.1 |
| LangChain / LangGraph | `runback.RunbackCallbackHandler` | langchain-core 1.6.9, langgraph 1.2.14 |
| OpenAI Agents SDK | `runback.openai_agents.RunbackTracingProcessor` | openai-agents 0.20.0 |
| CrewAI, LlamaIndex, Pydantic AI, Google ADK (OTel recipes) | the stock OTLP exporter, or `runback.otel.RunbackOTLPJsonExporter` | see below |

## OpenAI and Anthropic clients

```python
import openai, anthropic
from runback import Collector
from runback.openai import wrap_openai
from runback.anthropic import wrap_anthropic

run = Collector(run_name="support-agent", input=task)
client = wrap_openai(openai.OpenAI(), run)            # or openai.AsyncOpenAI()
# claude = wrap_anthropic(anthropic.Anthropic(), run)  # or anthropic.AsyncAnthropic()
... # your calls, unchanged
run.finish(output=answer)
```

Captured per model call: messages, system prompt, tool definitions, parameters, response text, tool calls, finish/stop reason, token usage, latency, and errors (recorded, then re-raised). Tool results you send back in the next request become tool steps paired to the model's call id. For OpenAI streams, pass `stream_options={"include_usage": True}` or the stream carries no token counts. If capture itself fails, your call still returns.

## LangChain / LangGraph

```python
from runback import RunbackCallbackHandler

handler = RunbackCallbackHandler(run_name="my-graph")
result = graph.invoke(inputs, config={"callbacks": [handler]})
handler.finish()
```

Captured: each chat-model call (messages, system prompt, the bound tools, streamed chunk count and time to first token, tokens, latency, `metadata.graph_node`), each tool call with its full output and the model's call id, and each graph node as a step. Values are kept whole — there is no character or item cap on anything that is JSON. Not captured: routing rationale for conditional edges (*why* an edge fired, not just which node ran), and checkpoint/interrupt (pause, resume).

## OpenAI Agents SDK

```python
from agents import set_trace_processors
from runback.openai_agents import RunbackTracingProcessor

set_trace_processors([RunbackTracingProcessor()])   # replaces the SDK's default exporter
```

One Runback run per Agents SDK trace. Model calls become llm steps attributed to their agent, function tools become tool steps, and handoffs and guardrails become steps of their own (`[handoff] A → B`, `[guardrail] name: passed|triggered`); a tripped guardrail marks the run `error`. Keyword arguments are passed to each run's `Collector`. The processor observes only: it cannot block a tool call.

## OTel recipes: CrewAI, LlamaIndex, Pydantic AI, Google ADK

These frameworks reach Runback through its OTLP endpoint (`/api/otel/v1/traces`), which accepts protobuf or JSON, gzipped or not. The stock exporter works at its defaults: set `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://<host>/api/otel/v1/traces` and `OTEL_EXPORTER_OTLP_TRACES_HEADERS=authorization=Bearer <RUNBACK_API_KEY>`. A trace exported in several batches is still one run: the server buffers its spans and writes the run once the trace goes quiet.

```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

tp = TracerProvider()
tp.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(tp)
```

Optional: `runback.otel.RunbackOTLPJsonExporter` (the `otel` extra) sends OTLP as JSON and holds each trace until its root span ends, sending it in one request; it reads `RUNBACK_API_KEY` and `RUNBACK_INGEST_URL`. The capture tests drive the recipes through it. You need it only against a self-hosted Runback without `sql/add_otel_trace_buffer.sql` and `sql/create_jobs.sql` applied, where each request is stored as its own run.

Then instrument the framework. Each line below is the setup the test uses:

| Framework | Instrumentation | Tested against | What the stored run is missing today |
|---|---|---|---|
| Pydantic AI | `Agent.instrument_all(InstrumentationSettings(tracer_provider=tp))` | pydantic-ai-slim 2.31.1 | — |
| LlamaIndex | OpenInference `LlamaIndexInstrumentor().instrument(tracer_provider=tp)`; pass `additional_kwargs={"stream_options": {"include_usage": True}}` to the OpenAI LLM or tokens are zero | llama-index-core 0.14.25, openinference-instrumentation-llama-index 4.5.4 | Two llm steps per model call: one complete, with tokens and tool calls; the duplicate has no tokens or tools, and its response text is the instrumentor's repr of the messages |
| CrewAI | OpenInference `CrewAIInstrumentor` **and** `OpenAIInstrumentor`, both with `tracer_provider=tp`. Needs crewai ≥ 1.10.1: below it the instrumentor silently records nothing | crewai 1.15.26, openinference-instrumentation-crewai 1.1.20 | — |
| Google ADK | OpenInference `GoogleADKInstrumentor().instrument(tracer_provider=tp)`. Needs google-adk ≥ 2.10. Without the instrumentor, ADK's own spans make tool calls show up as model calls | google-adk 2.11.0, openinference-instrumentation-google-adk 1.0.3 (model via LiteLLM) | Provider shows as `gcp.vertex.agent`, not the model's vendor |

In all four, every model call arrives as an llm step with its model id, messages, system prompt, tool definitions, response text, tool calls and tokens, and every tool call as a tool step. The remaining gaps are asserted as current behaviour in `packages/sdk/test/capture/pythonFixtures.test.ts`, so the tests fail, and are updated, the day the mapping changes.

Redaction on this path happens **server-side**: spans leave your process unredacted and Runback redacts them before storing. If that is not acceptable, use one of the first-party integrations above, which redact in-process. The same spans exported twice with one key are stored once: the server's trace buffer keeps one copy per span id (without the buffer migration, each request is its own run).

## Digest

SDK-captured runs carry a real `cassette_digest`, computed with the same oracle-chain hashing as the TypeScript SDK and the server, so a digest computed here matches the one ingest computes from the same stored events.
