Metadata-Version: 2.5
Name: reenact
Version: 0.2.0
Summary: Deterministic tests for LLM agents - re-run the agent a change contains against recorded runs, offline, and re-run only what changed against the real model.
Project-URL: Homepage, https://github.com/Manas-33/Reenact
Project-URL: Issues, https://github.com/Manas-33/Reenact/issues
Author: Manas Dalvi
License-Expression: MIT
License-File: LICENSE
Keywords: agents,evals,langgraph,llm,regression-testing,replay,testing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.12
Requires-Dist: pydantic>=2.7
Requires-Dist: typer>=0.12
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.39; extra == 'anthropic'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3; extra == 'langchain'
Description-Content-Type: text/markdown

# Reenact

**Deterministic tests for LLM agents, and an eval cache that only pays for what a
change touched.**

Reenact records a multi-step agent run once - every model call, tool call, and MCP
call - and keeps it as a test fixture. On every pull request, `reenact check` re-runs
the agent the pull request contains against those recordings, offline: the model's
responses come from the recording, while the requests, tool calls, and control flow
are the current code's own. When nothing the agent sends has changed, the test is
deterministic, free, and needs no API key. When something has, Reenact shows exactly
what changed - and can re-run just those scenarios against the real model.

![reenact check re-runs the demo agent offline: an edit to its prompt diverges and shows the change, reverting it is green again](https://raw.githubusercontent.com/Manas-33/Reenact/main/docs/demo.gif)

## Install

```bash
pip install reenact
```

`reenact[langchain]` adds LangChain and LangGraph support; `reenact[anthropic]` adds
the client that model-judged criteria use.

## Quickstart

Write one entrypoint that runs your agent with its model client and tools taken from
`rt`; Reenact calls it both to record a scenario and to check it:

```python
import reenact

def run(rt: reenact.Session, *, question: str) -> None:
    client = rt.anthropic()                       # records, or answers from the recording
    search = rt.tool(search_docs, side_effect="read_only")
    reply = rt.tool(post_reply, side_effect="mutating", stub=fake_post_reply)
    my_agent(client, tools=[search, reply], question=question)
```

Name it in a suite with each scenario's arguments, cassette, and checks, then:

```bash
reenact record evals/suite.toml    # run each scenario for real, write its cassette
reenact check evals/suite.toml     # re-run the current agent against them, offline
```

`reenact init` scaffolds the entrypoint, a suite skeleton, and a pull-request
workflow; `reenact suggest` drafts the checks from a recording.

## What you get

- **Tests of the agent's plumbing with the model frozen** - $0, no API key.
- **Request snapshots** - a change to what the agent sends is reported field by field.
- **Side-effect safety for every tool** - a mutating tool never runs again on replay.
- **A live re-run for what changed** - `--on-divergence live` re-runs only the
  scenarios whose requests changed against the real model, and gates on their checks.
- **Baseline-relative gating** - fail only on drift, including a check deleted to go
  green; a GitHub Action posts a sticky comment and a merge-gating check-run.

Works with the **Anthropic SDK**, **OpenAI SDK**, **LiteLLM**, **LangChain**, and
**LangGraph**.

## Documentation

Full docs, the demo, and the GitHub Action: **https://github.com/Manas-33/Reenact**

## License

MIT
