Metadata-Version: 2.4
Name: rolloutdev
Version: 0.2.0
Summary: Python SDK + CLI for the rollout agent-environment platform.
Project-URL: Homepage, https://rollout.mv37.org
Author: rollout
License-Expression: MIT
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Description-Content-Type: text/markdown

# rollout — Python SDK + CLI

Author agent environments in Python, push them to the [rollout][platform]
platform, and run the closed loop (rollouts → verifier → fine-tune → compare)
from your terminal.

[platform]: https://rollout.mv37.org

## Install

```bash
pip install rolloutdev
```

> Note: the package is published as **`rolloutdev`** on PyPI because
> `rollout` was taken in 2011 by an unrelated, unmaintained library.
> The Python module name is still `rollout`, so `from rollout import …`
> works as expected. We've filed a PEP 541 claim to recover the name.

## Authenticate

1. Mint an API key at <https://rollout.mv37.org/settings/api-keys>.
2. `rollout login` and paste it.

```bash
$ rollout login
Get an API key at https://rollout.mv37.org/settings/api-keys
Paste API key (rk_…): ********
✓ saved credentials for https://rollout.mv37.org
```

The key lives in `~/.config/rollout/credentials` (mode 0600). Override per-call
with `ROLLOUT_API_KEY` / `ROLLOUT_ENDPOINT` env vars or pass them to `Client()`.

## Define an environment

```python
# refund_agent.py
from rollout import Environment, Task

env = Environment(
    slug="refund-agent",
    name="Refund Agent",
    template="workflow",
    description="Customer-support refund workflow",
    config={
        "tools": [
            {
                "name": "lookup_order",
                "kind": "lookup",
                "fixturePath": "orders.${args.order_id}",
            },
            {
                "name": "refund",
                "kind": "action",
                "appendTo": "refunds",
                "value": {
                    "order_id": "${args.order_id}",
                    "amount": "${args.amount}",
                },
            },
        ],
        "fixtures": {
            "orders": {
                "O-1": {"customer": "Alice", "amount": 100, "status": "shipped"},
            },
        },
        "initialMutableState": {"refunds": []},
    },
    tasks=[
        Task(
            instruction="Refund order O-1 for $100 for Alice",
            hidden_ground_truth={
                "refunds": [{"order_id": "O-1", "amount": 100.0}],
            },
        ),
    ],
)
```

For larger task sets, load from JSONL:

```python
env.tasks = Task.from_jsonl("tasks.jsonl")
```

Each line: `{"instruction": "...", "hidden_ground_truth": {...}, "split": "dev"}`.

## Push, run, improve

```bash
# Upload the env
rollout push refund_agent.py
# → https://rollout.mv37.org/environments/refund-agent

# Run a baseline
rollout run refund-agent --model openai/gpt-4o-mini --n 50

# List recent runs
rollout runs

# Fine-tune on the passing rollouts
rollout improve --run <run-id> --model openai/gpt-4o-mini-2024-07-18 \
                --method sft --suffix v1

# Track training
rollout jobs
rollout job <job-id>
```

DPO instead of SFT (Together AI only):

```bash
rollout improve --run <run-id> --model together/deepseek-ai/DeepSeek-V3.1 \
                --method dpo
```

## Programmatic use

The CLI is a thin wrapper over the `Client`:

```python
from rollout import Client

c = Client()
run = c.create_run(env_slug="refund-agent", model="openai/gpt-4o-mini", n=50)
print(run["runId"])

# When the run completes, kick off SFT on its passing rollouts
ids = c.passing_rollout_ids(run["runId"])
job = c.create_training_job(
    source_run_id=run["runId"],
    rollout_ids=ids,
    base_model="openai/gpt-4o-mini-2024-07-18",
    method="sft",
    suffix="v1",
)
print(job["id"])
```

## Templates

| Template | What the config holds |
|---|---|
| `sql` | `schema` (DDL string) + optional `seed` (INSERT SQL). Tools are fixed: `describe_schema`, `run_sql`, `submit_answer`. |
| `workflow` | `tools[]` (lookup / action / mock kinds), `fixtures{}` (read-only), `initialMutableState{}` (state arrays the agent can append to). |
| `code-editing` | `files{}` (path → content). Verifier checks final file state against task ground truth. |
| `custom` | `tools[]` with paramsSchema + templated response + optional state mutates. The escape hatch. |
| `browser` | `pages[]` with elements + initialUrl. Mock DOM for navigation/form tasks. |

The exact JSON shapes match what the dashboard's "New environment" form
expects — those defaults are the easiest reference.

## Local rollouts (`rollout test`)

Run a single rollout locally, no platform round-trip — fastest way to
iterate on an environment definition.

```bash
rollout test my_env.py --model openai/gpt-4o-mini
rollout test my_env.py --model openai/gpt-4o --task 0 --temperature 0.0
```

The local engine reads your `Environment`, calls the model directly
(through the AI Gateway, or direct OpenAI for `openai/*` models),
executes the env's declarative tools in-process, and runs a subset of
verifier primitives:

- workflow / custom / code-editing templates: ✅ supported
- sql / browser templates: not yet (need sqlite + DOM runtimes)
- verifier primitives: `state_match`, `contains_terms`,
  `tool_was_called`, `no_tool_errors`

Output is a per-task pass/fail with the trace of tool calls. Push to the
platform once you're happy with the env shape.

## Template helpers

The SDK ships builders so you don't hand-roll the platform's JSON shapes:

```python
from rollout import (
    Environment, Task,
    lookup_tool, action_tool, mock_tool,
    custom_tool, custom_config,
    workflow_config, code_editing_config,
    browser_page, browser_config,
    primitive, verifier_config,
)

env = Environment(
    slug="refund-agent",
    name="Refund Agent",
    template="workflow",
    config=workflow_config(
        tools=[
            lookup_tool("get_order", from_array="orders", match_field="id",
                        description="Read an order.", params_schema={"id": "string"}),
            action_tool("refund", state_array="refunds",
                        description="Issue a refund.",
                        params_schema={"order_id": "string", "amount": "number"}),
        ],
        fixtures={"orders": [{"id": "O-1", "customer": "Alice", "amount": 100}]},
        initial_mutable_state={"refunds": []},
    ),
    tasks=[
        Task(instruction="Refund order O-1 for $100",
             hidden_ground_truth={"refunds": [{"order_id": "O-1", "amount": 100}]}),
    ],
)

env.verifier = verifier_config(
    primitive("called_lookup", "tool_was_called", tool="get_order", weight=1),
    primitive("refunds_count", "state_match",
              expected={"refunds.length": 1}, weight=2),
    primitive("no_errors", "no_tool_errors", weight=1),
)
```

## Status

v0.2 — adds template helpers, `rollout test` local rollouts, and the
`compare` CLI command. Real test execution for the code-editing
template (running `npm test` in a sandbox) is on the v0.3 roadmap;
needs sandbox infrastructure (Vercel Sandbox or Modal).
