Metadata-Version: 2.4
Name: predict-rlm
Version: 0.7.0a3
Summary: Production-grade RLMs (Recursive Language Models) with tool use, built on DSPy
Project-URL: Homepage, https://www.trampoline.ai/
Project-URL: Repository, https://github.com/Trampoline-AI/predict-rlm
Project-URL: Issues, https://github.com/Trampoline-AI/predict-rlm/issues
Author: Trampoline AI
License: MIT
License-File: LICENSE
Keywords: ai,dspy,language-models,reasoning,rlm,tool-use
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: deno>=2
Requires-Dist: dspy>=3.1.2
Requires-Dist: nest-asyncio>=1.6.0
Requires-Dist: pydantic<3,>=2.8.2
Requires-Dist: pygments>=2.0.0
Requires-Dist: tenacity>=8.2.0
Provides-Extra: codex-lm
Requires-Dist: litellm>=1.50; extra == 'codex-lm'
Requires-Dist: openai>=1.60; extra == 'codex-lm'
Requires-Dist: tenacity>=9.1.4; extra == 'codex-lm'
Provides-Extra: examples
Requires-Dist: formulas>=1.2.0; extra == 'examples'
Requires-Dist: openpyxl>=3.1.0; extra == 'examples'
Requires-Dist: pymupdf>=1.24.0; extra == 'examples'
Provides-Extra: gepa
Requires-Dist: gepa>=0.0.26; extra == 'gepa'
Requires-Dist: tqdm>=4.0.0; extra == 'gepa'
Provides-Extra: gepa-viz
Requires-Dist: kaleido>=0.2.0; extra == 'gepa-viz'
Requires-Dist: plotly>=5.0.0; extra == 'gepa-viz'
Provides-Extra: sbx
Requires-Dist: websockets<16,>=15.0; extra == 'sbx'
Description-Content-Type: text/markdown

> [!NOTE]
> Read the launch post for our optimized
> [SpreadsheetBench skill](https://x.com/GabLesperance/status/2048072367876735415).

# predict-rlm

Production focused Self-harnessed LM runtime (RLM) that allows the LM to call
its sub-lm with [DSPy](https://dspy.ai) signatures. Define your inputs, outputs,
and tools — the model handles its own control flow. Get fully interpretable
trajectories and performance that scales directly with model improvements.
Without context rot.

Based on the [Recursive Language Models](https://arxiv.org/abs/2512.24601v1)
paper by [Alex L. Zhang](https://x.com/a1zhang),
[Tim Kraska](https://x.com/tim_kraska), and
[Omar Khattab](https://x.com/lateinteraction) from MIT CSAIL.<br/>

<br>
<p align="center">
  <a href="https://github.com/Trampoline-AI/predict-rlm/actions/workflows/tests.yml"><img src="https://img.shields.io/github/actions/workflow/status/Trampoline-AI/predict-rlm/tests.yml?label=Tests" alt="Tests"></a>
  <a href="https://codecov.io/gh/Trampoline-AI/predict-rlm"><img src="https://img.shields.io/codecov/c/github/Trampoline-AI/predict-rlm?token=NNS3R7OIT2&color=brightgreen&label=codecov" alt="codecov"></a>
  <a href="https://pypi.org/project/predict-rlm/"><img src="https://img.shields.io/pypi/v/predict-rlm?color=blue" alt="PyPI"></a>
  <a href="https://pypi.org/project/predict-rlm/"><img src="https://img.shields.io/pypi/pyversions/predict-rlm" alt="Python"></a>
  <a href="https://pypi.org/project/predict-rlm/"><img src="https://img.shields.io/pypi/dm/predict-rlm" alt="PyPI downloads"></a>
  <a href="https://discord.gg/BAkd288sGN"><img src="https://img.shields.io/badge/Discord-Join-5865F2?style=flat&logo=discord&logoColor=white" alt="Discord"></a>
  <a href="https://github.com/Trampoline-AI/predict-rlm"><img src="https://img.shields.io/github/stars/trampoline-ai/predict-rlm?cacheSeconds=3600" alt="GitHub stars"></a>
  <br/>
  crafted  with ♥ in MTL · NYC · FLP<br>by <a href="https://trampoline.ai">Trampoline AI</a>
</p>

## Installation

```bash
uv add predict-rlm
```

Optional extras are available for adjacent tooling:

```bash
# GEPA optimization support
uv add "predict-rlm[gepa]"

# Codex-backed DSPy LM and the `codex-lm` CLI
uv add "predict-rlm[codex-lm]"
```

With the Codex LM extra installed, import `CodexLM` or use the script. The
vendored backend supports current Codex model slugs including `gpt-5.5`.

```python
from dspy_codex_lm import CodexLM
```

```bash
codex-lm auth list
codex-lm usage
```

## Why RLMs?

<p align="center">
  <img src="https://raw.githubusercontent.com/Trampoline-AI/predict-rlm/main/docs/bitter_lesson_spectrum.svg" alt="Bitter Lesson Spectrum — from hand-written prompts to RLMs" width="680"/>
</p>

- **Avoid context rot** — The root LM only interacts with its context
  programmatically through the REPL, staying well within its comfortable
  operating range — enabling complex, long-horizon tasks that would otherwise
  cause models to silently degrade.
- **Bitter lesson-proof: RLMs improve as LMs improve** — Unlike harnesses, which
  can cap or constrain the base model's capabilities, the performance, speed,
  and cost of RLM calls correlate directly with improvements to base model
  capabilities.
  [If the base model handles 10M tokens tomorrow, the RLM handles 100M.](https://alexzhang13.github.io/blog/2025/rlm/)
- **Symbolic reasoning & recursion** — like algebra, RLMs express the
  _structure_ of computation rather than performing each operation individually;
  a single line can represent 1M sub-calls — in direct contrast to agents like
  Claude Code that must mechanically emit each sub-agent call one at a time.
- **Interpretability** — RLM trajectories are fully readable: you can trace
  every peek, chunk, sub-call, and verification step the model takes. This not
  only reveals _how_ the model decomposed a problem, but provides concrete
  optimization signals which tools like [GEPA](https://gepa-ai.github.io/gepa)
  can ingest to evolve the RLM's strategies.
- **Ideal for improving performance per token** — RLMs allow small models to
  punch way above their weight (RLM(GPT-5-mini) outperforms base GPT-5)
  providing great opportunities for reducing costs or stretching limited compute
  budgets without sacrificing quality.

## Features

<p align="center">
  <img src="https://raw.githubusercontent.com/Trampoline-AI/predict-rlm/main/docs/harness_vs_rlm.svg" alt="Classic harness vs RLM architecture" width="600"/>
</p>

- **Multimodal** — process images, documents, audio, and video through sub-LM
  calls using native provider multimodal APIs.
- **Async tool calling** — native RLM async support in the WASM sandbox,
  enabling concurrent sub-LM invocations and tool calls
- **Prompt-optimized skills & tools** — predict-rlm skills comes tested and
  optimized to ensure maximum LM interoperability and performance, bundling
  instructions, PyPI packages, and tools for domain-specific tasks
- **Simple file I/O** — pass local or cloud files as typed inputs and outputs
  via `File`, keeping interop with your existing data pipelines straightforward.
  (S3 files support soon)
- **Structured sub-LM calls** — native Pydantic and DSPy signature support for
  type-safe sub-LM invocations with structured outputs

## Demos

| Description                                                                                                                                              | Input / Output                                                                                                                   | Preview                                                                                                                                                                                                                          |
| -------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Document Analysis](examples/document_analysis/) — Analyze documents and extract key dates, entities, and financial information into a structured report | **Input:** PDFs<br>**Output:** Structured briefing report ([example output](examples/document_analysis/sample/output/report.md)) | <a href="examples/document_analysis/sample/output/report.md"><img src="https://raw.githubusercontent.com/Trampoline-AI/predict-rlm/main/examples/document_analysis/sample/output/screenshot.png" width="280"></a>                |
| [Document Redaction](examples/document_redaction/) — Redact PII from PDFs based on a policy, then verify the redactions visually                         | **Input:** PDFs<br>**Output:** Redacted PDFs ([example output](examples/document_redaction/sample/output/output.md))             | <a href="examples/document_redaction/sample/output/output.md"><img src="https://raw.githubusercontent.com/Trampoline-AI/predict-rlm/main/examples/document_redaction/sample/output/screenshot.png" width="280"></a>              |
| [Invoice Processing](examples/invoice_processing/) — Extract vendor info, line items, and totals from PDF invoices into a consolidated Excel spreadsheet | **Input:** PDF invoices<br>**Output:** Excel spreadsheet ([example output](examples/invoice_processing/sample/output/))          | <a href="examples/invoice_processing/sample/output/output.md"><img src="https://raw.githubusercontent.com/Trampoline-AI/predict-rlm/main/examples/invoice_processing/sample/output/screenshot.png" width="280"></a>              |
| [Contract Comparison](examples/contract_comparison/) — Compare two contract versions and produce a structured diff report with per-section analysis      | **Input:** 2 PDF contracts<br>**Output:** Structured diff report ([example output](examples/contract_comparison/sample/output/)) | <a href="examples/contract_comparison/sample/output/comparison-report.md"><img src="https://raw.githubusercontent.com/Trampoline-AI/predict-rlm/main/examples/contract_comparison/sample/output/screenshot.png" width="280"></a> |

## Quick start

### With your coding agent

Install the [predict-rlm skill](.agents/skills/rlm/SKILL.md) in Claude Code,
Codex, Cursor, or any compatible coding agent:

```bash
npx skills add Trampoline-AI/predict-rlm
```

Then ask your agent to build an RLM:

```
❯ /rlm build an RLM that extracts line items from PDF invoices into a spreadsheet
```

### Quick Example

```python
import dspy
from predict_rlm import File, PredictRLM

class AnalyzeImages(dspy.Signature):
    """Analyze images and answer the query. Load each image as a base64 data
    URI and use predict() with dspy.Image to extract visual information."""
    images: list[File] = dspy.InputField()
    query: str = dspy.InputField()
    answer: str = dspy.OutputField()

rlm = PredictRLM(
    AnalyzeImages,
    lm="openai/gpt-5.4",
    sub_lm="openai/gpt-5.1",
)

result = rlm(
    images=[File(path="page.png")],
    query="Extract all visible text, then count each letter A-Z (case-insensitive).",
)

print(result.answer)
```

### Observability

Every `PredictRLM` call returns a structured `prediction.trace` with iterations,
code, output, tool calls, `predict()` subcalls, timings, token usage, and errors.
Human-readable colored trace blocks are printed to stderr by default; pass
`verbose=False` for quiet execution. Use `debug=True` for timestamped RLM and
sandbox lifecycle diagnostics; error-like debug records are colored red.

### Optional: Docker Sandboxes backend

JSPI/Deno/Pyodide remains the default sandbox. Use Docker Sandboxes (`sbx`) when
you want an explicit opt-in Linux Python runner:

```bash
brew install docker/tap/sbx
sbx login
```

```python
from predict_rlm import PredictRLM, SbxConfig, SbxPool

rlm = PredictRLM(
    "question -> answer",
    sandbox_backend="sbx",
    sbx_config=SbxConfig(name="my-predict-rlm-sbx"),
)
```

By default, `SbxConfig` passes the explicit non-Docker shell template
`docker.io/docker/sandbox-templates:shell` to `sbx create`. Pass a custom
`template="..."` to override it, or `template=None` to omit `--template` and use
Docker's CLI default behavior.

For throughput-sensitive evals or optimization loops, create a pool of prewarmed
runners and pass it explicitly:

```python
with SbxPool(size=4, config=SbxConfig()) as pool:
    rlm = PredictRLM(
        "question -> answer",
        sandbox_backend="sbx",
        sbx_pool=pool,
    )
```

The backend mounts only a per-run staging directory under `.predict_rlm_sbx/` by
default, preserving model-facing paths such as `/sandbox/input/...` and
`/sandbox/output/...` without exposing the rest of the repo workspace. Use
`SbxConfig(extra_workspaces=[...])` only when the sandbox needs explicit
additional host mounts. Real `sbx` integration tests are skipped by default; run
them with `PREDICT_RLM_RUN_SBX_TESTS=1 uv run pytest -m sbx` after the CLI is
installed and logged in.

The native execution stack uses a persistent supervisor plus a persistent Python kernel so successful iterations preserve full REPL state. Per-iteration timeouts are recoverable when the backend can interrupt execution cleanly; native hard-kill fallback restores only a pre-timeout pickleable snapshot and tells the RLM which globals / imports were lost.

See [predict-rlm Architecture](ARCHITECTURE.md) for the component model,
timeout behavior, and shared backend contracts.

### Using the spreadsheet skill

The optimized spreadsheet skill is built in. Import it and pass it through
`skills=[spreadsheet]` so the RLM gets the spreadsheet-specific instructions,
`openpyxl`, `pandas`, `formulas`, and the `formula_eval` verification module
inside its sandbox.

```python
import dspy

from predict_rlm import File, PredictRLM
from predict_rlm.skills import spreadsheet


class UpdateWorkbook(dspy.Signature):
    """Update the workbook following the request.

    Use openpyxl for workbook edits, Excel formulas for derived values, and
    verify formulas before returning the final .xlsx file.
    """

    workbook: File = dspy.InputField(desc="Input .xlsx workbook")
    request: str = dspy.InputField(desc="Requested spreadsheet changes")
    updated_workbook: File = dspy.OutputField(desc="Updated .xlsx workbook")


rlm = PredictRLM(
    UpdateWorkbook,
    lm="openai/gpt-5.4",
    sub_lm="openai/gpt-5.1",
    skills=[spreadsheet],
)

result = rlm(
    workbook=File(path="model.xlsx"),
    request="Add a Summary sheet with revenue by quarter and formulas for totals.",
)

print(result.updated_workbook.path)
```

For workflows that combine source documents and spreadsheets, compose skills:

```python
from predict_rlm.skills import pdf, spreadsheet

rlm = PredictRLM(ProcessInvoices, skills=[pdf, spreadsheet])
```

## Lifecycle callbacks

Hook into the RLM iteration loop to broadcast progress to a UI, write
structured logs, or feed an observability pipeline. `PredictRLM` extends
DSPy's existing callback contract (`dspy.utils.callback.BaseCallback`)
with two RLM-specific handlers:

| Handler | Fires | Receives |
|---|---|---|
| `on_rlm_iteration_start` | Before the action LM is called for iteration N | `call_id`, `instance`, `iteration`, `max_iterations` |
| `on_rlm_iteration_end` | After iteration N finishes (code executed, `IterationStep` built — or an error was raised) | `call_id`, `instance`, `iteration`, `step: IterationStep \| None`, `is_final: bool`, `exception: Exception \| None` |

`call_id` matches the parent module's `on_module_start/end` ID, so events
correlate cleanly with DSPy's own callback events when you invoke the RLM
through DSPy's public module call path (`rlm(...)` or `await rlm.acall(...)`).
Existing `BaseCallback` subclasses keep working unchanged — handlers we
call are opt-in via `getattr`.

**Async-aware.** If you use `await rlm.acall(...)` your handlers may be
coroutines and they will be awaited. Sync `rlm(...)` calls sync handlers;
if it encounters an async handler the coroutine is closed and a warning
is logged.

**Failure-isolated.** Handler exceptions are logged and swallowed — a
broken callback can never break the run.

### Broadcasting a "loading" status to a websocket

```python
import json
from dspy.utils.callback import BaseCallback
from predict_rlm import IterationStep, PredictRLM

class ProgressBroadcaster(BaseCallback):
    def __init__(self, websocket):
        self.ws = websocket

    async def on_rlm_iteration_start(self, *, iteration, max_iterations, **_):
        await self.ws.send_json({
            "type": "iteration_start",
            "iteration": iteration,
            "max_iterations": max_iterations,
        })

    async def on_rlm_iteration_end(
        self, *, iteration, step: IterationStep | None, is_final, exception, **_
    ):
        await self.ws.send_json({
            "type": "iteration_end",
            "iteration": iteration,
            "is_final": is_final,
            "step": step.model_dump(mode="json") if step else None,
            "error": str(exception) if exception else None,
        })

rlm = PredictRLM("query -> answer")
rlm.callbacks = [ProgressBroadcaster(ws)]
result = await rlm.acall(query="...")
```

Register globally instead with `dspy.configure(callbacks=[...])` and the
same handlers fire for every `PredictRLM` instance.

## Next steps

- [How it works](docs/how-it-works.md) — understand the sandbox, REPL loop,
  signatures, and file I/O
- [Architecture](ARCHITECTURE.md) — compare backend component and
  state/recovery semantics
- [API reference](docs/api.md) — constructor params for `PredictRLM`, `File`,
  and `Skill`
- [Skills](docs/skills.md) — define, compose, and mount custom skills
- [RLM-GEPA](src/rlm_gepa/README.md) — optimize RLM skills from traces and
  configure `AgentSpec`
- [Examples](examples/) — end-to-end demos with setup instructions
