Metadata-Version: 2.5
Name: rsikit
Version: 0.0.1
Summary: Class-based policies and isolated Gymnasium evaluation for API-only agent research.
Project-URL: Repository, https://github.com/StuartFarmer/rsikit
Project-URL: Issues, https://github.com/StuartFarmer/rsikit/issues
Requires-Python: >=3.10
Requires-Dist: cloudpickle<4,>=3.1
Requires-Dist: gymnasium<2,>=1.3
Requires-Dist: numpy<3,>=1.24
Requires-Dist: pydantic<3,>=2
Requires-Dist: rich<16,>=13
Requires-Dist: slick-ai<0.4,>=0.3
Requires-Dist: sqlmodel<0.1,>=0.0.24
Provides-Extra: box2d
Requires-Dist: box2d-py==2.3.8; extra == 'box2d'
Requires-Dist: gymnasium[classic-control]<2,>=1.3; extra == 'box2d'
Provides-Extra: dev
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: openai
Requires-Dist: slick-ai[openai]<0.4,>=0.3; extra == 'openai'
Provides-Extra: openrouter
Requires-Dist: slick-ai[openai]<0.4,>=0.3; extra == 'openrouter'
Provides-Extra: video
Requires-Dist: gymnasium[classic-control]<2,>=1.3; extra == 'video'
Requires-Dist: moviepy<3,>=2; extra == 'video'
Requires-Dist: pillow<13,>=11; extra == 'video'
Description-Content-Type: text/markdown

# RSIKit

Evolve and evaluate class-based policies in
[Gymnasium](https://gymnasium.farama.org/) environments.

Install the shared library with `pip install rsikit` (Python 3.10+).
Research algorithms and examples require a clone of this repository.
This is an experimental release; APIs may change.

- `Policy`: the agent interface: `reset`, `act`, and `close`.
- `generate`: returns a named `Policy` subclass from the LLM.
- `Run`: stores policies, scores, and returned artifacts.
- `Executor`: owns concurrency and timeouts, using a configurable sandbox.
- `DockerSandbox`: reuses one container across a run, with independent evaluation processes.
- `InProcessDockerSandbox`: runs policy and environment together inside Docker, avoiding per-action IPC.
- `AlphaEvolve`: evolutionary search using Slick and Gymnasium feedback.
- `ShinkaEvolve`: island archives, adaptive model selection, and diff/rewrite/crossover search.
- `LineageSearch`: diverse approach families, measured refinement and pivots, and stagnation-based completion.

The shared library lives in `rsikit/`. The four independent search algorithms
live under [`research/`](research/README.md) and import shared functionality from
`rsikit`, never from another algorithm. Run research examples from the repository
root; the library wheel includes only `rsikit`.

```python
from pathlib import Path

import gymnasium as gym
from slick import prompts
from slick.providers import OpenRouterAPI

import rsikit.generation as generation
from rsikit import Executor, Run, generate

# Configure Slick once at application startup.
prompts.TEMPLATE_ROOT = Path(generation.__file__).parent / "prompts"
provider = OpenRouterAPI(model="openai/gpt-oss-120b:nitro")

# Inside an async function:
policy = await generate(
    "Balance CartPole-v1. Observation: cart position/velocity, pole angle/angular "
    "velocity. Action 0 pushes left and 1 pushes right. Maximize surviving steps.",
    provider=provider,
)
executor = Executor(concurrency=4)
with gym.make("CartPole-v1", max_episode_steps=500) as environment:
    async with Run.create(name="cartpole-comparison", environment=environment, executor=executor) as run:
        scores = await run.evaluate([policy], seeds=[0, 1, 2])
        print(policy.name, scores[policy.id])
        print(run.path)
```

The policy name is generated by the LLM. Pass the returned class directly to the
run; there is no source/file argument to assemble. Automatic Python export is on
by default (`export=False` disables it). Use `run.evaluate(policies)`
for a group; it returns one mean score per policy ID. See [execution, runs, and recovery](docs/RUNS.md).

Docker episodes default to a 60-second wall-clock limit per seed. Exceeding it
raises a recoverable `PolicyTimeout` and terminates that episode's workers while
other episodes continue. With `DockerSandbox`, `call_timeout` separately limits each policy call.

`Executor` defaults to `InProcessDockerSandbox`. To configure its episode timeout:

```python
from rsikit import Executor, InProcessDockerSandbox

executor = Executor(sandbox=InProcessDockerSandbox(episode_timeout=60), concurrency=4)
```

It keeps the restricted Docker container and a fresh process per episode, but
lets the policy share memory with the environment. It protects the host rather
than hidden environment state or scoring integrity. Its external `episode_timeout`
covers the complete rollout; `call_timeout` is not applied. Rebuild the sandbox
image before using it. See [performance measurements](docs/IN_PROCESS_SANDBOX.md).

## Setup

```sh
uv venv
uv pip install --python .venv/bin/python -e '.[dev]'
.venv/bin/python -B -m examples.cartpole
```

Runtime dependencies are Gymnasium, NumPy, Slick (`slick-ai`), Pydantic, SQLModel, cloudpickle, and Rich.

The Docker worker additionally includes SciPy, python-control (`control`), CVXPY
with OSQP/Clarabel/SCS, scikit-learn (`sklearn`), and CPU-only PyTorch (`torch`).
All generators and repair prompts share scientific-library guidance. Image builds
verify Riccati/LQR, convex solvers, regression and Torch autograd; installed versions
are recorded in `/opt/worker/libraries.json`. Rebuild the image after dependency changes.
No sibling checkout or `slick-bits` dependency is required.

## Isolated programs

```sh
docker build -t rsikit-sandbox:local -f rsikit/sandbox/Dockerfile .
.venv/bin/python -B -m examples.circle_packing
.venv/bin/python -B -m examples.circle_packing /path/to/solution.py
```

A program exports `Solution(Policy)`. One policy instance persists throughout an
episode. Run evaluates both environment and agent inside the container, in separate
processes. The lower-level `run_program` helper still keeps its environment on the host.

See the [API guide](docs/INNER_LOOP.md) for task instructions, class contracts,
results, and execution limits.

For finite-shoe blackjack with betting and memory across hands, see the
[blackjack environment](docs/BLACKJACK.md). Benchmark completed hands with
`.venv/bin/python -m examples.blackjack --compare-gym`.
Run a Blackjack test with automatic leader videos after each generation using
`.venv/bin/python -m examples.blackjack_train --output runs/blackjack-test`.

For daily BTC portfolio allocation with transaction fees and a seven-year training /
three-year validation split, see the [Bitcoin environment](docs/BITCOIN.md).
Use `--env Bitcoin` with the optimizer examples, or benchmark locally with
`.venv/bin/python -m examples.bitcoin` (100,000 steps/second minimum).

## Generate and compare five policies with OpenRouter

```sh
uv pip install --python .venv/bin/python -e '.[openrouter]'
docker build -t rsikit-sandbox:local -f rsikit/sandbox/Dockerfile .
export OPENROUTER_API_KEY='your-key'
.venv/bin/python -B -m examples.inner_loop
```

This makes five real generation calls through Slick's `OpenRouterAPI` using
[`openai/gpt-oss-120b:nitro`](https://openrouter.ai/openai/gpt-oss-120b%3Anitro).
OpenRouter's [Nitro routing](https://openrouter.ai/docs/guides/routing/model-variants/nitro)
selects providers for throughput. The example requests random, angle-only,
proportional-derivative, full-state-feedback, and predictive-rollout policies.

Each generated `Solution` class runs in Docker on `CartPole-v1`, using identical
seeds 0–4 and a 500-step cap. The run submits the five policies as one group,
with two evaluations active at a time (`--concurrency` changes this). The terminal shows mean and per-seed scores; higher
is better. Policies with failed episodes are reported without a mean score.
There is no evolutionary search or repair loop in this example.

Each run saves `run.sqlite`, automatically exported Python files under `exports/`,
and comparison reports under a new `runs/cartpole-comparison-*` directory. These are measured results
from your run; the example contains no preset scores or fallback policies.
To shorten evaluation or choose a new output folder:

```sh
.venv/bin/python -B -m examples.inner_loop --seeds 1 2 3 --max-steps 200 --output runs/my-comparison
```

The chosen output directory must not already exist. Inspect the policy code and
report together: the requested strategy is guidance, not a guarantee that the
model follows it. Capped returns can tie, and five seeds do not establish general
performance. This demo needs the OpenRouter key only on the host for generation.

To retry unfinished evaluations without generating anything, supply the same environment
options (`--max-steps`, `--video`) as the original run:

```sh
.venv/bin/python -B -m examples.inner_loop --resume runs/YOUR_RUN
```

For videos, the example configures Gymnasium RecordVideo before creating the run.
Add `--video` (the worker image includes rendering dependencies):

```sh
uv pip install --python .venv/bin/python -e '.[video]'
.venv/bin/python -B -m examples.inner_loop --video
```

## AlphaEvolve

See the [AlphaEvolve guide](docs/ALPHAEVOLVE.md) for the Python API, search controls,
and structured generation contracts. Run the CartPole example with an API model:

```sh
uv pip install --python .venv/bin/python -e '.[openrouter]'
# Set OPENROUTER_API_KEY first; this command makes paid model calls.
.venv/bin/python -B -m examples.alphaevolve --generations 25 --batch-size 10
```

Build the Docker worker above before running the search. The default `paper`
variant uses `openai/gpt-oss-120b:nitro` and overlaps generation with evaluation:

```python
from research.alphaevolve.paper import search

# evaluate_batch returns policy.id -> EvaluationResult with metrics and descriptors.
await search(generator, evaluate_batch, proposals=250, evaluation_batch_size=10)
```

Run saves policies, scores, and artifacts during evaluation, and exports Python
automatically. The optimizer saves its population in `population.sqlite`, with
metric-specific elites in descriptor bins on each island. Lower primary-score
candidates remain available when they win another metric or niche. Ten limits the
evaluation batch size; the retained population is separate. Rich progress shows each policy's
name and description as it is generated, then scores as evaluations finish.
A score table follows each evaluation batch; `run.log` keeps messages and error details.
Self-healing allows two model repairs per policy for malformed generation or sandbox
policy failures (`--max-repairs` changes the limit). Successful scores are reused.
An exhausted repair budget discards that policy; surviving policies and later
generations continue. A generation can return fewer policies than `--batch-size`.
Generation runs four proposals concurrently by default, including their repair calls.
Use `--generation-concurrency` to change this; `--concurrency` controls sandbox
evaluation separately. Set generation concurrency to 1 for sequential requests.
The optimizer lives outside the core in `research/alphaevolve/paper/`. This independently
implements the published mechanisms; the paper does not disclose exact database
sampling/migration rules, so those choices are documented in the guide.
Select `--variant original` for the historical scalar-feedback baseline or
`--variant improved` for the historical independent-founding baseline.
Run names include the variant and `experiment.json` saves resolved settings.
Continue a `paper` run with its saved population and settings:

```sh
.venv/bin/python -B -m examples.alphaevolve --resume runs/YOUR_RUN --generations 25
```

This adds 25 batches using the saved batch size. Logs/history stay in the same
directory. Interrupted proposals are not replayed; older baseline runs have no
optimizer checkpoint. See [resume details](docs/ALPHAEVOLVE.md#resume-a-paper-run).
See [the comparison setup](docs/ALPHAEVOLVE.md#comparing-the-variants).
The CLI also saves per-proposal outcomes, ancestry, repair versions, and per-generation
island champions and resets in SQLite. See [the history schema and island-curve query](docs/ALPHAEVOLVE.md#stored-experiment-history).

In the improved variant, initial proposals compete on separate islands; the same program cannot found
multiple islands just by changing its name. Mutation prompts include each parent's
and inspiration policy's per-seed rewards. The CLI supplies this feedback from Run
automatically; selection still ranks policies by mean reward.

For LunarLander (continuous controls with wind) and then BipedalWalker (normal terrain):

```sh
uv pip install --python .venv/bin/python -e '.[openrouter,box2d]'
docker build -t rsikit-sandbox:local -f rsikit/sandbox/Dockerfile .
.venv/bin/python -B -m examples.alphaevolve --env LunarLander-v3 --seeds 0 1 2
.venv/bin/python -B -m examples.alphaevolve --env BipedalWalker-v3 --seeds 0 1 2
```

Both default to a budget of 250 proposals. BipedalWalker uses full rewrites by
default (`--mode diff` overrides). Duplicate top-level `Solution` definitions are
rejected and repaired. Each command creates a separate run.
The native time limits apply (1,000 steps for LunarLander, 1,600 for BipedalWalker);
`--max-steps` overrides them. `--seeds` evaluates every policy on the same starting
conditions and returns their mean score. The environment wrapper owns observation,
action, reward, and termination instructions, used by both generation and execution.
The `box2d` extra builds `box2d-py` for Python 3.14; uv supplies SWIG during the build.
A C++ compiler is required (on macOS, Xcode Command Line Tools).

After a search finishes, record the three highest-scoring saved policies:

```sh
.venv/bin/python -B -m examples.replay runs/YOUR_RUN --env LunarLander-v3 --top 3 --seeds 0 1 2
```

Use `--env BipedalWalker-v3` for a walker run. Match any original `--max-steps`
override. No API key or generation is needed. Gymnasium records fresh episodes in
Docker; a separate video Run saves their scores and MP4 files and prints each file's
path. Open an MP4 with your video player (`open /path/to/video.mp4` on macOS).
Ranking uses original mean scores and excludes policies with unfinished evaluations.
Replay scores are stored separately, so previously cached scores cannot skip recording.

## ShinkaEvolve

The previous `main` implementation is now available in `research/shinkaevolve/`, using the
same environments, Docker executor, Run storage, and video replay:

```sh
.venv/bin/python -B -m examples.shinkaevolve --env LunarLander-v3 --generations 10 --batch-size 25 --seeds 0 1 2 3 4
.venv/bin/python -B -m examples.shinkaevolve --env BipedalWalker-v3 --generations 10 --batch-size 25 --seeds 0 1 2 3 4
```

Use the OpenRouter/Box2D setup above. The default model is
`openai/gpt-oss-120b:nitro`; repeat `--model MODEL` for adaptive allocation across
an ensemble. The CLI saves proposal ancestry, repair revisions, full island
populations, migration, reflection, and model allocation history in SQLite.
See [the ShinkaEvolve guide](docs/SHINKAEVOLVE.md) for mechanisms, optional novelty
checks, the Python API, and analysis queries.

## EliteSearch

Keep a leaderboard of 10 elites and evaluate 50 new organisms per generation.
Candidates are new ideas, focused edits of an elite, or remixes combining multiple
elites. Only measured scores decide promotion; the top 10 from old elites plus
newcomers survive. Generation and Docker evaluation overlap, with bounded repair
and a Rich leaderboard.

```sh
.venv/bin/python -B -m examples.elitesearch --env BipedalWalker-v3 \
  --elites 10 --population 50 --generations 20 \
  --generation-concurrency 100 --concurrency 8 --max-repairs 5
```

See [the EliteSearch guide](docs/ELITESEARCH.md) for the operator mix, scoring,
saved generations and Python API. This is independent of LineageSearch.

## LineageSearch

Generate broad approach families, evaluate their implementations, and expand promising
parents within each family until all families stagnate or the attempt budget is spent.
MAKER's typed taxonomy adaptation elects the families first, then all initial
approaches beneath them, before any policy implementation. Rich displays the tree;
`decomposition.json` saves it. `--decomposition-k 3` uses five competing partitions
and requires a discriminator vote lead of three.
Each family receives exploration time; additional batches favor strong scores and
recent confirmed gains. `--cull-percent 90` keeps the top 10% globally across
survivors plus newly scored candidates, then expands from those survivors again.
Families with no survivors retire; culled candidates stay archived but cannot
parent later trials.
The optimizer records ancestry, hypotheses, failures, and
completion reasons in the Run database.
Malformed generations and failed policies receive up to two repair calls per
policy (`--max-repairs` changes the limit). Rich progress shows generation,
repairs, evaluation, family completion, and score tables; `run.log` retains the messages.

```sh
.venv/bin/python -B -m examples.lineagesearch --env CartPole-v1 --max-attempts 500
```

Use the OpenRouter and Docker setup above. Search uses seeds 0–4; the final incumbent
is evaluated on separate seeds 100–104. See [the LineageSearch guide](docs/LINEAGESEARCH.md)
for the API, selection rules, stagnation settings, and stored history.

## Checks

```sh
.venv/bin/python -B -m unittest discover -s tests -v
.venv/bin/ruff check .
.venv/bin/ruff format --check .
```

Host and sandbox Python minor versions must match. The Dockerfile defaults to Python
3.14; use `--build-arg PYTHON_VERSION=X.Y` when building for another version.

Docker checks skip explicitly when Docker or the worker image is unavailable.
Research algorithms, examples, and tests are included in the source distribution
and excluded from the library wheel. The core supplies policy generation, shared
edit validation, execution, environments, run storage, and progress display.
