Metadata-Version: 2.4
Name: rl-scape
Version: 0.1.4
Summary: Controlled server-authoritative RuneScape environment for continual RL
Project-URL: Homepage, https://github.com/StevenDavenport/RLScape
Project-URL: Repository, https://github.com/StevenDavenport/RLScape
Project-URL: Issues, https://github.com/StevenDavenport/RLScape/issues
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: gymnasium<2,>=1.0
Requires-Dist: numpy>=1.23
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Provides-Extra: manual
Requires-Dist: pygame>=2.5; extra == "manual"

# RLScape

### An expandable RuneScape environment for continual and multi-task reinforcement learning

[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-3776AB?logo=python&logoColor=white)](https://www.python.org/)
[![Gymnasium](https://img.shields.io/badge/API-Gymnasium-0081A5)](https://gymnasium.farama.org/)
[![Environment](https://img.shields.io/badge/environment-RLScape--v1-6C5CE7)](docs/api_v1.md)
[![Tests](https://img.shields.io/badge/tests-78%20passing-2EA44F)](#development)

RLScape turns a 2006-era RuneScape world into a controlled, Gymnasium-compatible
environment for continual learning, multi-task learning, and general reinforcement
learning research.

The first environment is a compact Lumbridge arena with five atomic goals,
randomized valid spawn tiles, raw mouse control, fast server-side resets, factual
events, and portable snapshots. The policy sees pixels; privileged world state
stays in `info` for research diagnostics.

## Install

```bash
pip install RLScape
rlscape runtime install
rlscape runtime status
rlscape doctor
```

The lightweight Python package and matching Java/game runtime are versioned
separately. The runtime is SHA-256 verified and installed in the user's data
directory rather than inside the Python package. Offline archives and custom
runtime locations are also supported.

<p align="center">
  <img src="assets/sample_frames/frame_765x503.png" alt="RLScape Lumbridge environment" width="765">
</p>

> **Very early release:** RLScape is at the beginning of its development. APIs,
> tasks, content, and compatibility may change substantially while the first
> stable research interface takes shape.

RuneScape is unusually well suited to an expandable learning benchmark: skills,
resources, combat, inventories, quests, tools, locations, and long-horizon task
chains can be introduced gradually without changing the fundamental interaction
model. The compact Lumbridge arena is only the first controlled slice of that
much larger possibility.

## Why RLScape?

| Capability | Contract |
| --- | --- |
| Temporally clear steps | action → logical tick → aligned pixels, state, events, reward, and tick ID |
| Controlled episodes | seeded task reset with randomized valid arena spawns |
| Continual-learning lifetimes | task reset, lifetime reset, and disposable evaluation clones are distinct |
| Reproducible experiments | immutable content-addressed snapshots include controlled state and RNG streams |
| Policy-safe diagnostics | RGB observations are separate from server-authoritative privileged information |
| Accessible mouse control | absolute normalized coordinates can reach any visible target in one action |

Warm server fixture resets take roughly **1 ms**. Complete Gym resets, including
position synchronization and an RGB observation, have measured around **20 ms
median** on the development machine.

## Atomic task arena

Each environment instance keeps its current goal across episodes unless the
caller explicitly requests `goal_switch=True`.

| Goal | Objective | Reset fixture |
| --- | --- | --- |
| `kill_goblin` | Kill one arena goblin | equipped weapon |
| `bury_bones` | Bury one set of bones | bones in inventory |
| `chop_logs` | Chop normal or oak logs | axe in inventory |
| `light_fire` | Light one fire | tinderbox and log |
| `catch_fish` | Catch one fish | fishing equipment and bait |

The arena world, player state, inventory, equipment, skills, and task event ledger
are restored server-side between episodes. The controlled `agent` account starts
with level 50 in every skill.

## Quick start from source

The current supported development workflow uses Python 3.11, Java 8, and Maven:

```bash
git clone https://github.com/StevenDavenport/RLScape.git
cd RLScape

conda create -n rlscape python=3.11 pip -y
conda install -n rlscape -c conda-forge maven=3.9 -y
conda activate rlscape
pip install -e ".[dev,manual]"

rlscape doctor
rlscape build
python scripts/manual_play_env.py --name agent --goal light_fire
```

Manual controls are printed at launch. Use `R` to reset the same goal, `G` to
switch to a sampled goal, `1`–`5` to select a particular task, and `E` to print
the server-authoritative event ledger.

## Gymnasium API

```python
import gymnasium as gym

env = gym.make(
    "RLScape-v1",
    goal_switch=False,
    reward_mode="sparse_success",
    camera_mode="birdseye",  # default; use "follow" for the previous camera
)

observation, info = env.reset(
    seed=7,
    options={"goal": "light_fire"},
)

observation, reward, terminated, truncated, info = env.step({
    "mode": 2,                 # 0=no-op, 1=move, 2=left click, 3=right click
    "position": [0.0, 0.0],   # normalized absolute (x, y) in [-1, 1]
})

# The goal persists; only the valid spawn tile changes.
observation, info = env.reset()

# Explicitly sample a different goal.
observation, info = env.reset(options={"goal_switch": True})
```

The default sparse contract returns `+1.0` exactly once when the atomic task
succeeds and `0.0` otherwise. Use `reward_mode="none"` when an external agent or
experiment owns reward construction. Death is a zero-reward terminal failure;
reaching the logical-tick budget is a truncation.

See the [v1 API reference](docs/api_v1.md) for observation spaces, action
adapters, reset options, termination reasons, and `info` fields.

## Mouse action space

```python
spaces.Dict({
    "mode": spaces.Discrete(4),
    "position": spaces.Box(-1.0, 1.0, shape=(2,), dtype=np.float32),
})
```

Coordinates cover the complete returned frame and map deterministically to
pixels. Absolute control lets one decision click any visible target without
making cursor movement a hidden multi-step motor task.

RLScape also provides:

- pixel-coordinate helpers for manual tools;
- an optional flat discrete, left-click-only grid wrapper for discrete-only RL
  stacks;
- one-hot mode plus normalized coordinates for world-model action encodings;
- the exact executed normalized, rendered-pixel, and raw-client coordinates in
  `info["executed_action"]`.

The canonical mixed-mouse action remains the default. To constrain a policy to
one left-click at a cell centre on every step:

```python
from rl_scape import ClickGridActionWrapper

env = ClickGridActionWrapper(rl_scape.make())
assert env.action_space.n == 28 * 18
```

Grid actions use row-major ordering from the top-left:
`action = row * columns + column`. Try candidate resolutions against all five
tasks with the centre-overlay validation tool:

```bash
python scripts/manual_grid_test.py --grid 28x18 --goal kill_goblin
python scripts/manual_grid_test.py --grid 40x26 --goal kill_goblin
python scripts/manual_grid_test.py --grid 48x32 --goal kill_goblin
python scripts/manual_grid_test.py --grid 32x21
python scripts/manual_grid_test.py --grid 32x21 --goal catch_fish --scale 1
```

The tool advances the environment only on a grid click, so waiting behavior
must also be tested using grid clicks rather than hidden no-op actions.

Resized observations use `resize_filter="area"` by default. It averages a
complete partition of the raw frame so thin features are not discarded during
downsampling. `resize_filter="nearest"` remains available for old checkpoints
and exact compatibility with the earlier observation pipeline:

```python
env = rl_scape.make(resize=(384, 252), resize_filter="area")
```

Game clients launched by RLScape are always muted, including headless
experiments, manual tools, and evaluation clones.

The default `camera_mode="birdseye"` follows the player at the engine's
steepest supported overhead angle, with the game view, minimap, and compass
fixed north-up. Use `camera_mode="follow"` to select the previous
transition-synchronous camera that turns with player movement.

## Experimental architecture

```mermaid
flowchart LR
    A[Gymnasium policy] -->|normalized mouse action| B[Python environment]
    B -->|prepared action| C[Client bridge]
    C -->|action barrier| D[Authoritative game server]
    D -->|one logical tick| D
    D -->|state + events + tick ID| B
    C -->|aligned RGB frame| B
    B -->|observation, reward, done, info| A
```

The server controls the transition boundary. Frames, factual events, privileged
state, reward, and logical tick identifiers describe the same post-action
instant. Repeated, skipped, delayed, or out-of-order actions are rejected rather
than silently folded into a trajectory.

Snapshots are immutable and content-addressed. Restoring a Gym snapshot rewinds
the controlled server state together with the environment RNG and task stream.
Evaluation clones run in disposable private runtimes with distinct ports and
lifetime identifiers so evaluation cannot contaminate training.

## Runtime installation

The Python distribution is intentionally small. Java binaries and game data live
in a separately versioned, SHA-256-verified runtime bundle.

The runtime defaults to the user's data directory rather than modifying the
installed Python package. Offline archives, custom locations, explicit removal,
and source staging are supported. Developers can stage a runtime locally with:

```bash
rlscape build
rlscape runtime install --source . --install-dir /tmp/rlscape-runtime
```

## Development

```bash
# Python tests
python -m pytest

# Build Java client and server
rlscape build

# Inspect the controlled arena
rlscape survey-arena --output /tmp/lumbridge_v1-survey.json

# Benchmark warm resets
rlscape control-reset-benchmark --username agent --iterations 100 --warmup 10
rlscape gym-reset-benchmark --username agent --iterations 100 --warmup 10

# Snapshot, replay, and lifetime-isolation smoke test
rlscape control-snapshot-smoke --username agent
```

Python edits are immediately visible with an editable install. Re-run
`rlscape build` after modifying Java sources, resources, or Maven configuration.

## Documentation

- [Controlled CRL development checklist](docs/controlled_crl_roadmap.md)
- [Gymnasium API v1](docs/api_v1.md)
- [Control protocol v2](docs/protocol_v2.md)
- [Experiment specification](docs/v1_experiment_spec.md)
- [Repository inventory](docs/repository_inventory.md)

## Repository layout

```text
configs/                 Arena and experiment configuration
docs/                    API, protocol, experiment, and roadmap documentation
scripts/                 Manual-play, capture, and smoke-test utilities
src/rl_scape/            Python Gymnasium package
tests/                   Python contract and regression tests
third_party/2006scape/   Upstream Java client/server source and assets
```
