Metadata-Version: 2.5
Name: rushhour-gym
Version: 0.5.0
Summary: Rush Hour as a Gymnasium environment, sharing its rules with the psychophysics experiment
Project-URL: Homepage, https://github.com/chrplr/Rush-Hour
Author-email: Christophe Pallier <christophe@pallier.org>
License: Apache-2.0
Keywords: gymnasium,planning,puzzle,reinforcement-learning,rush-hour
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: gymnasium>=1.0
Requires-Dist: numpy>=1.21
Provides-Extra: dev
Requires-Dist: pytest-timeout>=2.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: rl
Requires-Dist: sb3-contrib>=2.3; extra == 'rl'
Requires-Dist: stable-baselines3>=2.3; extra == 'rl'
Description-Content-Type: text/markdown

# rushhour-gym

Rush Hour as a [Gymnasium](https://gymnasium.farama.org/) environment.

The rules are not reimplemented here. They run in the Go program this
repository is built around — the same code the human participants play — and
are served to Python over a pipe. **One action is one cell, which is exactly
what one mouse click is for a participant**, so an agent's trace and a
participant's trace count the same events and can be compared directly.

If you have not used Gymnasium before, start with
[README-AI.md](../README-AI.md), which walks through the same ground slowly.

```python
import gymnasium
import rushhour_gym  # registers the environment ids

env = gymnasium.make("RushHour-Easy-v0")
obs, info = env.reset(seed=0)

terminated = truncated = False
while not (terminated or truncated):
    legal = info["action_mask"].nonzero()[0]
    obs, reward, terminated, truncated, info = env.step(legal[0])

print(info["puzzle"], "solved in", info["n_slides"], "moves;",
      "the optimum is", info["min_moves"])
env.close()
```

## Installing

```sh
pip install rushhour-gym          # from PyPI
pip install -e python[dev]        # from a checkout, for development
```

The Python package needs the `rushhour-env` binary. It is looked for in
`$RUSHHOUR_ENV_BIN`, then on `PATH`, then in the repository root, then built
from source when the checkout and a Go toolchain are at hand:

```sh
go build -o rushhour-env ./cmd/rushhour-env
```

Failing all of that -- the PyPI case -- it is fetched once from the GitHub
release whose tag is the package version (`v` + `rushhour_gym.__version__`),
verified against that release's `SHA256SUMS`, and kept in
`~/.cache/rushhour-gym/<tag>/` (`$XDG_CACHE_HOME` is honoured). Releases carry
a binary for Linux x86-64, macOS arm64 and Windows x86-64; elsewhere, build it.
`$RUSHHOUR_ENV_OFFLINE=1` forbids the download.

Because the package version names the release it fetches from, a release is
cut by bumping `__version__` in `src/rushhour_gym/__init__.py` to the tag's
number and pushing the tag; the release workflow refuses a tag that does not
match, and publishes the package to PyPI after the archives.

## The environment

| id | puzzles | step budget |
|---|---|---|
| `RushHour-v0` | all 49 | 500 |
| `RushHour-Easy-v0` | optimum ≤ 12 moves | 200 |
| `RushHourFixed-v0` | `p02`, the classic board | 100 |
| `RushHourHuman-v0` | as configured; a *person* plays (see below) | none |

**Action space** — `Discrete(32)`: `action = slot * 2 + direction`, direction 0
being left/up and 1 right/down. The vehicle moves exactly one cell, or nothing
happens.

Slot 0 is the red car on every board; slots 1 upward are the other vehicles in
reading order of their starting position, frozen for the episode. Boards have
between 7 and 15 vehicles, so the remaining slots are padding: their actions are
always masked off and their observation rows are zero.

**Observations** — `obs_mode` picks the encoding, all built from the same wire
data:

| mode | space | for |
|---|---|---|
| `planes` (default) | `Box(0, 1, (19, 6, 6))` | one binary plane per slot, plus horizontal / vertical / exit-corridor planes |
| `grid` | `Box(0, 16, (6, 6))` | compact and readable; cell values are names, not quantities |
| `cars` | `Box(0, 6, (16, 4))` | row, col, length, orientation — for an MLP baseline |
| `dict` | `Dict(grid, cars, action_mask)` | SB3's `MultiInputPolicy` |

Whichever you pick, slot identity is visible in the observation: without it an
agent cannot tell which vehicle action `2k` refers to.

**Masking** — `info["action_mask"]` on every `reset` and `step`, and an
`action_masks()` method, which is the name `sb3-contrib`'s `MaskablePPO` looks
for.

**Rewards** — `reward_scheme`:

- `step_penalty` (default): `-1` per step. The return is minus the number of
  cells moved, so the optimal policy is the shortest solution *in clicks* — the
  quantity the human data measures.
- `sparse`: `+1` on solving, `0` otherwise.
- `shaped`: step penalty plus potential-based shaping on the optimal distance to
  go. Policy-invariant, but it runs a search per step; fine for teaching, far
  too slow for the hard end of the library.

**Termination** — `terminated` means solved, and nothing else: Rush Hour has no
dead ends, since every move is reversible. An unsolved episode therefore ends
only by `TimeLimit` truncation, which is why a bare `RushHourEnv()` never
truncates while the registered ids do.

## A person at the board — `RushHourHuman-v0`

`Discrete(32)` names a vehicle and a direction outright; a person with four
buttons cannot. `RushHourHuman-v0` (`rushhour_gym.human.RushHourHumanEnv`) is
the experiment program's own interface as an environment, so that a harness
which presents games to participants needs nothing but a keymap:

```python
env = gym.make("RushHourHuman-v0", puzzle_order="library", n_trials=12)
obs, info = env.reset(seed=0)
frame = env.render()                      # (768, 1024, 3), rushui's picture
obs, r, done, _, info = env.step(rushhour_gym.human.SELECT_NEXT)
```

* **Actions**: `Discrete(8)`, the meta-actions of the program's `rushinput`:
  choose the car above/below/left/right (spatial, the gamepad d-pad), the
  previous/next car (the four-button box and the arrow keys), slide the chosen
  car back/forward. `DEFAULT_KEYS` is the program's keyboard map by key name.
  Choosing is local; a slide becomes the engine's action (`info["env_action"]`).
  Selection follows `rush.Board` (`Neighbour`, `Cycle`), and `movable_only`
  (default on, as in the program) skips cars that cannot move.
* **Rendering**: `rgb_array`, the same picture the program draws — white
  outline on the chosen car, a white arrow at each end it can still slide
  towards, status line. Text needs pygame or Pillow.
* **Trial flow** (`paced`, on by default when a puzzle sequence is given):
  "Puzzle *i* of *N*, press any key" before every puzzle but the first, a
  blank interval (`iti`, 0.8 s), the board, a "PUZZLE SOLVED!" hold
  (`solved_feedback`, 1.2 s). Time-driven transitions happen in `render()`,
  so keep rendering between presses.
* **Puzzles**: `puzzle_order="library"` (easiest first, what the program's
  `-n` presents) or `puzzle_indices=[...]`; otherwise the seeded draw from
  the pool, as `RushHour-v0`.
* **`info`** carries the columns of the program's results file: `event`
  (`trial_start`/`start`/`select`/`move`/`blocked`/`trial_end`/`ignored`),
  `trial`, `puzzle`, `min_moves`, `car`, `orientation`, `from_*`/`to_*`,
  `n_slides`, `solved`, `t_ms`, `trial_ms`, plus `env_action`, `selected`,
  `moved`, `illegal`, `phase`.

No step budget: a participant on a hard puzzle must not be cut off.

## Vector environments

`RushHourVectorEnv` runs every board in one child process and advances them all
in a single request:

```python
envs = gymnasium.make_vec("RushHour-Easy-v0", num_envs=16)
```

The step cost here is the pipe round trip, not the game, so 16 sub-processes
would be 16× the overhead for no gain. `SyncVectorEnv`/`AsyncVectorEnv` over
plain `RushHourEnv` still work if you want them.

## Curricula

The optimal move count of every puzzle arrives in the handshake, so selection
needs no extra round trips:

```python
env = RushHourEnv(min_moves_range=(3, 12))     # at construction
env.set_puzzle_filter(min_moves_range=(3, 20)) # between episodes
env.reset(options={"puzzle": "p07"})           # a specific board
env.reset(options={"spec": "BCCCoo BoooDo oAAEDo oooEoo FFoEoo ooGGGo"})
```

All randomness lives in `reset`, driven by `self.np_random`, so a seeded run is
reproducible.

## Baselines

`rushhour_gym.run_optimal` plays a breadth-first optimal solution — the ceiling
any learning curve should be read against, and the generator for
behaviour-cloning data. `run_random` is the floor. Neither learns anything.

```python
from rushhour_gym import RushHourEnv, run_optimal
with RushHourEnv(puzzle="p02") as env:
    print(run_optimal(env))
```

## Talking to the server directly

The protocol is one JSON object per line and is meant to be driveable by hand:

```console
$ ./rushhour-env -board
{"id":1,"cmd":"hello"}
{"id":2,"cmd":"reset","puzzle":"p02"}
{"id":3,"cmd":"step","action":12}
```

Commands: `hello`, `reset`, `step`, `state`, `reset_batch`, `step_batch`,
`solve`, `close`. The server reports facts and never a reward — the reward
scheme, the termination rule and the observation tensor all live in Python, so
changing them never means rebuilding Go.

## Notes for training

- 6×6 is too small for SB3's `NatureCNN`, which wants at least 36×36. Use
  `MlpPolicy` on a flattened observation, or a small custom extractor with 3×3
  convolutions and no downsampling.
- A random walk on a hard board can wander for a very long time without ever
  being stuck. Start on `RushHour-Easy-v0`.
- Measured throughput: **28k steps/s** for a single environment (36 µs a step)
  and **35k** for `RushHourVectorEnv(16)`, on an Intel Core Ultra 7 165H. It is
  dominated by the pipe round trip and Python's `json`, not by the game. If that
  ever binds, batch first, then try `orjson`.
