Metadata-Version: 2.5
Name: simhub
Version: 0.2.1
Summary: Define an environment, a policy and a task in Python; run them on hosted GPUs
Project-URL: Homepage, https://github.com/General-Deployment/sim
License: Proprietary
Keywords: evaluation,mujoco,reinforcement-learning,robotics,simulation
Requires-Python: >=3.9
Requires-Dist: httpx>=0.27
Requires-Dist: numpy>=1.24
Requires-Dist: pydantic>=2.7
Requires-Dist: tomli>=2.0; python_version < '3.11'
Requires-Dist: typing-extensions>=4.12
Provides-Extra: all
Requires-Dist: huggingface-hub>=0.26; extra == 'all'
Requires-Dist: imageio-ffmpeg>=0.5; extra == 'all'
Requires-Dist: ipython>=8.0; extra == 'all'
Requires-Dist: matplotlib>=3.7; extra == 'all'
Requires-Dist: msgpack>=1.0; extra == 'all'
Requires-Dist: mujoco==3.11.*; (python_version >= '3.10') and extra == 'all'
Requires-Dist: pandas>=2.0; extra == 'all'
Requires-Dist: pillow>=10.0; extra == 'all'
Requires-Dist: pyarrow>=15.0; extra == 'all'
Requires-Dist: websockets<16,>=15; extra == 'all'
Provides-Extra: dataset
Requires-Dist: pyarrow>=15.0; extra == 'dataset'
Provides-Extra: mujoco
Requires-Dist: mujoco==3.11.*; (python_version >= '3.10') and extra == 'mujoco'
Provides-Extra: notebook
Requires-Dist: ipython>=8.0; extra == 'notebook'
Requires-Dist: matplotlib>=3.7; extra == 'notebook'
Requires-Dist: pandas>=2.0; extra == 'notebook'
Requires-Dist: pillow>=10.0; extra == 'notebook'
Provides-Extra: openpi
Requires-Dist: msgpack>=1.0; extra == 'openpi'
Requires-Dist: pillow>=10.0; extra == 'openpi'
Requires-Dist: websockets<16,>=15; extra == 'openpi'
Provides-Extra: typing
Requires-Dist: datamodel-code-generator==0.35.0; extra == 'typing'
Provides-Extra: video
Requires-Dist: imageio-ffmpeg>=0.5; extra == 'video'
Requires-Dist: pillow>=10.0; extra == 'video'
Provides-Extra: warp
Requires-Dist: mujoco-warp==3.14.0; (python_version >= '3.10') and extra == 'warp'
Requires-Dist: mujoco==3.14.0; (python_version >= '3.10') and extra == 'warp'
Requires-Dist: warp-lang==1.15.0; (python_version >= '3.10') and extra == 'warp'
Provides-Extra: weights
Requires-Dist: huggingface-hub>=0.26; extra == 'weights'
Description-Content-Type: text/markdown

# SimHub SDK

SimHub is a Python SDK for building simulation environments, running robot
policies, and evaluating their results. Write your environment and controller
in Python, submit their source to hosted SimHub, and inspect runs, metrics,
recordings, and task outcomes in the
[SimHub console](https://simhub-console.pages.dev/).

This guide covers SDK **0.2.0**. Start with installation and the hosted example;
use the authoring reference when building your own project. The examples submit
to hosted SimHub by default, including the complete single-file starter. Local
execution is an explicit debugging option.

- [Install and connect](#install-and-connect)
- [Run your first hosted simulation](#run-your-first-hosted-simulation)
- [Author a project](#author-a-project)
- [Submit a complete single-file project](#submit-a-complete-single-file-project)
- [Compare policies and vary environments](#compare-policies-and-vary-environments)
- [Debug and recover runs](#debug-and-recover-runs)
- [Guidance for coding agents](#guidance-for-coding-agents)

## How SimHub fits together

An **environment** is the simulated world: it accepts actions, produces
observations, and reports measurements. A **policy** is the controller: it reads
observations and chooses the next action. A **task** defines success using the
environment's measurements, such as placing an object without a collision.

A **project** is the Python source and assets needed to build that world and run
its policies. An **episode** is one execution of an environment and policy with
a particular seed. Hosted episodes produce **runs**; the service groups work
into batches and schedules jobs for compilation, simulation, and requested
postprocessing. An **experiment** freezes a collection of episodes and places
limits on their total work.

For new hosted integrations, use `EnvironmentDefinition`, `PolicyDefinition`,
`TaskDefinition`, and `RunRequest`. These are validated descriptions of your
code and parameters. The backend constructs the actual runtime objects.
Decorator-based projects using `@simhub.environment` and `@simhub.policy` are
also supported and submit to the same hosted service.

## Physics backend

Native MuJoCo remains the default. Manifests and single-file declarations
without an image use `python-cpu`; typed packages without a manifest retain
`python-gpu` for policy/rendering resources with native physics. Existing
explicit runtime selections are preserved.

MuJoCo Warp is a supported opt-in backend. Select `image = "python-mujoco-warp"`
in `simhub.toml` and, for typed declarations, set
`physics=sh.MuJoCoPhysics(backend="mujoco-warp")`. It requires an allocated NVIDIA
GPU and a deployed runtime advertising `physics:mujoco-warp-v1`, with no silent
CPU fallback. `sh.MuJoCoPhysics()` and `sh.create_physics(model)` use native physics.

Construct `sh.create_physics(model, config=context.physics)` in the runtime. Use
its `step`, `reset`, `upload_state` and `bind_recorder` methods instead of directly
calling `mujoco.mj_step`. `MujocoEnvironment` subclasses honor the explicitly
selected hosted backend automatically.

See [the Warp runtime guide](../docs/SIMHUB_MUJOCO_WARP.md) for state synchronization,
cloth recording, migration and the hosted acceptance example. GPU acceleration
does not itself repair scene collisions or guarantee a speedup for one world.

## Install and connect

The core SDK supports Python 3.9 or newer. Use Python 3.10 or newer for local
MuJoCo and the repository examples in this guide. In your Python environment:

```bash
python -m pip install simhub
python -c "import simhub; print(simhub.__version__)"
simhub --help
```

The base installation contains everything needed to submit the examples in this
guide. Hosted workers provide the simulation engines; MuJoCo, Blender, and model
weights do not need to be installed on the submitting machine. Optional extras
support client-side analysis and explicitly requested local development:

```bash
python -m pip install 'simhub[mujoco]'    # Local MuJoCo, pinned to the supported series
python -m pip install 'simhub[video]'     # Local video encoding and image support
python -m pip install 'simhub[notebook]'  # Tables, plots, and notebook display
python -m pip install 'simhub[openpi]'    # OpenPI WebSocket client
python -m pip install 'simhub[dataset]'   # LeRobot state/action export
python -m pip install 'simhub[weights]'   # Hugging Face checkpoint access
python -m pip install 'simhub[typing]'    # Generate parameter classes from schemas
python -m pip install 'simhub[all]'       # Runtime extras above, excluding typing
```

The standard hosted Python runtimes include MuJoCo, NumPy, Pillow, video support,
and the OpenPI client transport. Projects do not need to repeat these packages
in their manifests. Declare only additional project-specific libraries. Blender
and its `bpy` module need separate installation only for explicit local use.

For hosted work, sign in to the
[SimHub console](https://simhub-console.pages.dev/), open **API keys**, and create
a key. Make it available to your process:

```bash
export SIMHUB_API_KEY="your-api-key"
```

```python
import simhub as sh

with sh.connect() as sim:
    print(sim)
```

The default API is `https://api.g-deployment.com`. To use another deployment,
set `SIMHUB_API_URL` or pass `sh.connect(url="https://your-api.example.com")`.
Explicit `url=` and `key=` arguments take precedence over environment variables.
Keep keys out of project source and uploaded assets.

`connect()` is hosted-only by default and prevents `project.run_local(...)`
through that session. No additional flag is needed for hosted execution.
Use `sim.run(...)` for typed requests or `sim.project(...).launch(...)` for
factory projects. Both upload source and submit hosted jobs whose results appear
in the console. The CLI's `simhub launch PROJECT` and `simhub run PROJECT`
commands also default to hosted submission.

For repository contributors: the published SDK and the server distribution
share the `simhub` import name. Use separate environments for them. Installing
`./simhub` directly selects the server's build configuration; see the
[SDK build and release instructions](https://github.com/General-Deployment/sim/blob/main/simhub/PUBLISHING.md)
when you need an SDK built from source.

## Run your first hosted simulation

The repository includes a complete warehouse project: a scripted robot scans
three shelf labels and returns to its starting position. Its classes are
project code, so get the example before importing them:

```bash
git clone https://github.com/General-Deployment/sim.git
cd sim/examples
```

With the SDK installed and your API key configured, save the following as
`run_warehouse.py` in this `examples` directory:

```python
from pathlib import Path
import simhub as sh
from warehouse_barcode_scan.definitions import Warehouse, BarcodeController, ScanLabels

request = sh.RunRequest(
    environment=Warehouse(),
    policy=BarcodeController(),
    task=ScanLabels(),
    seed=0,
    limits=sh.RunLimits(max_steps=960, timeout_seconds=1800, gpus=1),
    outputs=sh.Outputs(cameras=()),
)

with sh.connect() as sim:
    plan = sim.plan(request, project="warehouse_barcode_scan")
    print(plan.describe())

    run = sim.run(
        request,
        project="warehouse_barcode_scan",
        request_id="warehouse-first-run-1",
    )
    run.save(Path("warehouse-run.json"))
    run.wait(outputs=True)

    print("Run:", run.id)
    print("Evaluation:", run.evaluation)
    for artifact in run.artifacts:
        print(artifact.id, artifact.path)
```

```bash
python run_warehouse.py
```

This submits real hosted work. The SDK validates the request, packages the
source, and selects compatible hosted runtimes. The backend builds the scene,
executes the policy, and records the result. Your machine does not need MuJoCo,
Blender, a GPU, or an operator account to submit it. The example requires a
deployment that advertises the corresponding typed scene and runtime variation
capabilities; the client reports missing capabilities before submission.

`sim.plan(...)` reads source files and the hosted catalog without uploading or
starting jobs. It describes stages and limits; it is not a price quote or a
capacity reservation. `sim.run(...)` submits the work. The saved receipt lets
you resume monitoring after closing the terminal.

Open the [SimHub console](https://simhub-console.pages.dev/), select the same
organization as your API key, and open **Runs** to follow the submitted run.
Its run page shows execution status, task evaluation, metrics, and available
recordings. SDK submissions appear there automatically; no separate result
upload is needed. The **Scenes** view contains stored scene previews and
snapshots when the project produces them.

The first run above requests no extra recording cameras or cinematic film.
To request a film, replace its `outputs` with:

```python
outputs = sh.Outputs(video=sh.VideoOutput())
```

Use this value in a new `RunRequest` and a new request ID. The default cinematic
is 1920×1080, 60 fps, 20 seconds, and 24 samples, with overlays enabled. Film
delivery also requires a compatible hosted renderer and takes additional work.
After `run.wait(outputs=True)`, `run.video`, `run.poster`, and `run.provenance`
identify the delivered artifacts. Ordinary artifact URLs require authentication;
use the session's client to download their bytes:

```python
# Inside the open session, after waiting for a run that requested a film:
video = run.video
if video is not None:
    Path("warehouse.mp4").write_bytes(sim.client.download(video.id))
```

Execution and task success are separate. `run.wait()` confirms execution and
requested delivery; inspect `run.evaluation.outcome` and
`run.evaluation.checked` to determine whether the stored task passed. A policy
can finish normally and fail to scan the labels.

See the [warehouse source and assumptions](https://github.com/General-Deployment/sim/tree/main/examples/warehouse_barcode_scan)
for the scene, controller, evaluation, and cinematic hooks.

## Author a project

### Project layout and dependencies

A typed project is a Python package directory. For example:

```text
my_project/
    __init__.py
    simhub.toml
    parameters.py     # Validated environment and policy inputs
    definitions.py    # Public environment, policy, and task contracts
    runtime.py        # Executable environment and policy implementations
    recipe.py         # Optional scene construction
    parts.py          # Optional reusable scene components
    assets/           # Meshes, textures, and other input files
run_project.py        # Submission script, beside the package
```

Run `python run_project.py` from the directory containing `my_project`, import
from `my_project.definitions`, and pass `project="my_project"` to `sim.run`.
The source root must have a valid Python package name, such as `my_project`.
The manifest's display name may contain hyphens.

Put `simhub.toml` **inside** the package directory:

```toml
name = "my-project"
image = "python-gpu"
include = ["*.py", "assets/**"]
```

- `image` selects a runtime offered by the deployment, commonly `python-cpu`
  or `python-gpu`.
- `include` controls which source and asset files enter the bundle. Add patterns
  for nonstandard inputs such as `*.blend`. Files outside the bundle are not
  available to hosted code unless resolved through a supported asset mechanism.

Bundles are deterministic and content-addressed. Keep heavy imports and model
loading inside runtime construction or factories so that importing definitions
remains lightweight. Use paths relative to `__file__` or the supplied runtime
context instead of absolute paths from your laptop.

For projects that need libraries beyond the standard hosted runtime,
`requirements` is an optional list of additional simulation dependencies.
`scene_requirements` optionally supplies the complete list of additional
packages for scene compilation and previews; omitting it uses `requirements`,
while `[]` adds no scene packages. Keep policy-only libraries out of the scene
list. Configure these once in the project rather than reconstructing dependency
lists for each run. The hosted runtime owns the supported MuJoCo version.

### Typed definitions

Subclass `sh.Parameters` to declare inputs with types and Pydantic constraints.
For example, in `my_project/parameters.py`:

```python
from pydantic import Field
import simhub as sh

class SceneParameters(sh.Parameters):
    target_count: int = Field(default=3, ge=1, le=10)
    spacing_m: float = Field(default=0.5, gt=0)

class ControllerParameters(sh.Parameters):
    speed: float = Field(default=0.2, gt=0, le=1)
```

Definitions describe the runtime's interface. The following is a contract
example for `my_project/definitions.py`; it requires the matching runtime
implementations described below before it can be submitted:

```python
import simhub as sh
from .parameters import SceneParameters, ControllerParameters

STATE = sh.ArrayDefinition(name="state", shape=(6,), dtype="float32", unit="rad")
ACTION = sh.ArrayDefinition(name="action", shape=(6,), dtype="float32", unit="rad")

environment = sh.EnvironmentDefinition[SceneParameters](
    name="my-environment",
    parameters=SceneParameters(),
    runtime=sh.SourceRef(entrypoint="my_project.runtime:MyEnvironment"),
    observations=(STATE,),
    action=ACTION,
    metrics=(sh.MetricDefinition(name="targets_reached"),),
    control_hz=30,
)
policy = sh.PolicyDefinition[ControllerParameters](
    name="my-controller",
    parameters=ControllerParameters(),
    runtime=sh.SourceRef(entrypoint="my_project.runtime:MyController"),
    observations=(STATE,),
    action=ACTION,
    control_hz=30,
)
task = sh.TaskDefinition(
    name="reach-all-targets",
    spec=sh.ConditionsSpec(conditions=(
        sh.MetricPredicate(metric="targets_reached", op="gte", value=3),
    )),
)
request = sh.RunRequest(environment=environment, policy=policy, task=task)
```

`RunRequest` validates that the action schemas and control rates match, the
policy's observations exist with the declared shapes, dtypes, and units, and
the task references declared metrics. Supported predicate operators are `lt`,
`lte`, `gt`, `gte`, `eq`, and `ne`. Use `StagesSpec` with `StagePredicate` for
ordered progress or `ScoreSpec` for a threshold on one metric.

Contracts reject unknown fields and nonfinite numbers and are immutable.
Construct a new value to change inputs. `model_dump_json()` serializes a
contract; `sh.export_schema(type(value))` exposes its JSON Schema. Preserve
concrete parameter types when loading serialized definitions or experiments.

`SourceRef` is a `package.module:ClassOrFactory` reference to code in the bundle.
`SourceRef.of(MyClass)` also works for importable classes; classes defined in
`__main__` or nested inside a function cannot be used that way. A typed
`EnvironmentDefinition` describes an environment; it is not the environment
instance that steps physics.

### Runtime contracts

For typed projects, the environment constructor or factory accepts
`parameters: SceneParameters` and `context: sh.RuntimeContext` as keyword
arguments. The policy accepts `parameters: ControllerParameters`,
`context: sh.RuntimeContext`, and `environment`. Annotate `parameters` with its
concrete `Parameters` subclass: the hosted runtime uses that annotation to
validate the input. Model-backed typed policies also accept `model` when a
model binding is declared.

An environment implements:

- `spec: sh.EnvSpec`: its action width, observation schemas, control rate,
  episode limit, cameras, and metrics. The identifier field is `id`.
- `reset(seed) -> dict`: reset episode state and return initial observations.
- `step(action) -> (dict, sh.StepInfo)`: advance one control tick. This is a
  two-item tuple; it is not the Gymnasium five-item return value.
- `observe(keys) -> dict`: provide the requested observations and render only
  requested image keys.
- `score() -> dict`: return the declared metrics. Missing or nonfinite required
  metrics prevent a clean result.
- `done`: a boolean property indicating that the episode has ended.

`sh.Environment` supplies `spec` from your `describe()` method, a step counter,
and a default `done` based on `spec.max_steps`. Your implementation resets
`self._steps` in `reset()` and increments it in `step()`. Initialize fields used
by `describe()` or `done` before calling `super().__init__()`.

A policy implements `spec: sh.PolicySpec`, `reset()`, and `act(obs) -> action`.
Use `sh.PolicyBase` for a stateless default `reset`, and override it to clear
stateful controllers between episodes. `PolicySpec.expects` names the
observations the loop requests. Implement `setup(context)` and `close()` when
resources need initialization or cleanup. Both base classes provide these hooks.

`StepInfo(progress=..., metrics=..., state=..., done=...)` describes an
individual step. Progress and `done` do not themselves prove task success;
the task evaluates the recorded measurements. Keep runtime specs consistent
with their typed definitions: the worker checks them during construction.

`RuntimeContext` supplies `source_root`, `output_root`, an optional compiled
`scene_path`, and deterministic seed streams. Use `context.seeds` or
`context.rng("sensor_noise", namespace="wrist")` for independent random streams.
Seed third-party simulators and frameworks explicitly.

For existing engines, use `sh.MujocoEnvironment` or adapt a Gymnasium environment
with `sh.GymnasiumAdapter`. The
[Gymnasium adapter guide](https://github.com/General-Deployment/sim/blob/main/docs/SDK_TYPED_PARAMETERS.md#adapting-an-existing-gymnasium-environment)
and [warehouse runtime](https://github.com/General-Deployment/sim/blob/main/examples/warehouse_barcode_scan/runtime.py)
show working integrations.

### Scene recipes, native engines, and assets

Set an environment's `recipe=sh.SourceRef(...)` when it needs a separate scene
compilation stage. An `EnvironmentRecipe` composes `Component` objects through
a `SceneBuilder`. MuJoCo and Blender components can supply native programs or
files while keeping their public inputs typed. The runtime consumes the compiled
scene through `context.scene_path`.

Start with the [native component guide](https://github.com/General-Deployment/sim/blob/main/docs/SDK_NATIVE_COMPONENTS.md)
and [warehouse recipe](https://github.com/General-Deployment/sim/blob/main/examples/warehouse_barcode_scan/recipe.py).
`ObjectSlot`, `ComponentInterface`, `SlotSelection`, and `InstanceOptions` let
components expose compatible joints, frames, materials, and instance settings.

For reusable robot or mesh files, upload a named asset:

```bash
simhub asset upload robot.urdf --env robots --name arm
```

Resolve declared assets in project code with `sh.asset("arm").path`. URDF
references need their meshes and other dependencies packaged too. Follow
[named assets](https://github.com/General-Deployment/sim/blob/main/simhub/docs/named-assets.md)
for aliases and version pinning, and
[asset metadata](https://github.com/General-Deployment/sim/blob/main/docs/SDK_ASSET_METADATA.md)
for links, joints, materials, and semantic roles.

Blender-authored factory projects can provide `write_mjcf(out, seed=0, **kwargs)`.
The `simhub.blender` helpers mark static objects, dynamic bodies, collision
proxies, and excluded visual geometry before export. Include `.blend` files in
the manifest if the scene is authored in Blender's UI. See
[the procedural Blender example](https://github.com/General-Deployment/sim/tree/main/examples/blender_depot)
and [the saved Blender scene example](https://github.com/General-Deployment/sim/tree/main/examples/blend_file_dock).
MuJoCo collision geometry is convex; use separate parts or proxies where a
single hull would fill an opening.

## Submit a complete single-file project

This small environment moves a number toward a seeded target. It demonstrates
the complete environment, policy, task, and hosted submission using only the
base SDK and your API key. Save it as `counter.py`:

```python
import numpy as np
import simhub as sh

sh.manifest("counter-demo", image="python-cpu")

class Counter(sh.Environment):
    def __init__(self):
        self.position = 0.0
        self.target = 1.0
        super().__init__()

    def describe(self):
        return sh.EnvSpec(
            id="counter", action_dim=1, control_hz=10, max_steps=20,
            observation={"state": sh.ArraySpec(shape=(2,))},
            randomised=True,
            metrics={"reached": sh.MetricSpec(required=True)},
        )

    def reset(self, seed=0):
        self._steps = 0
        self.position = 0.0
        self.target = float(np.random.default_rng(seed).integers(3, 8))
        return self.observe(("state",))

    def observe(self, keys):
        if "state" not in keys:
            return {}
        return {"state": np.array([self.position, self.target], dtype="float32")}

    def step(self, action):
        self.position += float(np.clip(action[0], 0, 1))
        self._steps += 1
        return self.observe(("state",)), sh.StepInfo(done=self.done)

    @property
    def done(self):
        return self.position >= self.target or super().done

    def score(self):
        return {"reached": float(self.position >= self.target)}

class Advance(sh.PolicyBase):
    def __init__(self, env):
        super().__init__(sh.PolicySpec(
            action_dim=1, control_hz=env.spec.control_hz, expects=("state",),
        ))

    def act(self, obs):
        position, target = obs["state"]
        return np.array([min(1.0, max(0.0, target - position))], dtype="float32")

@sh.environment("counter")
def counter():
    return Counter()

@sh.policy("advance")
def advance(env):
    return Advance(env)
```

Save the submission script beside it as `submit_counter.py`. The simulation
source stays in `counter.py`; this script loads and submits it:

```python
import simhub as sh

with sh.connect() as sim:
    project = sim.project("counter.py")
    batch = project.launch(
        env="counter", policies=["advance"], seeds=(3,),
        task=sh.Task(reached="eq:1"), record=sh.Record(), gpu=False,
        request_id="counter-first-run-1",
    )
    print("Batch:", batch.id)
    print("Open https://simhub-console.pages.dev/ and select Runs.")
    batch.wait()
    for run in batch.runs:
        print(run.id, run.state, run.outcome, run.metrics)
```

```bash
python submit_counter.py
```

This uploads the single-file project and runs one episode on a hosted CPU.
Find it in the console under the `counter-demo` environment. It should report
task success with `reached=1.0`; its metrics, evaluation, and result are stored
on SimHub. It has no camera or physics body, so it produces no video. Reuse the
request ID only when recovering the same submission; choose a new ID when you
intend to run changed code or another episode.

A larger factory project uses the same decorators in its package's
`__init__.py` and the same `simhub.toml` layout described above.
`sh.Project.load(path)` imports a factory project to discover its registrations;
typed submission collects source using declarations without importing its
runtime code.

## Compare policies and vary environments

### Hosted batches for factory projects

To compare the counter's policy against a baseline, submit a hosted batch from
the directory containing `counter.py`, with an API key configured:

```python
import simhub as sh

with sh.connect() as sim:
    project = sim.project("counter.py")
    batch = project.launch(
        env="counter",
        policies=["advance", "simhub:hold"],
        seeds=range(8),
        task=sh.Task(reached="eq:1"),
        record=sh.Record(),
        gpu=False,
    )
    print("Batch:", batch.id)
    batch.wait()
    print(batch.report().table())
```

This creates 16 episodes: two policies × eight seeds. `simhub:hold` is a built-in
baseline. Install the `notebook` extra for DataFrame tables and plots. Inside an
open session, `batch.dataframe()`, `batch.videos()`, and `batch.plot.success()`
help inspect recorded results; `batch.cancel()` requests cancellation. Resume
an existing batch with `sim.batch("batch_...")`.

For a fixed benchmark, define a suite and evaluation, then use the same project:

```python
suite = sh.Suite.seeds("counter/v1", range(8))
evaluation = sh.Eval(
    suite=suite, task=sh.Task(reached="eq:1"),
    primary="success", truncated_is="failure",
)
# Inside the session above:
batch = project.evaluate(
    evaluation, env="counter", policies=["advance", "simhub:hold"],
    record=sh.Record(), gpu=False,
)
report = batch.wait().report(evaluation)
print(report.table())
print(report.compare("advance", "simhub:hold"))
```

A suite freezes seeds and parameter overrides, and its content digest identifies
its version. Reports distinguish coverage, execution failures, checked task
verdicts, and runner claims. Comparisons are paired across the same suite and
include uncertainty. Decide explicitly how truncated episodes count.

### Typed experiments

For typed projects, use `Variations` and `ExperimentPlan` to validate and freeze
the entire comparison before submitting. From the `sim/examples` directory
used in the hosted quickstart:

```python
from pathlib import Path
import simhub as sh
from warehouse_barcode_scan.definitions import Warehouse, BarcodeController, ScanLabels
from warehouse_barcode_scan.parameters import WarehouseParameters

variations = sh.Variations.grid(
    WarehouseParameters,
    base=WarehouseParameters(width=320, height=240, verify_physics=False),
    axes={"carton_color": ("natural", "blue")},
)
plan = sh.ExperimentPlan.vary(
    Warehouse(), variations,
    policies=(BarcodeController(),), task=ScanLabels(), seeds=(0, 1),
    limits=sh.RunLimits(max_steps=960, timeout_seconds=300, gpus=1),
    outputs=sh.Outputs(cameras=()),
    budget=sh.ExperimentBudget(
        max_runs=4, max_concurrent_jobs=2, max_gpus=1,
        timeout_seconds=3600, max_gpu_seconds=3600,
    ),
)
print(plan.describe())  # Two colors × two replicates × one policy = four episodes

with sh.connect() as sim:
    experiment = sim.experiment(
        plan, project="warehouse_barcode_scan", request_id="color-comparison-1",
    )
    experiment.save(Path("experiment.json"))
    experiment.wait()
    print(experiment.report(group_by=("policy", "carton_color")))
```

`Variations.grid` takes a cross product; `zip` pairs axis entries; `cases`
accepts explicit parameter values; `sample` draws bounded random cases.
Portable recipes add serializable distributions, correlated choices, and
constraints. Rejected cases are recorded, and an unsuccessful bounded sample
does not silently submit a smaller experiment.

`RunLimits` bounds an episode's steps and stage resources. `ExperimentBudget`
separately bounds the number of episodes, concurrent jobs, GPUs, elapsed time
including queueing, and reserved GPU seconds. Compilation, simulation,
rendering, and retries consume the applicable budget. Tenant and fleet limits
still apply. A plan requires a backend that can enforce its limits.

Parameters are treated as compilation inputs by default. Reusing a compiled
scene across reset-only changes requires explicit `ParameterEffect`
declarations, runtime ABI 2, and a `VariationRuntime.apply_variation` adapter.
See [typed experiments](https://github.com/General-Deployment/sim/blob/main/docs/SDK_TYPED_EXPERIMENTS.md)
and [portable variation recipes](https://github.com/General-Deployment/sim/blob/main/docs/SDK_VARIATION_RECIPES.md)
for budget accounting, deterministic seed derivation, and reuse rules.

## Recordings, rendering, and datasets

Recording is explicit. Factory runs use `sh.Record(cameras=(...), video="hero")`;
typed requests use `sh.Outputs(cameras=(...), video=sh.VideoOutput(...))`.
Camera names must exist in the environment. `PolicySpec.expects` controls what
the policy observes, while recording options can request additional cameras.
Use `Outputs(cameras=())` to omit additional recording cameras; the default is
`("hero",)`. A policy's required image observations still have to be rendered.

`sh.Record(video="hero")` defaults to **60 fps** for environments with
`step_with_substeps(action, on_substep)`. `MujocoEnvironment` provides this hook
when using its built-in `step`; subclasses that override `step` must provide
their own hook. It runs the normal control step and calls `on_substep(fraction)`
after each physics substep, ending at `1`. The recorder renders only when an
output frame is due, using `render(camera)` at the current physics state.
Custom renderers must not reuse a frame cached for the whole control step.
Use `sh.Record(video="hero", fps=30)` to choose another presentation rate
between 1 and 60 fps. Frames are labeled with the completed control step's
progress and state, including its terminal status.
The smooth presentation is saved as `film/video.mp4` and selected as the run's
primary video. Policy-camera recordings retain the control rate and the exact
images the policy saw; trajectory sampling and policy calls are unchanged.
Older environments without the hook keep their original control-rate video
and log that limitation. Typed cinematic requests also default to 60 fps;
use `sh.VideoOutput(fps=30)` to choose another rate.

Depending on the environment and requested outputs, artifacts include:

```text
metrics.json                 Final measurements
result.json                  Local episode result
evaluation.json              Metric metadata and local task evaluation
trajectory/meta.json         Recording schema, body names, and units
trajectory/000000.npz         Chunks of recorded state, actions, and poses
video/hero.mp4               Direct camera recording, when requested
renders/<camera>/*.png        Camera frames, when a frame sink is selected
checkpoints/*.npz            Checkpoints, when supported and enabled
film/video.mp4               Smooth presentation or hosted cinematic output
film/poster.png              Hosted cinematic poster
film/render-manifest.json    Cinematic provenance
```

Not every environment produces every file. Body trajectories require recorder
bindings; checkpoints require environment support. `Recorder.channel(name,
sampler)` adds fixed-shape finite numeric fields. Pose quaternions use **wxyz**,
and the trajectory metadata records the convention. Direct camera recordings
preserve what the policy saw; cinematic rendering replays the saved motion.
The hosted run's stored outcome and checked flag remain authoritative over a
local evaluation sidecar.

After simulation, render a different film without running the policy again:

```python
from pathlib import Path
import simhub as sh

with sh.connect() as sim:
    run = sh.RunHandle.resume(sim, Path("warehouse-run.json"))
    render = run.render(
        sh.VideoOutput(width=1280, height=720, fps=24, seconds=12),
        request_id="warehouse-film-2",
    )
    render.save(Path("warehouse-render.json"))
    render.wait()
    print(run.video.url)
```

The recording must contain the scene and trajectory inputs required by the
renderer. A render cannot add observations that were never recorded.
Presentation projects can customize cameras, materials, overlays, and
composition; see [hosted rendering and recovery](https://github.com/General-Deployment/sim/blob/main/docs/SDK_HOSTED_RECOVERY.md).

For dataset export, install `simhub[dataset]` and use
`simhub.datasets.from_trajectories(...)` for local `trajectory/` directories or
`simhub.datasets.lerobot(batch, ...)` for a completed batch. The current exporter
writes LeRobot v2.1 state/action data and requires recorded `qpos` and `action`
channels. Batch export includes complete runs regardless of task success and
does not copy camera video files. Select appropriate episodes, match the
recorded control rate, and validate against your training loader. See the
[dataset export guide](https://github.com/General-Deployment/sim/blob/main/simhub/skills/simhub-datasets/SKILL.md)
for the complete example and current limits.

## Learned policies and model credentials

`sh.OpenPiPolicy` manages a connection to an OpenPI server and executes returned
action chunks. Subclass it, declare `expects`, and implement
`encode_observation(obs)`. Override `decode_action(action, obs)` or
`decode_chunk(actions, inference_obs)` when the checkpoint's output needs
conversion. Match camera keys, joint ordering, gripper units, action convention,
and control rate to the checkpoint.

The standard hosted Python runtimes include OpenPI's client dependencies.
For hosted factory runs, pass the endpoint through `policy_kwargs` to your
factory; it must be reachable from the worker. Install `simhub[openpi]` only
when using the client locally. See the
[OpenPI policy guide](https://github.com/General-Deployment/sim/blob/main/simhub/docs/openpi-policies.md)
for a full adapter, authentication, timeouts, and relative actions.

For private Hugging Face checkpoints:

```bash
simhub model credentials set my-hf-token
simhub model register --help
```

The first command reads the token through a hidden prompt. Register a pinned
policy revision bound to that credential, then select it with
`project.launch(policy_version_id=...)`. Typed policies use `HuggingFaceModel`
with a full 40-character revision and a credential ID; token values are never
part of the definition. See
[hosted model credentials](https://github.com/General-Deployment/sim/blob/main/simhub/MODEL_CREDENTIALS.md).

### Google Cloud Storage models

GCS is a checkpoint storage source; policies still execute on SimHub's
Lambda/Fleet compute backend. Use SDK, API, worker, and runner images from the
release containing `GCSModel` and `simhub model resolve`. Older 0.2.0 builds may
lack these APIs; check `simhub model resolve --help`. The submission SDK needs
only its normal dependencies. Google libraries run in the trusted API/worker,
and neither `gcloud` nor `gsutil` is required to resolve or download models.
See the [server setup](README.md#gcs-model-source-deployment) for deployment
prerequisites. Legacy Modal and other backends without trusted staging reject
GCS submissions before execution.

Resolve a public checkpoint into a frozen manifest:

```bash
simhub model resolve \
  gs://gdm-robotics-open-x-embodiment/open_x_embodiment_and_rt_x_oss/rt_1_tf_trained_for_000400120/ \
  --recursive --layout files --out models/rt1-converged.json
```

`--recursive` requires a nonempty prefix ending in `/`. Without it, the URI
names exactly one object. Resolution reads metadata, not weights. The resulting
JSON records the bucket, object names, exact generation strings, byte counts,
CRC32C checksums, relative destination paths, layout, and model root. Commit
this manifest with your experiment. Running it always requests those generations;
if a version disappears, the run fails instead of switching to newer weights.
Use completed release prefixes: listing is not an atomic snapshot of a directory
that a publisher is still uploading. Long-term reproducibility also requires
retaining the pinned objects or mirroring them to a controlled bucket.

For private buckets, obtain a service-account JSON key with permission to read
the intended objects. Exact-object resolution and downloads need
`storage.objects.get`; prefix expansion also needs `storage.objects.list`.
A bucket-scoped Storage Object Viewer role supplies both. Import the key once
through a file or stdin; never put key contents in your source or a command-line
argument:

```bash
simhub model credentials set research-gcs \
  --provider gcs --service-account-file /secure/research-reader.json
simhub model credentials list
# Replace gcsc_returnedid with the opaque ID returned above.
simhub model resolve gs://my-policy-bucket/releases/policy.zip \
  --layout zip --root policy --credential gcsc_returnedid \
  --out models/private-policy.json
```

The imported key is encrypted and scoped to your organization. The manifest
contains only the credential revision ID. The trusted worker uses it to stage
files; policy code receives no Google key or access token. Omitting
`--credential` means anonymous access, even when your laptop has a Google login.
The initial private-access format is a `service_account` JSON key with Google's
standard OAuth token endpoint. Federation, signed URLs, Requester Pays billing,
and custom storage endpoints are not supported.

To rotate access, import a new key revision, resolve a manifest using its ID,
and register a new policy revision. Revoke old access with
`simhub model credentials revoke gcsc_oldid`. Revocation prevents new staging
and is checked again after downloads; it cannot retract bytes from an already
running policy. Do not delete an old object generation if experiments still
need it.

For buckets where listing is disallowed, pass a JSON array of exact names in
`--objects-file`, with `gs://bucket` as the URI. Names are retained as relative
destinations, so `--root` can select their shared model directory:

```bash
simhub model resolve gs://my-policy-bucket \
  --objects-file checkpoint-objects.json --root releases/policy \
  --credential gcsc_returnedid --out models/private-policy.json
```

Declare the resolved manifest in an existing typed project. Here `environment`, `task`,
`ControllerParameters`, and `Controller` are your project definitions:

```python
from pathlib import Path
import simhub as sh

model = sh.GCSModel.model_validate_json(
    Path("models/private-policy.json").read_text()
)
policy = sh.PolicyDefinition(
    name="gcs-policy",
    parameters=ControllerParameters(),
    runtime=sh.SourceRef.of(Controller),
    observations=environment.observations,
    action=environment.action,
    control_hz=environment.control_hz,
    model=model,
)
# Use this policy in your RunRequest; session.run registers the binding.
request = sh.RunRequest(environment=environment, policy=policy, task=task)
with sh.connect() as session:
    run = session.run(request, project=Path("my_project"), request_id="gcs-trial-001")
```

Include a manifest loaded by project code in the source bundle's `include`
patterns. Resolve paths relative to the module when reading inside your project,
for example `Path(__file__).parent / "models/private-policy.json"`. Project
inventory embeds the resolved source in `PolicyDefinition.model`; no separate
TOML model registry is needed. Current hosted model policies must declare
`observation/state` in addition to any image observations.

Your typed policy constructor receives `model: sh.GCSModel` and a
`RuntimeContext` whose `model_path` is the verified local model directory. Load
TensorFlow, PyTorch, JAX, or other formats from that path using your policy's
own loader. Decorator-based factories receive `model` as a contract dictionary
and `model_path` as a local path string. Both inputs are bound to the registered
revision and cannot be overridden through `policy_kwargs`.

For an existing decorator-based policy, register and launch explicitly:

```bash
simhub model register gcs-policy --contract models/private-policy.json \
  --project-policy gcs-policy --observation-dim 8 --action-dim 7 --control-hz 3
```

```python
# project is your existing HostedProject; use the version ID returned above.
batch = project.launch(policy_version_id="pvr_returnedid")
```

Python clients can use `session.client.resolve_model_source(...)`,
`set_gcs_credential(name, service_account_dict)`, and
`register_model_policy(...)` for the same workflow. Existing
`register_hf_policy(...)`, `HuggingFaceModel`, HF dependencies, and HF credential
commands retain their behavior and policy digests.

`layout="files"` preserves the manifest's relative paths. `layout="zip"`
requires one pinned ZIP, verifies its bytes before extraction, and uses `root`
to select the directory inside it. The default root is `.`. Inputs are mounted
read-only and staged separately for each run; there is no shared download cache.
Limits are 1 MiB of contract JSON, 1,024 selected objects, 20 GiB downloaded,
20 GiB extracted, 10,000 ZIP entries, and 16 MiB of ZIP directory metadata.
ZIP links, traversal, encryption,
special files, and colliding destinations are rejected. Disk headroom and a
bounded staging deadline also apply.

Common setup failures:

- **Access denied:** check the credential revision, bucket permissions, and
  whether a prefix requires list permission. Ambient Google credentials are
  intentionally ignored.
- **Generation unavailable:** restore the pinned version or explicitly resolve
  and register a new release. A retry never chooses the latest bytes.
- **Verification/layout failure:** check object checksums and sizes, safe file
  paths, ZIP contents, limits, and the selected root directory.
- **Capability unavailable:** deploy matching Fleet API/worker/runner and node
  configuration. Google libraries belong in the trusted services, while model
  inference dependencies must be baked into the offline policy image.

[SIMPLER example manifests](examples/gcs_models/README.md) pin the three RT-1
training-stage checkpoints and RT-1-X ZIP. Keep Octo on a pinned HF revision and
record all model digests with ranking results. These declarations download
checkpoints; RT-1/Octo adapters, language encoders/tokenizers, the Google Robot
environment, and observation/action parity are separate requirements before
comparing policy ordering. Auxiliary assets must be in the declared bundle or
pinned inference image; GCS models currently contain one file bundle or archive.

## Debug and recover runs

Persist a request ID **before** submitting if your application must recover
from a lost response. Reuse the same ID and identical inputs after
`SubmissionUncertain`; changing the ID requests new work. Use a new ID for an
intentional change to source, parameters, or outputs. Save the returned receipt
immediately. Receipts contain IDs and the API address, not an API key.

```python
from pathlib import Path
import simhub as sh

with sh.connect() as sim:
    run = sh.RunHandle.resume(sim, Path("warehouse-run.json"))
    for stage in run.stages:
        print(stage.kind, stage.state, stage.attempts, stage.job_id)
    try:
        run.wait(outputs=True, timeout=3600)
    except sh.HostedExecutionError as error:
        print(error.failure.model_dump_json(indent=2))
        print("Inspect job:", error.job_id)
```

`run.wait(outputs=False)` waits for simulation and required evaluation while
allowing a requested film to remain unfinished. `outputs=True` also requires
requested delivery. A client wait timeout does not cancel remote work. Resume
with the same API origin and credentials; keep the session open while using
its handles. `run.events()` yields typed execution events.

Use `run.cancel()` or `experiment.cancel()` to request cancellation and inspect
the resulting stage states. Cancellation acknowledgement alone does not prove
that all remote containers have stopped. A render receipt can be resumed with
`sh.RenderHandle.resume`; an experiment receipt with
`sh.ExperimentHandle.resume`.

Fetch saved logs by job ID or by environment:

```bash
simhub logs job_... --tail 100
simhub logs --env warehouse-barcode-scan --tail 100
simhub logs job_... --json > job-logs.json
```

```python
with sh.connect() as sim:
    logs = sim.job_logs("job_...", tail=100)
    for log in logs.logs:
        print(log.path, log.text)
```

`sim.environment_logs(name_or_id, tail=100)` provides the environment form in
Python. It selects the newest submitted job across all versions and job kinds,
including queued and running jobs. It does not fall back to an older job if the
newest has no saved logs. These are snapshots of retained logs and structured
errors, not a live stream. Logs commonly become available after an attempt ends.
`tail` trims each log after download.

Common starting points for a failure:

- **Import or missing-file errors:** check the package name, entry points,
  manifest `include` patterns, and runtime dependency lists.
- **Schema errors:** compare the environment and policy action, observation,
  and control-rate declarations; check task metric names.
- **Missing camera or output:** declare the camera and recorder bindings, or
  remove an output you do not need. Inspect the render stage separately from
  the simulation result.
- **Unsupported capability or budget refusal:** inspect the plan and deployment
  catalog. Installing a local extra does not add a hosted capability.
- **Authentication errors:** confirm the API address and that the key belongs
  to the intended deployment and organization.

### Job retries

New simulations default to three attempts. `max_attempts` accepts 1–10 when
submitting a job. Failures marked retryable use exponential backoff with jitter
(2.5–5 seconds initially, capped at five minutes). Invalid inputs and ambiguous
container launch failures stop immediately. Deployment admission pauses wait
without consuming an attempt, and expired worker leases cannot execute beyond
the attempt budget.

The job's `summary.retry` records `next_attempt_at` and the most recent 50 failed
attempts, including their timestamps and errors. The console shows this history
and retry timing on run pages. The jobs menu also lists recent failed jobs,
including compiles and failures before run creation, with the same recovery
controls. A pending retry clears the earlier verdict and metrics from the live
run; those remain available in that attempt's saved artifacts. Logs and artifacts from earlier attempts remain
available. A simulation waiting to retry is pending, and cancellation during
that wait stops delivery.

After inspecting a retryable failure, grant that job one additional attempt:

```python
import simhub as sh

with sh.connect() as sim:
    logs = sim.job_logs("job_...")
    job = sim.retry_job(logs.job.id, expected_attempt=logs.job.attempts)
```

The equivalent request is `POST /api/jobs/{job_id}/retry` with
`{"expected_attempt": 3}` for a job whose last observed attempt was 3. Repeating
the same request does not launch duplicate work or replenish the budget again,
even if its response was lost. Stale attempts and cancelled, successful, live,
or non-retryable jobs are rejected. An exhausted worker lease requires execution
reconciliation first. This retries one job; cancelled downstream jobs and
cancelled batches are not restarted.


## Pull, edit, preview, and share

To edit an environment already stored on the service:

```bash
simhub pull warehouse-pick ./warehouse_pick
# Edit the source in ./warehouse_pick.
simhub push ./warehouse_pick --message "Update the picking scene"
```

`push` publishes a source version to hosted SimHub, where it is available in the
console. To execute it as well, submit a typed request or use
`sim.project(...).launch(...)`; publishing source alone does not start a run.

`pull` accepts an environment name or ID. It downloads the latest version by
default; use `--version current`, a version number, or a version ID to select
another. The destination must be new or empty. Pull verifies checksums and
restores editable source and retained named assets without importing source or
installing dependencies. A compiled-only version cannot be pulled as editable
Python; an older source version is not silently substituted.

Keep `.simhub-package` and `.simhub-checkout.json` with the checkout. They retain
package identity, API origin, environment, and parent version. Push refuses a
stale checkout; pull into another directory and reapply edits. A concurrent
push may create a branch with `current: false`. Typed checkouts retain their
published definitions and parameters; submit a new typed request to change
those parameter values.

The Python equivalents are `sim.pull("warehouse-pick", into="warehouse_pick")`
and `sim.project("warehouse_pick").push("Update the picking scene")`.

For a factory project with an environment named `warehouse-pick`, publish an
initial scene preview without stepping its policy:

```bash
simhub launch ./warehouse_pick --preview --env warehouse-pick --seed 7
```

In Python, use `sim.project(path).preview(env="warehouse-pick", seed=7)`.
Preview generation is hosted compilation and requires a compatible GLB compiler.
It builds and resets the environment and can incur compute charges. The console
shows stored previews and run snapshots without starting another compile merely
because you open them.

Run pages are organization-scoped. When you want to give someone access without
an account, explicitly create a revocable, expiring public share:

```python
# Inside the session, with a completed run handle:
print(run.share(expires_in_days=7).url)
```

Treat that URL as access to the scoped results. Never put an organization API
key in a shared URL.

## Guidance for coding agents

Default to hosted submission and console-visible results. Use local simulation
only when the user explicitly requests local debugging. Do not replace a hosted
run with a local rollout when credentials or hosted capabilities are missing;
report the missing prerequisite.

Use this sequence when implementing an integration:

1. Check the installed SDK version and choose typed hosted definitions for a
   new hosted project, or preserve an existing factory project's API.
2. Define parameter models, observation/action shapes and units, control rate,
   required metrics, and task predicates before implementing the runtime.
3. Keep every referenced module and asset in the source bundle. Keep runtime
   construction out of definition imports. Use the hosted runtime's bundled
   dependencies and declare only additional project-specific libraries.
4. Validate definitions without starting a simulator, inspect `sim.plan(...)`
   for typed requests, and set explicit output and resource limits. Submit a
   small hosted run before expanding to a batch.
5. Persist a stable request ID, submit, and save the receipt. Resume existing
   work after a timeout or uncertain response instead of creating duplicates.
6. Direct the user to the run in the SimHub console. Report execution state,
   checked task outcome, metrics, and requested artifact delivery separately.
   Inspect failed stages and saved logs before retrying.
7. Freeze cases and seeds for comparisons. Use experiment budgets for typed
   sweeps, and record the source version, plan, and receipt with results.

The SDK includes **15 portable agent skills** covering setup, typed runs,
scene composition, variations, assets, policies, tasks, debugging, evaluation,
recording, Blender, and datasets. Installing the Python package does not
activate skills in an agent. Copy complete skill directories from the wheel or
repository into your agent's skill folder and reload the agent session. Follow
[installation instructions](https://github.com/General-Deployment/sim/blob/main/simhub/skills/README.md#install-for-your-agent)
for the copy commands. Start with `$simhub-typed-runs` for hosted work or
`$simhub-projects` for factory authoring.

For exact signatures and additional examples:

- [Typed contracts](https://github.com/General-Deployment/sim/blob/main/simhub/src/simhub/contracts.py)
  and [runtime base classes](https://github.com/General-Deployment/sim/blob/main/simhub/src/simhub/base.py).
- [Hosted plans, receipts, cancellation, and rerendering](https://github.com/General-Deployment/sim/blob/main/docs/SDK_HOSTED_RECOVERY.md).
- [Parameter schema generation](https://github.com/General-Deployment/sim/blob/main/docs/SDK_TYPED_PARAMETERS.md)
  from an immutable published version.
- [Agent skill index](https://github.com/General-Deployment/sim/blob/main/simhub/skills/README.md)
  with focused workflows and implementation limits.
- [Repository examples](https://github.com/General-Deployment/sim/tree/main/examples),
  including robotics, Blender sets, and the specialized
  [single-file ceiling simulation](https://github.com/General-Deployment/sim/blob/main/examples/asbestos_ceiling.py).

## Optional local debugging

Local execution is available when explicitly needed and requires the project's
engine dependencies on your machine. For factory projects, `simhub launch
PROJECT --check` runs scene checks and `simhub launch PROJECT --episode` runs a
local episode. In Python, `sh.check(...)` checks a scene and the top-level
`sh.run(...)` executes registered factories locally. These operations do not
publish runs to the console; use the hosted submission APIs above for that.
To enable `project.run_local(...)` through a session, explicitly opt in with
`sh.connect(hosted_only=False)`. The default connection rejects local execution,
including the deprecated `project.run(...)` alias.
The [local debugging guide](https://github.com/General-Deployment/sim/blob/main/simhub/skills/simhub-local-debugging/SKILL.md)
documents setup, registration, camera checks, and local output files.
