Metadata-Version: 2.4
Name: lekit-robot
Version: 0.1.7
Summary: Small Python SDK for observing and controlling Lekit Robot Node
Project-URL: Repository, https://github.com/sorelferris/lekit
Project-URL: Issues, https://github.com/sorelferris/lekit/issues
Author: Sorel Ferris
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: eclipse-zenoh<2,>=1.9
Requires-Dist: lz4<5,>=4.4
Requires-Dist: numpy<3,>=1.26
Requires-Dist: pillow<13,>=10
Requires-Dist: pydantic<3,>=2
Requires-Dist: rich<16,>=13.7
Requires-Dist: simplejpeg<2,>=1.9
Requires-Dist: typing-extensions<5,>=4.6
Requires-Dist: zstandard<1,>=0.23
Description-Content-Type: text/markdown

# lekit-robot

Small Python SDK for a running **Lekit Robot Node**. Discover a robot by name,
read joint/pose feedback and RGB/depth images, and send absolute motion targets.
The client does not install LeRobot, Torch, OpenCV, robot drivers, or camera drivers.
Python 3.10+ is required.

## Install

Install or upgrade:

```sh
pip install -U lekit-robot
```

From this repository:

```sh
pip install ./packages/lekit_robot
```

Robot Node must already be running on the robot computer. Clients need network
access to its Zenoh peer; the browser's HTTP port 8080 is not the SDK endpoint.
LAN multicast discovery is the default. If discovery is blocked, configure both
Node (`lekit-robot --zenoh-router tcp/ROUTER_IP:7447`) and client
(`connect(name="NERO Dual", router="tcp/ROUTER_IP:7447")`) to use an existing Zenoh router.
This package installs the client CLI, not the Robot Node server. In a full
`lekit` installation, `lekit-robot-node` starts the server; the former
`lekit-robot --host ... --port ...` invocation remains compatible.

## Continuous observation test

```bash
lekit-robot nero-dual --observation
# Exact advertised name; 30 Hz RGB-only observations over an explicit Zenoh endpoint:
lekit-robot "NERO Dual" --observation --router tcp/192.168.3.241:7447 --hz 30 --mode rgb
# State only, or restrict the selected cameras:
lekit-robot "NERO Dual" --observation --mode robot
lekit-robot "NERO Dual" --observation --camera head_view --camera left_view
```

Run this on the **policy/client computer** to include its network path. It calls
the real `observation()` API using persistent subscriptions, without acquiring
control or sending actions. The Rich panel continuously reports achieved return
frequency, call P50/P95/P99, startup, errors, repeated samples, period overruns,
estimated source ages at return (latest and P95), source ages at T, skew,
decode/cache metrics, CPU and memory. CPU 100% means one
core; RSS is available on Linux. Measurement includes diagnostic overhead.

Default: 60 Hz, RGB and enabled depth from all cameras, runs until **Ctrl+C**.
Use `--hz`, `--mode rgbd|rgb|encoded|robot`, `--camera NAME` (repeatable),
`--timeout .2` (selection wait seconds; decoding can extend total call time),
`--connect-timeout 3`, or optional `--count 300` for a bounded run. The panel
refreshes twice per second; rates/percentiles cover a bounded three-second window.
First successful call is shown separately from steady-state latency; counts are
cumulative. Ctrl+C/SIGTERM closes the client and retains the final panel.

The positional name first matches the advertised name (case insensitive). If not
found, a hyphenated CLI name such as `nero-dual` retries as `nero dual`. Ambiguous
names are rejected. `--obervation` is also accepted as a compatibility spelling.
`python -m lekit_robot ...` is equivalent to the console command.

The **observation freshness** table distinguishes two ages:

- **Return age ≈**: client UTC at `observation()` return minus the selected sample's
  Node capture UTC. It includes the default 50 ms lookback and transport/decode
  waiting. The adjacent P95 covers successful observations in the last three seconds.
  Reusing a sample increases its return age.
- **Age at T**: alignment target T minus sample time in the Node clock domain.
  It excludes the default lookback and is not capture-to-return latency.

Return age is an **estimate requiring NTP/PTP synchronization** between client
and Node; this tool does not calibrate clocks. Clock offset affects even positive
values. Missing/timezone-naive capture times display `—`; negative ages display
`时钟偏差` and are excluded from percentiles. Cross-host monotonic clocks are never
subtracted.
Camera ages describe frame groups; robot ages describe Node snapshots, not each
underlying CAN message. On observation failures the table is labeled as the last
successful observation. Node loss terminates the test; rerun after reconnecting.

The observation test command is available in lekit-robot 0.1.6 and later.
Return-age freshness metrics require 0.1.7 or later.

## Three primary interfaces

Version 0.1.6 provides `observation()`, `send_action()`, and `meta_info` in the
existing package. `send_action()` requires
the matching Robot Node `target-v1` extension; there is no silent fallback to the
older fire-and-forget stream. Existing public methods remain available.

```python
from lekit_robot import connect

with connect(name="NERO Dual") as robot:
    meta = robot.meta_info                   # cached; no network query here
    obs = robot.observation()                # robot + all configured camera streams
    print(obs.robot.state, obs.quality)
    print(meta.manifest.joint_target.fields) # ordered names, units, bounds
    print(meta.calibration)                  # None if no calibration is available

    # Other selections, using the same persistent subscriptions:
    state = robot.observation(cameras=(), images=False)
    images = robot.observation(cameras=["head_view"], robot=False, depth=False)
```

### Observation contract

- Default: decoded RGB and enabled depth, robot state, all advertised cameras,
  `strict=True`. `images="encoded"` retains compressed frames; `images=False`
  returns sample metadata without decoding; `depth=False` omits depth payloads.
- The default alignment target is the latest received Node timeline minus 50 ms.
  Each source contributes its newest capture **at or before** that target. This
  small alignment delay allows independently delivered metadata and images to meet.
  Robot timestamps describe the SDK snapshot read, not simultaneous acquisition
  of every CAN feedback field. Alignment is not hardware synchronization.
- `at=T` uses seconds in the connected **Node monotonic clock**, not client
  `time.monotonic()` or Unix time. For example, use a prior observation's
  `aligned_at_monotonic_ns / 1e9`. Only already buffered history can be selected.
- Defaults: age at T <= 250 ms, source timestamp skew <= 50 ms. Override with
  `max_age={"robot": .1, "camera": .2, "head_view": .15}` and `max_skew=.03`.
  Local time since the source last arrived also detects a stalled subscription.
- Missing/stale sources, mismatched image payloads, excessive skew, and decoding
  failures raise `ObservationUnavailableError` after the bounded wait (decode
  failures fail immediately). Its `.quality.reasons` explains the failure;
  `.quality` is `None` if no target could be established. `strict=False` permits
  partial/invalid results; always inspect `obs.quality.valid` in that mode.
- Subscriptions start on first observation request and are reused. History is
  bounded to 128 robot samples, 64 camera descriptors and, per image stream,
  64 encoded frames / 32 MiB (whichever limit is reached first). Close releases
  subscriptions. Reconnect after robot/camera generation or Node identity changes.

### RGBD decode performance

Decoded observations use up to three reusable worker threads per connection.
After a decoded observation, newly received frames from its requested streams
are predecoded in the background for up to 250 ms after the latest request.
There are at most three background jobs and one pending frame per active stream;
new arrivals replace pending work. `images=False`, `images="encoded"` and
robot-only requests stop prefetch interest (already running jobs may finish).
JPEG decoding uses `simplejpeg` directly into RGB arrays, with full-quality DCT
and upsampling. Pillow remains the PNG/CMYK JPEG and compatibility path. Depth
keeps its lossless LZ4/Zstandard codecs and uint16 values; tightly packed decoded
depth can use the decompressor's own writable buffer.

The decoder keeps a short history of decoded buffered frames, bounded to
96 entries / 64 MiB total (including retained compressed payloads). This lets
the existing 50 ms causal lookback select an already decoded frame. Identity
matching includes the exact buffered object: a new frame, generation or payload
does not reuse another frame's result. Arrays in
returned observations remain writable and independent: modifying one does not
alter another observation or the cache. Calls to the same decoder are serialized
so concurrent consumers cannot accumulate an unbounded worker queue. Selection
and quality checks run before looking in the decode cache. A miss waits for the
exact selected frame; it never substitutes an older decoded frame. Closing the
connection joins workers and releases cached frames. Background decoding reduces
call latency, not capture-to-client latency or the 50 ms alignment lookback.
The first call and cache misses can take longer than steady-state reads.

Read-only timing/resource check (does not send robot actions):

```bash
python examples/benchmark_observation.py --name "NERO Dual" --hz 60 --count 1800
```

The report separates startup from call latency, counts repeated samples and
deadline overruns, and reports source skew, sample age relative to the alignment
target, process CPU usage and Linux RSS. The age metric does not include the
50 ms lookback or network transit; it is not end-to-end latency.

`meta_info` contains the capability manifest, robot description when available,
camera configuration and current calibration bundle/reference. Connection loads
it, and observation delivery refreshes it when newer configuration/calibration
versions are detected. Property access returns a copy and performs no I/O.
Use an observation's calibration reference to identify historical calibration;
the metadata cache describes current configuration, not a historical frame.

### Unified actions

```python
# These functions send motion only when called with a connected robot.
def update_left_arm(robot, joint_targets, gripper_m):
    return robot.send_action({
        "left": {"joints": joint_targets, "gripper": gripper_m},
    }, timeout=.1)


def update_right_pose(robot, position_m, rpy_rad):
    return robot.send_action({
        "right": {"pose": {
            "position": position_m, "rpy": rpy_rad, "frame": "right.base",
        }},
    }, timeout=.1)
```

- Absolute targets only. Joint arrays follow advertised field order; mappings use
  exact advertised field names. Pose position uses metres and RPY uses radians in
  the selected arm's base (`left.base` / `right.base`, or `robot.base` for a single
  robot). Gripper units and limits come from the manifest. Omitted arms/grippers
  retain their own target until their own expiry. A single arm cannot specify
  joints and pose together. Node performs Cartesian IK using `nero-dual-ik` for NERO.
- `send_action()` is synchronous until **Node validation and driver dispatch have
  returned**, then returns an `ActionReceipt(status="accepted")`. It never waits
  for physical arrival. This is not an `async def` API. A receipt's
  `accepted_at_unix_ns` is a Node wall-clock timestamp.
- First send acquires control; a background thread renews the lease. Subsequent
  sends reuse it. Calls are serialized, one request in flight per client; there
  is no action chunk queue. A 60 Hz loop requires acceptance latency below its
  16.7 ms budget; no real-hardware 60 Hz guarantee has been established.
- `connect(target_ttl=.1)` requests a hold when an arm/gripper receives no update
  for 100 ms (range .05–2 s). Updates to another arm/gripper do not extend this
  deadline. `target_ttl=None` explicitly leaves targets active without a target
  timeout; abrupt client loss does not guarantee a hold in this mode. Graceful
  control release/close requests a hold while the lease is still valid.
- New absolute targets supersede the corresponding current targets. All parts
  are validated before dispatch, but physical writes across arms are not atomic.
  Lost replies or partial driver errors raise `ActionOutcomeUnknownError` with
  `.request_id`; **do not retry automatically**. A rejected request raises
  `ActionRejectedError` with `.code`. Acceptance is never arrival or stop confirmation.
- Node also checks request age using Unix clocks, so controller and Node clocks
  must be synchronized. A hold is subject to fresh feedback, generation and
  ownership checks; it is not a safety-rated emergency stop.

### Gradual migration

| Existing interface | Preferred entry point |
| --- | --- |
| `get_observation()` | `observation()` (strict by default) |
| `set_joints()`, `set_target()`, `set_gripper()` | `send_action()` |
| `descriptor`, current `get_calibration()` | `meta_info.manifest`, `meta_info.calibration` |

`observations()`, task lifecycle methods and historical `get_calibration(bundle_id)`
remain available. Existing `stream_joints()` / `set_target(mode="stream")` still
return `submitted`, not Node acceptance. Existing observation methods keep their
best-effort behavior. There is no dataset writer, episode model or recording scheduler
in this refactor.

## Legacy interfaces

## Observe (does not acquire control)

```python
from lekit_robot import connect

with connect(name="NERO Dual", timeout=5) as robot:
    obs = robot.get_observation(images=True, timeout=5)
    print(obs.robot.status, obs.robot.state)
    camera = obs.camera(name="head_view")
    if camera.color is not None:
        print(camera.color.shape)  # H x W x 3, RGB uint8
    print(robot.descriptor.joint_fields)
```

`images=False` omits images. `robot.observations(images=False, timeout=5)` aligns
independently published Robot and Camera samples with bounded latest-only buffers.
`get_observation()` is a one-shot convenience over the same stream alignment path;
the Node has no separate snapshot publication or observation query protocol.
For alignment time `T`, each source contributes its newest sample captured no later
than `T`; network arrival time is never used as capture time. Camera samples remain
independent, so an observation is not a hardware-synchronized frame set. Check each
sample's capture time and monotonic timestamp.
Depth is a uint16 array in the server's camera units, not automatically metres.
The Node's `orbbec` plugin outputs millimetres (`0` means invalid), with depth
aligned to RGB by default. Read it with `obs.camera(name="head_view").depth`;
convert to metres with `depth.astype("float32") / 1000`. The Orbbec SDK is only
needed on the server, not on this client. On the wire, aligned depth is always
losslessly compressed using the camera's configured LZ4 or Zstandard codec. The SDK transparently decompresses it for
`images=True`; use `images="encoded"` to retain the compressed payload and call
`camera.decode_depth()` only when needed.

Each observation also carries a small calibration reference that is timestamped
with the robot state. Fetch the complete, versioned bundle only when the reference
changes, then keep it in the recorder or application cache:

```python
with connect(name="NERO Dual", timeout=5) as robot:
    obs = robot.get_observation(images="encoded", timeout=5)
    ref = obs.calibration
    if ref is not None and ref.bundle_id is not None:
        bundle = robot.get_calibration(ref.bundle_id, timeout=5)
        camera = next(binding for binding in bundle.camera_bindings
                      if binding.camera_name == "head_view")
        print(ref.state, camera.intrinsics, camera.sensor_frame)
        print(bundle.edges)  # Check source_frame, target_frame, status and transform.
```

`robot.get_calibration()` returns the current bundle. Passing a `bundle_id` returns
that immutable historical bundle, so recorded observations remain reproducible
after a later recalibration. A reference state of `degraded` or `invalid` means the
bundle is available but one or more transform edges no longer match the active
camera bindings; downstream code decides whether those edges are usable.
For the geometry of a delivered image, read `obs.camera(name="head_view").geometry`.
If no validated bundle is active, `get_calibration()` raises
`CalibrationNotFoundError`.

## Control

Control requires enabled motors, `allow_control=True`, fresh feedback, normal mode,
and no emergency stop. Close Teleop before acquiring external control. The context
manager holds and renews an exclusive control lease and releases it on exit.
Multiple clients may observe; only one controller may own the lease.

The following function sends a real motion command **when called**:

```python
from lekit_robot import connect

def move_left_joint3(delta_rad):
    with connect(name="NERO Dual") as robot:
        with robot.control():
            obs = robot.get_observation(images=False)
            state = obs.robot.state
            if obs.robot.status != "connected" or not state or not state.get("left.feedback_fresh"):
                raise RuntimeError("Fresh left arm feedback is required")
            joints = [state[f"left.joint_{i}"] for i in range(1, 8)]
            joints[2] += delta_rad
            return robot.set_joints(joints, side="left")
```

| Method | Target |
| --- | --- |
| `set_joints(values, side=None, timeout=None)` | Absolute joint values in advertised order, or a mapping with full field names |
| `stream_joints(values, side=None, timeout=None)` | Publish a latest-only absolute joint target |
| `set_target(values, side=None, mode="point", timeout=None)` | Absolute flange/EEF pose in advertised order, or a full-key mapping |
| `set_target(x=..., y=..., z=..., roll=..., pitch=..., yaw=..., side="left")` | One arm's complete pose |
| `set_gripper(width, side="left", timeout=None)` | One gripper's absolute opening |
| `set_gripper(values, timeout=None)` | All grippers, in advertised order or full-key mapping |

For NERO Dual, `left` is can1 and `right` is can0. Joint/rotation units are radians;
position and gripper width units are metres. Pose fields represent the flange in
that arm's base frame. For other robots use the plugin-declared field semantics.
`side` selects only that arm; the other arm receives no target. Without `side`,
provide the complete advertised target (14 joint values or 12 pose values for NERO).
A scalar gripper width requires `side` on a dual-arm robot. There is no implicit
broadcast of one pose or gripper value to both arms.

NERO uses `nero-dual-ik` for Cartesian targets without an SDK or firmware IK
fallback. `stream_joints()` and `set_target(mode="stream")` use the Node's
latest-only stream channel and arm its loss-of-stream watchdog. Their immediate
receipt has `status="submitted"`; it does not say whether the asynchronous
device validation accepted the target. Streaming does not guarantee a hard
realtime servo or a straight path.
Unsupported modes, partial targets and out-of-range values are rejected.

A receipt acknowledges command submission, **not arrival or successful IK**.
Continue observing measured state and alarms. `ActionOutcomeUnknownError` means a
command may already have executed: do not retry it automatically. A control lease
is ownership arbitration, not a hardware emergency stop or a stop-on-disconnect guarantee.
Use the Robot Node emergency stop or physical stop as appropriate.

For a joint target that needs confirmed arrival, use a device-owned motion task:

```python
import time

from lekit_robot import connect

with connect(name="NERO Dual", timeout=5) as robot:
    task = robot.start_joint_task({"left.joint_3.pos": 0.2}, timeout_s=15)
    while not task.done:
        time.sleep(0.1)
        task = robot.get_motion_task(task.task_id)
    if task.state != "succeeded":
        raise RuntimeError(f"Motion task ended as {task.state}: {task.error}")
```

A task acquires its own exclusive control lease; do not enter `robot.control()`
around it. `start_joint_task()` returns an accepted task, not a completion
guarantee. `cancel_motion_task(id)` requests a stop; poll until `cancelled` with
`stop_confirmed=True` or inspect `failed`. If the start reply is lost,
`MotionTaskOutcomeUnknownError.task_id` identifies the task to query before
retrying. The Node currently supports joint tasks only.

## Compatibility

The wire protocol remains `lekit/v1`. New clients opt into the gripper manifest
extension; old clients continue to receive the original manifest shape. An old
Node can still supply observations and existing targets, but gripper commands need
an updated Node; unsupported grippers raise `UnsupportedCommandError` locally.
Per-arm commands also require the updated Node validation. Update client and Node
together for complete NERO Dual support. Reconnect after replacing the Node or its
robot configuration; the SDK never silently rebinds an existing control session.

Existing applications may keep `from lekit.robot import connect` when the main
project is installed: those imports re-export this package's implementation.

## Build and release

From `packages/lekit_robot`:

```sh
python -m pip install build twine
python -m build
python -m twine check dist/*
python -m twine upload --repository testpypi dist/*
# After testing the installed wheel in a clean environment:
python -m twine upload dist/*
```

Publishing needs a PyPI account and an API token or configured Trusted Publisher.
Never store credentials in source files. Version 0.1.0 is the initial release;
subsequent releases must use a new version number.

License: Apache-2.0, matching the repository's existing standalone package convention.

A build/test and manual publishing workflow is provided at
`.github/workflows/lekit-robot-release.yml`. It installs the built wheel in an
isolated environment before testing. Pushes and pull requests only build/test.
After the workflow is committed to the default branch, configure a PyPI pending
Trusted Publisher for a new project:

- Project: `lekit-robot`
- Owner: `sorelferris`
- Repository: `lekit`
- Workflow: `lekit-robot-release.yml`
- Environment: `pypi` (or `testpypi` on TestPyPI)

Then run the workflow manually, selecting `testpypi` first, and `pypi` for the
release. Configure the two registries separately. This avoids storing a long-lived
PyPI token. See https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/.
