Metadata-Version: 2.4
Name: forge-msgs
Version: 1.3.0
Summary: Python message models and Arrow serialization for Forge robotics dataflows.
Author: X-ERA
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Dist: numpy>=1.26,<3
Requires-Dist: pillow>=10,<13
Requires-Dist: pyarrow>=14,<24
Requires-Dist: pydantic>=2.0,<3
Requires-Dist: pytest>=8.0.0 ; extra == 'test'
Requires-Python: >=3.11
Project-URL: Changelog, https://gitlab.ex-ai.cn/PhyAgentOS/framework/forge/-/blob/master/CHANGELOG.md
Project-URL: Issues, https://gitlab.ex-ai.cn/PhyAgentOS/framework/forge/-/issues
Project-URL: Repository, https://gitlab.ex-ai.cn/PhyAgentOS/framework/forge
Provides-Extra: test
Description-Content-Type: text/markdown

# forge-msgs

Forge message definitions for Dora dataflow.

The canonical cross-language contract starts at
`interfaces/forge_msgs/forge_msgs.v1.yaml`. That manifest references the
versioned domain schemas followed by the Python, Rust, and C++ implementations.

## Core Messages

### `JointState`

Robot or simulator joint observation payload.

- `name: list[str]`
- `position: list[float]`
- `velocity: list[float]`
- `effort: list[float]`

`name` must be non-empty and unique. Numeric lists must either be empty or have the same length as `name`.

### `JointCommand`

Command payload for robot drivers and controllers.

- `name: list[str]`
- `mode: "position" | "velocity" | "effort" | "hybrid"`
- `position: list[float]`
- `velocity: list[float]`
- `effort: list[float]`
- `kp: list[float]`
- `kd: list[float]`

`mode` describes the command semantics. `position`, `velocity`, and `effort` match the common ROS2 command interfaces; `hybrid` is for low-level position/velocity/effort commands with optional `kp`/`kd` gains. Payloads written before `mode` existed are read as `mode="position"`.

`name` is a sparse update set: consumers update only the listed joints. Omitted joints retain their previous command target and must never be reset or filled with zero implicitly. Disjoint arm and gripper controllers can therefore share an ordered command stream without constructing a complete joint vector.

For Unitree-style low-level control, map `position -> q`, `velocity -> dq`, `effort -> tau`, `kp -> kp`, and `kd -> kd`.

### `LocomotionCommand`

Planar body-frame locomotion velocity command for ground robots.

- `vx: float`
- `vy: float`
- `wz: float`

`vx` and `vy` are body-frame linear velocities in m/s, with positive X forward and positive Y left. `wz` is angular velocity about the body-frame Z axis in rad/s, positive counter-clockwise, corresponding to ROS2 `Twist.angular.z` / yaw rate. Stop is represented by all zeros. This message intentionally omits `mode`, `duration`, `gait`, `body_height`, timestamp/frame fields, and full 6D Twist fields.

### `Image`

Uncompressed image payload.

- `height: int`
- `width: int`
- `encoding: str`
- `step: int`
- `data: bytes`

Supported encodings are `rgb8`, `bgr8`, `mono8`, `8UC1`, `16UC1`, `32SC1`,
and `32FC1`. Multi-byte pixels are little-endian. `8UC1` is available for
generic one-channel byte data such as label maps, while `32SC1` supports signed
32-bit label images. Compressed images use `CompressedImage`.

### `CompressedImage`

Compressed image bitstream payload.

- `format: str`
- `data: bytes`

Recommended formats are `jpeg`, `png`, and `webp`.

### `TeleopObservation`

XR device raw teleop observation for embodiment-specific teleop policies.

- `device: list[str]`: XR source ids; recommended ids are `left`, `right`, and
  `headset`. Match entries by id rather than list order.
- `x/y/z: list[float]`: positions in meters in the producer-configured XR
  tracking frame.
- `qx/qy/qz/qw: list[float]`: quaternion components in `xyzw` order.
- `confidence: list[float]`: finite producer-defined confidence values in
  `[0, 1]`; zero means that the aligned pose must not be used for control.
- `buttons_json: str`: JSON object of digital boolean and analog finite-number
  values keyed by button id.
- `axes_json: str`: JSON object of finite numbers or numeric arrays keyed by
  axis or documented derived-control id.

Recommended button ids include `A`, `B`, `X`, `Y`, the `left_`/`right_`
trigger and grip ids, and the joystick-click ids. `left_axis` and `right_axis`
are `[x, y]` joystick pairs. An unavailable pose may use zero position and an
identity quaternion with confidence `0`.

The payload does not carry the tracking-frame origin, handedness, axis
directions, or timestamp. Document the frame convention in producer
configuration and carry timing in Dora event context or an adapter layer.

### `Text`

Single UTF-8 text payload for ASR transcripts, LLM responses, TTS input, and other text streams.

- `text: str`

Producers should omit empty outputs when there is no meaningful text.

### `ToolMessage`

Cross-language single-row Arrow carrier for one `forge.tool.endpoint/v1alpha1` logical
message. Msgs `1.2.0` introduces the carrier; the first tagged/public `forge-tool`
package release using this protocol is `0.1.0`. Earlier untagged prototypes are
incompatible, and Python/Rust/C++ bindings plus Gateway and provider must deploy
atomically. Its exact ten-column order is `protocol`, `message_type`,
`request_id`, `invocation_id`, `attempt_id`, `endpoint_id`,
`endpoint_instance_id`, `operation`, `sequence`, and `payload_json`; all are
`utf8` except nullable `sequence: int64`. `request_id`, `invocation_id`,
`attempt_id`, `endpoint_instance_id`, `operation`, and `sequence` use Arrow null when
omitted. `endpoint_instance_id` is structurally nullable on pre-routing `tool.*` messages,
but a provider-originated `tool.event` must supply its concrete instance; every
`endpoint.*` message, including `endpoint.status`, requires it.

`payload_json` encodes the logical payload object rather than the full carrier
envelope. It must be a bounded strict JSON object with unique keys, valid Unicode,
finite numbers, interoperable integers, and at most 64 levels. The carrier has no
observation timestamp; use Dora event context for transport observation time.
`forge_msgs` validates the carrier and generic header/identity matrix, including the
`endpoint.registry.response` message class. Registration, unregister, and Registry response carriers
require `request_id`; unsolicited endpoint status requires null and
remains a health message rather than an ACK. Message-specific payload validation remains in
`forge-tool`. Python, Rust, and C++
implement the carrier, with bidirectional Python/C++ Arrow IPC compatibility
coverage.

### Perception result sets

The perception messages use parallel Arrow list columns in a single-row
`RecordBatch`:

- `Detection2DSet` stores oriented pixel-space boxes and flattened class
  hypotheses. Axis-aligned boxes use `rotation=0`, and high-level constructors
  may fill that default automatically.
- `Detection3DSet` stores oriented metric boxes and flattened class hypotheses.
  Axis-aligned boxes use identity orientation (`qx=0`, `qy=0`, `qz=0`, `qw=1`),
  and high-level constructors may fill that default automatically.
- `SegmentationMaskSet` stores cropped `mono8` instance masks and their source
  image offsets. Standalone masks can omit detection and track associations,
  and high-level constructors may fill empty associations and zero offsets.

Detection class names and model label maps belong in node configuration.
`class_id` is the stable identifier carried on the wire.
Local features such as ORB, SIFT, or SuperPoint descriptors are intentionally
kept out of the core streaming schema; publish higher-level outputs unless a
future pipeline needs cross-node feature reuse.

### `PointCloud`

`PointCloud` stores common XYZ point clouds as Arrow lists with optional
intensity and RGB columns. Organized clouds preserve `width` and `height`;
unorganized clouds use `height=1`. Coordinates use meters, while the coordinate
frame is supplied by Dora metadata or node configuration. High-level
constructors may infer unorganized clouds from `x/y/z` by setting `height=1`,
`width=len(x)`, and `is_dense` from finite XYZ values.

The owned `PointCloud` model uses Python lists and is intended for convenience,
tests, and small clouds. Treat an instance as an immutable message value after
construction. High-throughput producers can construct the same wire schema from
typed NumPy buffers:

```python
import numpy as np
from forge_msgs import PointCloudBatch, PointCloudView

x = np.arange(640 * 480, dtype=np.float32).reshape(480, 640)
y = np.zeros_like(x)
z = np.ones_like(x)

# Safe default: create an immutable payload snapshot.
batch = PointCloudBatch.from_numpy(x=x, y=y, z=z).to_arrow()

# Consumer-side field views share the Arrow value buffers and are read-only.
view = PointCloudView.from_arrow(batch)
assert view.x.shape == (640 * 480,)
```

A `PointCloudView` is not an immutable snapshot: mutations through any
independent writable alias to the input Arrow storage remain observable.

XYZ inputs are same-shape one- or two-dimensional `float32` arrays. A 2D shape
infers organized `height/width`; view properties are always flat row-major
arrays. Optional intensity is `float32` with the XYZ shape or a flat point-count
shape. RGB is `uint8`, supplied either as three matching channel arrays or one
interleaved `(..., 3)` array. `is_dense=None` scans XYZ once to infer density.

`PointCloudBatch.from_numpy()` has explicit payload-buffer policies:

- `copy="always"` is the safe default and snapshots every supplied attribute.
- `copy="if_needed"` borrows only exact-dtype, naturally aligned, C-contiguous
  arrays whose visible backing chain is read-only. Inputs that satisfy the shape
  and casting contract but cannot be borrowed are copied.
- `copy="never"` rejects any input that cannot be borrowed. Small scalar and
  Arrow list-offset buffers are still allocated.
- `casting="no"` is the default, so dtype mismatches are rejected. `safe` or
  `same_kind` casting must be requested separately and requires a copy when the
  dtype changes. Float32 and RGB range checks still apply.

A three-array `(red, green, blue)` tuple can use the borrowing path. Interleaved
`(..., 3)` RGB requires deinterleaving into the v1 SoA columns and is therefore
rejected by `copy="never"`.

When borrowing, Arrow retains the exported buffer lifetime, so callers do not
need to keep source-array references alive manually. Callers are still
responsible for ensuring that no remaining writable alias modifies the storage
or re-enables writes until all derived Arrow batches and views are released.
NumPy's read-only flag is a guard, not a guarantee against another alias. Use the
default `copy="always"` whenever immutability cannot be guaranteed. Buffer
sharing applies only to the local NumPy/PyArrow boundary; it does not imply that
Dora IPC, networking, recording, or downstream layout conversion is end-to-end
zero-copy.

### `PointCloudBuffer`

`PointCloudBuffer` is the decoded-sensor counterpart to normalized `PointCloud`.
It preserves a fixed-stride point-record buffer across a node boundary, including
field names, numeric datatypes, byte offsets, point/row padding, byte order, and
additional fields such as per-point time or laser channel. It is not an
undecoded device packet and does not imply a specific calibration, deskew,
filter, transform, or registration stage.

Every buffer contains scalar meter-valued `x`, `y`, and `z` fields of the same
`float32` or `float64` datatype. Other fixed-width numeric fields are optional.
Writers canonicalize Arrow payloads to little-endian bytes; readers can consume
both byte orders and reject overlapping or out-of-bounds descriptors. The only
empty shape is `width=0`, `height=1`, `row_stride=0`, and empty data while the
point stride and descriptors continue to declare the record layout.

`PointCloudBufferView` validates the Arrow payload, retains its buffer owner, and
returns read-only NumPy arrays with strides derived from `row_stride`,
`point_stride`, field offset, and element size:

```python
from forge_msgs import PointCloudBufferView

view = PointCloudBufferView.from_arrow(event["value"])
x = view.field("x")
if view.has_field("time_offset_ns"):
    time_offset_ns = view.field("time_offset_ns")
```

Scalar unorganized fields have shape `(width,)`; organized fields have shape
`(height, width)`, and `count>1` adds a trailing dimension. These arrays can be
non-contiguous, non-native-endian, and unaligned. The view does not silently
copy or normalize them. Use `np.ascontiguousarray()` explicitly when an algorithm
requires native contiguous columns. `PointCloudBuffer.from_arrow()` instead
materializes an owned `bytes` snapshot.

Use `PointCloudBuffer` only while dynamic record layout or per-point attributes
must cross a node boundary. Publish existing `PointCloud` after preprocessing
when only normalized XYZ/intensity/RGB data remains relevant.

### `Imu`

`Imu` carries one SI-unit inertial sample with required angular velocity in
rad/s and linear acceleration in m/s². Orientation is a nullable struct with
named `qx/qy/qz/qw` components, forcing adapters to convert vendor array order
explicitly. The message does not include redundant roll/pitch/yaw values and
never silently normalizes a quaternion.

Each covariance is either an empty list when unavailable or a row-major 3x3
`float64` matrix with a non-negative diagonal. A missing orientation uses Arrow
null and requires an empty orientation covariance; optional temperature also
uses Arrow null rather than NaN or a numeric sentinel. Timing and sensor frame
remain in Dora event context and stable node configuration.

## Motion Messages

The motion API provides single-row Arrow payloads for joint trajectories and
configured-group motion actions:

- `JointTrajectoryPoint`, `JointTrajectory`, and `JointTolerance`
- `FollowJointTrajectoryGoal`, `FollowJointTrajectoryFeedback`, and
  `FollowJointTrajectoryResult`
- `GripperCommandGoal`, `GripperCommandFeedback`, and `GripperCommandResult`
- `MoveJointsGoal`, `MoveJointsFeedback`, and `MoveJointsResult`
- `MovePoseGoal`, `MovePoseFeedback`, and `MovePoseResult`

Trajectory points and tolerances are encoded as nested Arrow structs, including
`list<struct>` trajectory and tolerance fields. `MovePose*` messages reuse
`Pose` as a nested struct. Optional scalar and pose fields are represented by
Arrow nulls rather than sentinel values.

`GripperCommand` addresses the coordinate configured for its controller endpoint;
it does not carry `joint_name` or a dynamic unit. Prismatic coordinates use meters,
meters/s, and Newtons. Revolute or rotary actuator coordinates use radians,
radians/s, and Newton-meters. Explicit velocity or effort limits remain nullable so
a position-only backend can reject unsupported requested limits rather than silently
ignore them.

`goal_id` and `goal_status` are Dora action metadata and are intentionally not
included in payload columns. Domain result codes are separate from Dora's
lowercase terminal `goal_status` values.

## Arrow Format

All core messages encode to a single-row `pyarrow.RecordBatch`.

```python
from forge_msgs import JointCommand, JointState

state = JointState(
    name=["joint1", "joint2"],
    position=[0.1, 0.2],
    velocity=[0.0, 0.0],
    effort=[],
)
batch = state.to_arrow()
back = JointState.from_arrow(batch)

command = JointCommand.from_np(
    values=[0.3, 0.4],
    order=["joint1", "joint2"],
    field="position",
)
```

Timing and frame data are intentionally not part of the core schema. Carry them in Dora event context, topic naming, node configuration, or adapter layers.
