Metadata-Version: 2.3
Name: anyverse-adp
Version: 0.5.3
Summary: CLI and Python SDK for the Anyverse Data Platform Episode catalog
Requires-Dist: tomli>=2.0 ; python_full_version < '3.11'
Requires-Dist: httpx>=0.28,<1
Requires-Dist: prompt-toolkit>=3.0,<4
Requires-Dist: rich>=13.9,<14
Requires-Dist: questionary>=2,<3
Requires-Dist: typer>=0.15,<1
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# Anyverse ADP CLI and Python SDK

`anyverse-adp` provides the `adp-cli` command and a Python SDK for reading the Anyverse Data Platform's registered LeRobot catalog, managing Episode annotations and clips, and exporting Episode metadata CSV files.

Install it with `pip install anyverse-adp`. Create a Platform API Key with `episodes:read`; annotation or clip writes also require `episodes:write`. Configure the key with an environment variable or reuse the CLI's secure `~/.adp-cli/config.toml`.

```text
adp-cli list
adp-cli --version
adp-cli browse
adp-cli info <dataset-slug>
adp-cli export <dataset-slug> <collection-key> --region <region> --output <directory-or-file.csv>
```

`adp-cli browse` is an inline terminal selector. It redraws a single `Anyverse Data Platform CLI` panel in the current terminal so the selected Dataset, Collection, region, and latest result stay together. Use Up/Down and Enter to select a Dataset, Collection, region, and export mode; type in Dataset and Collection menus to filter choices, and use Backspace to clear the filter. Press Esc to return to the previous level (or exit from Dataset). After a dry-run or completed export, its result remains visible and the selector offers continue or exit. It does not open a full-screen UI or switch to an alternate terminal screen.

The command reads catalog metadata, including the selected Artifact `oss://bucket/prefix/` root in the `oss_location` CSV column, the Episode UUID, and `video_refs` keyed by the original camera/view feature name. Each video reference contains its stable OSS path and indexed start/end offsets in nanoseconds. It does not read or download dataset objects, expose raw annotations, or grant object-storage access.

## Parquet references (0.4.0)

Version 0.4.0 appends `parquet_filepath`, `parquet_row_start`, and `parquet_row_end` to CSV exports. The filepath is a stable `oss://` URI resolved from the selected region's unique Artifact location and the API's relative file path. The row range is zero-based `[start, end)` inside that file, with `end - start == frame_count`; it is not the Episode index. Unindexed references produce three empty cells. Older APIs without all three metadata fields also produce empty cells, so deploy the updated API before using 0.4.0 for Parquet exports. Partial or invalid references fail the export instead of publishing an incomplete CSV.

SDK Episodes expose `parquet_relative_path`, `parquet_filepath`, `parquet_row_start`, and `parquet_row_end`. These are `None` when the reference is unavailable. Resolving a complete path requires exactly one Artifact location in the client's selected region; ambiguous locations raise `AdpResponseError`.

## Python SDK (0.5.3)

The SDK uses a synchronous `Client` and resource handles organized as `Dataset → Collection → LeRobot → Episode`. Client and Dataset lists are materialized for indexing; LeRobot and Episode collections remain lazy and page as you iterate.

Read the installed package version from Python with `import anyverse_adp; print(anyverse_adp.__version__)`. It reports `"unknown"` when package metadata is unavailable, such as when importing an unpackaged source checkout.

```python
from anyverse_adp import AdpAuthorizationError, AdpError, Client

client = Client()  # explicit api_key, ADP_API_KEY, ADP_CLI_API_KEY, then ~/.adp-cli/config.toml
try:
    datasets = client.get_datasets()
    dataset = client.get_dataset("agibot_world")
    collections = dataset.get_collections()
    collection = dataset.get_collection("dexhand_auto_views")
    lerobot = next(iter(collection.lerobots))
    episode = lerobot.get_episode(index=100)  # one direct server lookup by (LeRobot, index)
    print(episode.episode_uuid, episode.index)
    episode = client.get_episode(episode_uuid=episode.episode_uuid)  # direct lookup by Episode UUID
    episode = lerobot.get_episode(episode_uuid=episode.episode_uuid)  # scoped to this LeRobot
    episode = collection.lerobots.get_episode(episode_uuid=episode.episode_uuid)  # scoped to this Collection

    # Bulk traversal remains lazy and cursor paged.
    for item in lerobot.get_episodes():
        print(item.index)

    # These are complete lists scoped to this Episode. List entries contain summaries;
    # accessing .annotation fetches the individual JSON body.
    annotations = episode.annotations
    annotation = episode.get_annotation("scene", "prompt-v3")
    print(annotations[0].annotation_type, annotation.annotation)

    clips = episode.get_clips("scene", "prompt-v3")
    if clips:
        print(clips[0].label, clips[0].annotation)

    # Writes use episodes:write and are keyed upserts; repeated keys update the same record.
    episode.add_clip(
        annotation_type="scene",
        model_prompt_version="prompt-v4",
        segment_id=1,
        frame_start=12,
        frame_end=48,
        label="pick up object",
        annotation={"object": "cup"},
    )
    episode.add_annotation(
        annotation_type="scene",
        model_prompt_version="prompt-v4",
        artifact_uri="oss://annotations/episode-100.json",
        result_flag=True,
        annotation={"summary": "The robot picks up the object."},
    )
except AdpAuthorizationError as exc:
    print(f"This operation requires {exc.required_scope}: {exc.operation}")
except AdpError as exc:
    print(f"ADP request failed: {exc}")
finally:
    client.close()
```

The default endpoint is `https://data-platform.cloud.anyverse.work` and the default region is `cn-shanghai`. Pass `endpoint=`, `region=`, or `api_key=` to override them. API Key precedence is explicit `api_key`, `ADP_API_KEY`, `ADP_CLI_API_KEY`, then the CLI config file `~/.adp-cli/config.toml`. The SDK reads that file only, preserves its permission and symlink checks, never prompts, and never writes it.

Use `client.get_episode(artifact_id, episode_index)` or `lerobot.get_episode(index)` for one Episode. The Platform API looks up that `(artifact_id, episode_index)` key directly. `iter_episodes()` and `get_episodes()` continue to use lazy cursor pagination.

`get_episode()` accepts one selector: `client.get_episode(artifact_id, episode_index)` (or `artifact_id=` and `episode_index=`) for index lookup, or `client.get_episode(episode_uuid=...)` for UUID lookup. The same UUID selector is available on `lerobot.get_episode()` and `collection.lerobots.get_episode()`; those resource methods constrain the result to the current LeRobot Artifact or Dataset Collection. `client.get_episode_by_uuid(episode_uuid)` remains as a compatibility alias. The Platform API looks up the Episode UUID primary key and returns the same metadata and region-scoped Artifact locations as index lookup. Direct lookups require `episodes:read`; a missing UUID or a UUID outside the selected resource scope raises `AdpNotFoundError`. No database migration is required.

`episode.video_refs` maps each camera feature key to a tuple of `VideoRef` values. Each reference exposes `relative_path`, `oss_uri`, `time_start_ns`, and `time_end_ns`. `oss_uri` combines the reference with the unique Artifact location for the Client's selected region; it is `None` if that location is ambiguous. Direct Episode lookups and paginated Episode reads populate the same fields. The URI identifies an OSS object but is not a signed HTTP URL; the SDK does not download video files.

`episode.get_annotations()` / `episode.annotations` and `episode.get_clips()` / `episode.clips` return complete lists for one Episode. The server returns summary fields for lists, avoiding transfer of every JSON body. Use `episode.get_annotation(annotation_type, model_prompt_version)` for an Annotation body; Clip list items load their body through `clip.annotation`. Annotation and Clip writes are keyed upserts using the existing `episodes:write` scope. Read calls require `episodes:read`; writes without `episodes:write` raise `AdpAuthorizationError` with `required_scope` and `operation`. Invalid request data raises `AdpValidationError` with the safe server code and message. Other typed failures include authentication, not-found, region, timeout, network, response-shape, and unhandled HTTP errors. Exceptions never include the API Key or raw response body.

The SDK reads metadata and annotation/clip records through Platform API. It does not download OSS objects, sign URLs, or use OSS credentials.

可直接运行的完整示例位于 `examples/sdk_catalog.py`：

```bash
ADP_API_KEY='<完整 API Key>' \
python packages/anyverse-adp/examples/sdk_catalog.py agibot_world --collection dexhand_auto_views
```

Jupyter 代码示例位于 [`examples/sdk_catalog.ipynb`](examples/sdk_catalog.ipynb)，按单元依次演示 Dataset、Collection、LeRobot 和 Episode 的读取。它只读取目录元数据，不下载 OSS 对象。
