Metadata-Version: 2.5
Name: octoparse-client
Version: 0.2.9
Summary: Official Python SDK for Octoparse DataHub: discover, run and consume Data Apps
Project-URL: Homepage, https://www.octoparse.com
Author: Octoparse DataHub Team
License: MIT
License-File: LICENSE
Keywords: data-app,datahub,octoparse,sdk
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Description-Content-Type: text/markdown

# octoparse-client-python — the official DataHub Python SDK

The official Python client for Octoparse DataHub (`octoparse-client`): discover, run and consume Data Apps.
A small self-contained project with zero platform-side dependencies (`httpx` is the only runtime
dependency) that speaks only the platform's public `/v1` REST API.
Repositories follow the `<brand>-client-<language>` naming; clients for other languages sit alongside
(e.g. octoparse-client-js).

## Installation

```bash
pip install octoparse-client
```

## Usage

```python
from octoparse_client import Client

# base_url defaults to the production endpoint https://api-datahub.octoparse.com;
# pass it explicitly for local / staging environments
with Client(api_key="demo-key") as client:
    # Discover
    result = client.search("reviews", limit=10)
    detail = client.get_app("demo/reviews-store-query")   # or the card's app_id

    # Run (wait for the terminal state) and consume
    run = client.call("demo/partner-reviews-api", {"product": "p-9001"})
    for record in client.iterate_records(run["run_id"]):
        print(record["content"], record["rating"])

    # Inventory and reconciliation
    for r in client.iterate_runs(run_kind="production", created_from="2026-08-01T00:00:00Z"):
        print(r["run_id"], r["state"], r["billing"])
    print(client.billing(group_by="data_app", tz_offset=480))   # spend per local day / per App

    # Result data is kept for 90 days by default; mark datasets you want to keep
    client.set_dataset_retention(run["dataset_id"], retained=True)
```

`call()` starts a run and polls until it reaches a terminal state; for non-blocking semantics use
`run()` to get a `run_id`, then `get_run()` / `cancel()` yourself. Apps shared with you
point-to-point do not appear in marketplace results; query them with `search(shared_with="me")`.
All methods and the error mapping live in `src/octoparse_client/client.py`.

App references come in two forms, accepted interchangeably by `get_app()` / `run()` / `call()` and
by the `data_app` filter of the run listings: the two-part `<namespace>/<app_name>` reference
(taken from the card's `namespace` and `app_name`, human-readable, breaks once the publisher renames
the App); and the stable identifier `app_id` (`app_<hex>`, taken from the card's / detail's `app_id`
field, immune to renames). For long-term integrations such as config files or scheduled jobs, pin
the `app_id`.

## Configuration

When `Client()` is constructed without `base_url` / `api_key`, they fall back to the
`OCTOPARSE_BASE_URL` / `OCTOPARSE_API_KEY` environment variables (template in `.env.example`; the SDK
does not load `.env` itself, the caller does), then to the production endpoint
`https://api-datahub.octoparse.com` / anonymous access.

## Bring your own token (token_provider, advanced)

The standard credential for programmatic access is the API Key. Callers that already hold a valid
identity JWT elsewhere (gateway forwarding, the login state of an MCP OAuth authorization-code flow)
can inject their self-managed credential with `Client(token_provider=)`, which is mutually exclusive
with `api_key`: pass a zero-argument callable returning the currently valid token, or implement the
`TokenProvider` protocol (`token(min_ttl=)`, renewal is the implementation's job).
For providers with `min_ttl` semantics, the SDK by default requires a remaining token lifetime of
at least 12h before submitting a run (otherwise it first asks the provider for a fresher token;
tune with `Client(run_token_min_ttl=)`). The platform snapshots the credential at enqueue time and
replays it for billing at the terminal state, so the fresher the token at submission, the smaller
the window in which a long run's billing credential expires. The SDK ships no
username/password-to-token implementation (ROPC is deprecated by OAuth 2.1 and the platform no
longer accepts plaintext account credentials).

## Tests

```bash
uv run pytest -q    # offline unit tests (MockTransport, no server required)
```
