Metadata-Version: 2.5
Name: cmoney-datahub
Version: 0.2.1
Summary: Python client for the CMoney DataHub delivery API — Taiwan first-party retail behaviour & market data
Project-URL: Homepage, https://datahub.cmoney.tw
Project-URL: Documentation, https://datahub.cmoney.tw/beta
Author: CMoney DataHub
License: Proprietary
Keywords: alternative-data,cmoney,datahub,finance,market-data,parquet,quant,quantitative,retail-sentiment,taiwan,tw
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Office/Business :: Financial :: Investment
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Requires-Dist: pandas>=1.3
Requires-Dist: pyarrow>=8
Requires-Dist: requests>=2.25
Provides-Extra: all
Requires-Dist: duckdb>=0.9; extra == 'all'
Requires-Dist: polars>=0.20; extra == 'all'
Requires-Dist: tqdm>=4.0; extra == 'all'
Provides-Extra: duckdb
Requires-Dist: duckdb>=0.9; extra == 'duckdb'
Provides-Extra: polars
Requires-Dist: polars>=0.20; extra == 'polars'
Provides-Extra: progress
Requires-Dist: tqdm>=4.0; extra == 'progress'
Description-Content-Type: text/markdown

# cmoney-datahub

Thin Python client for the CMoney DataHub delivery API. It mirrors the portal's
pull path (`catalog → tree → presign → parquet`) and returns pandas DataFrames.
Downloads are **S3-direct** (the API only signs URLs), so large pulls are not
bound by any server request timeout.

## Install

```bash
pip install cmoney-datahub
```

Optional extras: `polars` (`engine="polars"`), `duckdb` (`.sql()`), `progress` (tqdm bar),
or everything — `pip install "cmoney-datahub[all]"`.

The package is a thin client; you still need a **CMoney DataHub API key** to pull data —
pass it as `Client("dpk_...")` or set `CMONEY_DATAHUB_KEY`. Contact your account manager for a key.

<details><summary>Alternative: install from the portal (no PyPI)</summary>

The key must go in an HTTP header, so download the wheel then install it:

```bash
curl -fsSL -H "X-API-Key: dpk_..." \
  "https://datahub.cmoney.tw/api/skill?format=wheel" -o cmoney_datahub.whl
pip install --force-reinstall cmoney_datahub.whl
```

In Jupyter: run the `curl` in a `!` cell, then `%pip install --force-reinstall <wheel>` and
restart the kernel. For local development: `pip install -e ./sdk`.
</details>

## Quickstart

```python
from cmoney_datahub import Client

c = Client("dpk_...")            # or set CMONEY_DATAHUB_KEY and call Client()
c.datasets()                     # datasets this key is entitled to (code · slug · title · category)

df = c.get("market-vote", start="2026-06", end="2026-06")
df.head()
```

The dataset id accepts the **client-facing slug** (`stock-pageviews`) or the
internal code (`a3-stock-pv`) interchangeably.

## The surface model — one `get()` = one surface

A dataset exposes one or more **surfaces**. A surface is either a bare panel
(e.g. `app-forum`) or a `panel/grain` pair (e.g. `app-forum/daily`); the grain
drives the column set. Each `get()` returns exactly one surface's schema — if a
multi-surface dataset is under-specified, it raises `AmbiguousSurfaceError`
rather than silently stacking mismatched columns.

```python
# grid dataset (panel × grain)
df = c.get("stock-pageviews", panel="app-forum", grain="daily",
           start="2026-01", end="2026-03")
# equivalently: surface="app-forum/daily"

# record / list dataset
df = c.get("paper-trading", surface="deal-record", start="2026-01", end="2026-03")

# single-surface dataset — nothing to pass
df = c.get("market-vote", start="2026-06", end="2026-06")
```

Discover surfaces with `c.surfaces(code)`; inspect one with
`c.schema(code, panel=..., grain=...)`.

## Core methods

| Method | Purpose |
|---|---|
| `Client(key=None, api=...)` | Connect. `key` falls back to `$CMONEY_DATAHUB_KEY`. |
| `.datasets()` | Entitled datasets (drops not-yet-uploaded ones). |
| `.surfaces(code)` | Pullable surface list for a dataset. |
| `.schema(code, *, surface=\|panel=+grain=)` | One surface's columns + types. |
| `.coverage(code, *, surface=\|panel=+grain=)` | `{from, to, months}` coverage. |
| `.estimate(code, start, end, *, surface=\|panel=+grain=)` | `{files, bytes}` without downloading. |
| `.get(code, start, end, *, surface=\|panel=+grain=, tickers=, columns=, version=, workers=4, engine="pandas", cache=True, progress=)` | Pull one surface → DataFrame. |
| `.get_all(code, start, end, ...)` | Every surface at once → `{surface: DataFrame}`. |
| `.iter_get(code, ..., columns=, version=, cache=True)` | Stream **one DataFrame per file** (bounded memory). |
| `.download(code, dest, ..., workers=4, cache=True)` | Write parquet files to disk (no pandas), keeping the S3 layout. |
| `.sql(query, code=None, *, surface=\|panel=+grain=, start, end, table="data")` | DuckDB over the cached slice → pandas. |
| `.versions(code)` | Available versions (months) → pass to `version=`. |
| `.usage()` | Trial quota vs cap. |

## Working with large datasets

A single `get()` concatenates every file into one DataFrame — fine for most
slices, but a wide multi-year pull (e.g. `stock-pageviews` ≈ 1 GB / 144M rows)
can exhaust memory. Two escape hatches:

```python
# stream one month at a time — memory stays bounded
for df in c.iter_get("stock-pageviews", panel="app-forum", grain="daily",
                     start="2024-01", end="2026-06"):
    process(df)

# or land parquet on disk and query with DuckDB / Polars (no pandas)
paths = c.download("stock-pageviews", "./pv", panel="app-forum", grain="daily")
import duckdb
duckdb.sql("select * from read_parquet('pv/**/*.parquet')")

# or let the SDK stage + query it for you (auto-caches; exposes the slice as view `data`)
top = c.sql("select stockid, sum(pv) pv from data group by 1 order by pv desc limit 10",
            code="stock-pageviews", panel="app-forum", grain="daily",
            start="2026-01", end="2026-03")
```

`get_all()` pulls every surface at once, and `engine=` changes the return type:

```python
panels = c.get_all("stock-pageviews", start="2026-01", end="2026-03")   # {surface: DataFrame}
df = c.get("market-vote", start="2026-06", end="2026-06", engine="polars")  # polars.DataFrame
```

## Behaviour notes

- **Ticker filter is client-side.** `tickers=["2330"]` filters after download; the
  server never receives symbols (PIT-safe, matches the flat-file delivery model).
- **Column projection.** `columns=["stockid", "pv"]` reads only those columns from
  parquet — less memory and parse time on wide datasets.
- **sha-verified local cache** (on by default). Files are cached by content hash,
  so repeat pulls read from disk and **cost no quota**. Disable with `cache=False`;
  relocate with `cache_dir=` or `$CMONEY_DATAHUB_CACHE`.
- **Quota is checked before downloading** — a request that won't fit the trial cap
  raises `QuotaError` up front (the server 403 is the hard backstop).
- **Window / entitlement are enforced server-side.** An over-range request warns
  (or raises with `strict=True`) and returns the available intersection.
- **Every call reports.** `get` / `download` / `iter_get` print a one-line receipt —
  file counts (new · cached · failed), bytes pulled this call, live quota (used/cap/left),
  and cumulative `c.session_bytes`. `get()` also attaches `df.attrs["cmoney"]` =
  `{files, downloaded, cached, failures, bytes, bytes_new, session_bytes}`. Silence with `progress=False`.

## Errors

All subclass `DataHubError` and carry a stable `e.code` + `e.status` (HTTP), so you can
branch by class *or* by code:

| Class | `.code` | HTTP |
|---|---|---|
| `AuthError` | `unauthorized` | 401 |
| `EntitlementError` | `not_entitled` | 403 |
| `QuotaError` | `quota_exceeded` (+ `.used_gb` / `.cap_gb`) | 403 |
| `RateLimitError` | `rate_limited` | 429 |
| `NotFoundError` | `not_found` | 404 |
| `AmbiguousSurfaceError` | `ambiguous_surface` (+ `.dataset` / `.options`) | — |

```python
from cmoney_datahub import DataHubError
try:
    df = c.get("stock-pageviews", panel="app-forum", grain="daily", start="2026-01", end="2026-03")
except DataHubError as e:
    print(e.code, e.status, e)   # e.g. quota_exceeded 403 ...
```
