Metadata-Version: 2.4
Name: learnml-sdk
Version: 0.1.5
Summary: Python SDK for the LearnML training data management platform
Home-page: https://github.com/milindjain0/learnml
Author: LearnML
Author-email: milindjain0@gmail.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: grpcio>=1.68.1
Requires-Dist: requests<3,>=2.32.0
Requires-Dist: requests-toolbelt<2,>=1.0.0
Requires-Dist: protobuf<6.0.0,>=5.28.1
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# LearnML Python SDK 0.1.4

HTTP(S) URLs select the website API; bare host:port addresses retain direct gRPC.

```python
from getpass import getpass
from learnml import LearnMLClient

with LearnMLClient("https://your-learnml-server.example") as client:
    client.login("you@example.com", getpass("LearnML password: "))
    universe = next(u for u in client.list_universes() if u["name"] == "My experiments")
    uid = universe["id"]
    run = client.create_training_run(uid, "my-experiment", model_name="my-model")
    client.log_metrics(uid, run["id"], step=1, loss=0.42, accuracy=0.91)
    client.upload_data(uid, "experiment-data", "CUSTOM", "experiment.csv")
    client.end_training_run(uid, run["id"])
```

Install with `python -m pip install --upgrade learnml-sdk`, or install a local checkout with `python -m pip install ./sdk`. Restart a notebook kernel after upgrading an already-imported package. HTTP support requires version 0.1.3 or later.

HTTP mode preserves the existing public methods and camelCase response dictionaries. `token` and refresh state work in both modes. Requests have connection/read timeouts; API and network errors use the SDK exception classes. HTTP multipart uploads stream from the client file; the current gateway still buffers uploads in server memory. `chunk_size` controls gRPC chunks; the HTTP transport controls its own multipart read sizes. HTTP collection batching uses paginated API calls.

The existing gateway cannot accept custom checkpoint metadata; HTTP `save_checkpoint(metadata=...)` raises an explicit error if nonempty metadata is supplied. Empty metadata and the normal checkpoint upload/download flow are supported.

Run regression tests with `python -m unittest discover -s sdk/tests` from the repository root after installing the SDK.

## API tokens for Colab and long-running jobs

Sign in to the website, open **API Tokens**, and select **Generate token**. Give the token a name and choose **No expiry — until revoked**, or a fixed expiry. Copy the full token immediately: it is shown only once. Token hashes, names, prefixes and timestamps are stored on the server; full secrets cannot be retrieved later.

Save it in Colab Secrets as `LEARNML_API_TOKEN`, enable notebook access, and use:

```python
from google.colab import userdata
from learnml import LearnMLClient

client = LearnMLClient(
    "your-learnml-server.example:50051",
    token=userdata.get("LEARNML_API_TOKEN"),
)
# No login or token-refresh loop is required.
print(client.list_universes())
```

HTTP clients also accept the same token. The `token=` argument works in older SDK versions; SDK 0.1.4 adds the explicit `api_token=` alias. Pass one of these arguments, not both.

To manage tokens from SDK 0.1.4, first sign in using `client.login(...)`, then call:

- `create_api_token(name, expires_in_days=0)` returns `{ "apiToken": {...}, "token": "lml_..." }` once. Zero days means no automatic expiry; 1–3650 days sets an expiry.
- `list_api_tokens()` returns metadata and token prefixes, never full secrets.
- `revoke_api_token(token_id)` disables an owned token on its next request.

API tokens inherit your current workspace permissions; removing membership removes access. They cannot create, list, or revoke credentials. Use a password-based login session to manage tokens. Store tokens like passwords; their presence does not add encryption to a plaintext HTTP/gRPC connection.

### Create and edit individual rows

Row content can live directly in PostgreSQL, without a bucket file. Inline content
is limited to 1 MiB per create/update request. File uploads remain available for
larger data. These methods require a server with the row-content update enabled.

```python
row = client.create_data_point(
    universe_id, "sample-001", "LLM_SFT",
    llm_data={"instruction": "Tag names", "input": "Hello Alice",
              "output": "Hello <NAME>Alice</NAME>",
              "metadata": {"spans": '[{"start": 6, "end": 11}]'}},
    labels={"split": "train"},
    metadata={"source": "manual"},
)
row = client.update_data_point(
    universe_id, row["id"],
    llm_data={"input": "Hello Bob", "output": "Hello <NAME>Bob</NAME>"},
)
collection = client.create_collection(universe_id, "Examples")
client.add_to_collection(universe_id, collection["id"], [row["id"]])
row = client.get_data_point(universe_id, row["id"])
for batch in client.stream_data_batch(universe_id, collection["id"], include_content=True):
    print(batch)
```

For text or arbitrary JSON rows, supply `raw_content=text.encode("utf-8")`.
For JSON, set `metadata={"content_type": "application/json"}`. Read responses
represent `rawContent` as base64; decode with `base64.b64decode(row["rawContent"])`.

Updates replace only supplied fields. Passing `metadata={}` or `labels={}` clears
that map; `raw_content=b""` saves an intentionally empty row. Replacing `llm_data`
replaces the whole structured example, so include every LLM field you want to keep.
Labels and both metadata maps use string values; encode nested annotations as JSON
strings. `GetDataPoint` returns inline content; list RPCs omit it to keep pages small.

RPCs: `DataService.CreateDataPoint`, `GetDataPoint`, `UpdateDataPoint` (with
`google.protobuf.FieldMask`), and `StreamDataBatch(include_content=true)`.
Collections use `CollectionService.AddDataToCollection`, `ListCollectionData`, and
`RemoveDataFromCollection`. Removing membership keeps the row itself. HTTP clients
use POST/GET/PATCH `/api/universes/{id}/data[/{row_id}]` and the collection endpoints.
