Metadata-Version: 2.3
Name: syvain-training-utils
Version: 0.0.205
Summary: Shared runtime, diagnostics, and checkpoint utilities for Syvain training runs
Requires-Dist: obstore>=0.11.0,<0.12.0
Requires-Dist: pydantic>=2.13.4
Requires-Dist: torch>=2.13.0
Requires-Dist: pytest>=8.0.0 ; extra == 'dev'
Requires-Dist: ruff>=0.15.12 ; extra == 'dev'
Requires-Dist: ty>=0.0.34 ; extra == 'dev'
Requires-Python: >=3.14, <3.15
Provides-Extra: dev
Description-Content-Type: text/markdown

# syvain-training-utils

Internal [Syvain](https://syvain.com/) helpers for small, explicit ML training
runs. No secret sauce here, just shared runtime, device-diagnostic, and
checkpoint patterns.

## Install

```bash
uv add syvain-training-utils
```

## Runtime setup

```python
from syvain_training_utils import (
    generate_run_id,
    require_torch_compile_toolchain,
    select_device,
)

run_id = generate_run_id()
device = select_device()
require_torch_compile_toolchain()
```

## Device smoke test

```python
import json

from syvain_training_utils import run_device_smoke_test

report = run_device_smoke_test(require_cuda=True)
print(json.dumps({"smoke": report}, indent=2, sort_keys=True))
```

## Checkpoint a training run

```python
from syvain_training_utils import (
    StorageConfig,
    TrainingLoopState,
    load_training_checkpoint_if_available,
    save_model_checkpoint,
)

storage_config = StorageConfig(
    bucket="my-training-bucket",
    s3_base_url="https://t3.storage.dev",
    region="auto",
    access_key_id="...",
    secret_access_key="...",
)
checkpoint_base_path = "models/my-model"

loop_state = TrainingLoopState(
    global_step=global_step,
    curriculum_stage=curriculum_stage,
    curriculum_step=curriculum_step,
)

save_model_checkpoint(
    storage_config=storage_config,
    base_path=checkpoint_base_path,
    experiment_slug=experiment_slug,
    run_id=run_id,
    model=model,
    optimizer=optimizer,
    scheduler=scheduler,
    loop_state=loop_state,
    checkpoint_label=f"step-{global_step:012d}",
)

resume = load_training_checkpoint_if_available(
    storage_config=storage_config,
    base_path=checkpoint_base_path,
    model=model,
    optimizer=optimizer,
    scheduler=scheduler,
    device=device,
)
```

The library owns the object-store clients. Every transient retry opens a fresh
client, and expired Tigris multipart sessions restart the complete upload at the
same checkpoint key. Checkpoint bodies are written through a temporary local
file, uploaded with an adaptive multipart size, and downloaded with resumable
range reads. Ensure the machine has temporary disk capacity for one checkpoint.

The manifest remains a pointer to the current checkpoint key within the
configured bucket and also records its byte size, ETag, and SHA-256 digest. The
manifest is published only after the checkpoint upload succeeds. Loading
verifies the complete digest before deserializing the model, optimizer, optional
scheduler, and PyTorch RNG state.

The storage transport uses a 10-second connect timeout, a 60-second
read-inactivity timeout, and a 10-minute overall request timeout. A bounded
15-minute outer no-progress retry window owns recovery and client replacement.
