Metadata-Version: 2.4
Name: hypercache-io
Version: 0.9.1
Summary: HyperCache: Host-Local NVMe Caching and Zero-Copy Direct Block DMA / GPUDirect Storage Acceleration for AI Workloads
Author: Looe Labs
License: Apache-2.0
Project-URL: Homepage, https://github.com/looelabs/hypercache
Project-URL: Repository, https://github.com/looelabs/hypercache
Project-URL: Issues, https://github.com/looelabs/hypercache/issues
Keywords: ai,pytorch,safetensors,vllm,nvme,gpudirect,cache,s3,fsspec
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Filesystems
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: fsspec>=2023.1.0
Requires-Dist: numpy>=1.22.0
Provides-Extra: gpu
Requires-Dist: kvikio-cu12>=24.0.0; extra == "gpu"
Requires-Dist: cupy-cuda12x>=12.0.0; extra == "gpu"
Requires-Dist: torch>=2.0.0; extra == "gpu"
Requires-Dist: numpy>=1.22.0; extra == "gpu"
Provides-Extra: dev
Requires-Dist: torch>=2.0.0; extra == "dev"
Requires-Dist: safetensors>=0.4.0; extra == "dev"
Requires-Dist: boto3>=1.28.0; extra == "dev"
Requires-Dist: requests>=2.28.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: numpy>=1.22.0; extra == "dev"

# HyperCache Python Client & Data Plane (`client-py`)

The `hypercache` Python package integrates transparently into AI training and inference pipelines (PyTorch, Hugging Face, Safetensors) to provide **zero-copy Direct Block DMA / GPUDirect Storage (GDS)** checkpoint loading.

---

## Key Features

- **Transparent `hypercache://` Protocol**: Seamless `fsspec` filesystem implementation.
- **Dual Direct / Buffered Handles**: Uses `O_DIRECT` with sector/page-aligned DMA buffers for maximum hardware read throughput (up to **~7 GB/s** per PCIe Gen 4 NVMe and **~14 GB/s** per PCIe Gen 5 NVMe), with automatic transparent fallback for arbitrary unaligned byte slices.
- **Sub-Millisecond Metadata**: Resolves physical drive offsets from the local HyperCache control plane daemon in **< 1 ms** over Unix Domain Sockets.
- **Zero-Copy GPU Direct**: Direct DMA transfers bypassing host CPU and Linux page cache.

---

## Installation

```bash
# Install from PyPI (imports as `import hypercache`)
pip install hypercache-io

# Or install in editable mode from the repository
cd client-py
pip install -e .

# Optional GPU Direct Storage acceleration dependencies
pip install "hypercache-io[gpu]" --extra-index-url https://pypi.nvidia.com
```

---

## Usage Examples

### 1. PyTorch & Safetensors Loading
```python
import torch
import safetensors.torch
from hypercache.fsspec_fs import register_hypercache_fsspec

# Automatically registers hypercache:// protocol
register_hypercache_fsspec()

# Load tensors directly from striped NVMe storage into VRAM/Host memory
with open("hypercache://models/llama-3/weights.safetensors", "rb") as f:
    tensors = safetensors.torch.load(f.read())
```

### 2. Direct Python Client API
```python
from hypercache.client import HyperCacheClient
from hypercache.datapath import get_data_reader

# Connect to local Control Plane Daemon (auto-resolves /run/hypercache/hypercache.sock)
client = HyperCacheClient()

# Check daemon health & discovered drives
devices = client.get_devices()
print(f"Discovered {len(devices)} active striped devices.")

# Locate checkpoint
layout = client.locate("hypercache://models/llama-3/weights.safetensors", auto_ingest=True)
if layout.is_hit:
    reader = get_data_reader()
    data = reader.read_chunks(layout.chunk_layout, layout.object_size)
    print(f"Read {len(data)/(1024*1024):.2f} MiB at hardware line rate!")
```

---

## Running Unit Tests

```bash
pytest tests/
```
