Metadata-Version: 2.4
Name: triton-hf-builder
Version: 0.1.2
Summary: Build Triton model repositories from local Hugging Face checkpoints
License-Expression: Apache-2.0
Project-URL: Repository, https://github.com/ai-lab-tech/triton-hf-builder
Project-URL: Issues, https://github.com/ai-lab-tech/triton-hf-builder/issues
Project-URL: Documentation, https://github.com/ai-lab-tech/triton-hf-builder#readme
Keywords: triton,huggingface,inference,model-repository
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: packaging>=23
Requires-Dist: protobuf<7,>=6.33.5
Requires-Dist: tritonclient[grpc]<3,>=2.73
Provides-Extra: e2e
Requires-Dist: huggingface-hub<3,>=0.30; extra == "e2e"
Provides-Extra: dev
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: twine<7,>=6; extra == "dev"
Requires-Dist: ruff<1,>=0.9; extra == "dev"
Dynamic: license-file

# triton-hf-builder

[![CI](https://github.com/ai-lab-tech/triton-hf-builder/actions/workflows/tests.yml/badge.svg)](https://github.com/ai-lab-tech/triton-hf-builder/actions/workflows/tests.yml)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/LICENSE)
[![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/pyproject.toml)
[![Status: Beta](https://img.shields.io/badge/Status-Beta-blue.svg)](https://github.com/ai-lab-tech/triton-hf-builder#adapters-and-compatibility)

**Turn a downloaded Hugging Face checkpoint into a Triton model repository.**

The builder generates Triton configuration, runtime scripts and dependency
files for Transformers, vLLM and TensorRT-LLM. Use it through the Python API,
the `triton-hf` CLI, or the code-server extension in
[Triton Control](https://github.com/ai-lab-tech/triton-control).

[Python API](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/docs/python-api.md) · [Serving guide](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/docs/serving.md) ·
[Serving tests](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/scripts/README.md) · [Contributing](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/CONTRIBUTING.md)

## Install

Requires Python 3.10+.

```bash
python -m pip install triton-hf-builder
```

For development from a source checkout, see
[Contributing](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/CONTRIBUTING.md).

## Quick start

Point `source` to a complete local checkpoint with weights and tokenizer files.
Choose a new `output` directory outside the checkpoint directory.

Generation copies the checkpoint assets without loading weights and runs without
a GPU. Allow disk space for a complete checkpoint copy.

```python
from triton_hf_builder import generate_repository

result = generate_repository(
    source="/workspace/hf-checkpoint",
    output="/workspace/model_repository",
    name="my_model",
    runtime="transformers",
    image="nvcr.io/nvidia/tritonserver:26.06-py3",
    device="cpu",
)
print(result.model_directory / "config.pbtxt")
```

**No manual `requirements` needed:** the builder writes runtime defaults into
`requirements.txt`. Optional overrides, all parameters and return values are
explained in the [Python API](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/docs/python-api.md).

The equivalent CLI:

```bash
triton-hf inspect /workspace/hf-checkpoint
triton-hf build /workspace/hf-checkpoint \
  --output /workspace/model_repository --name my_model \
  --runtime transformers --device cpu \
  --image nvcr.io/nvidia/tritonserver:26.06-py3
```

## Generated output

For the Transformers example above, a checkpoint with a single Safetensors weight
file produces the following layout. Checkpoint filenames vary with the source.

```text
model_repository/
├── my_model/
│   ├── config.pbtxt                 # Triton model configuration
│   └── 1/
│       ├── model.py                 # Triton Python adapter
│       ├── settings.json            # Task and generation settings
│       └── model/
│           ├── config.json
│           ├── model.safetensors    # Copied checkpoint weights
│           ├── tokenizer.json
│           ├── README.md            # Original model card, if supplied
│           └── LICENSE              # Original model license, if supplied
├── requirements.txt                # Serving dependencies
├── serving.json                    # Runtime, image and build metadata
├── LICENSE.triton-hf-builder        # License for generated support scripts
└── README.md                       # Serving instructions
```

TensorRT-LLM uses the same layout with its own Python adapter. For vLLM,
`my_model/1/model.json` replaces `model.py` and `settings.json`, and
`prepare_repository.py` is added at the repository root.

Deploy the `model_repository/` root to Triton after preparing its serving
dependencies; generation alone does not validate inference.
See the [serving guide](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/docs/serving.md) for deployment and vLLM path preparation.

## Adapters and compatibility

| Builder adapter (`runtime`) | Triton backend | Model execution | Tasks | Device |
| --- | --- | --- | --- | --- |
| `transformers` | Python | Hugging Face Transformers with PyTorch | Generation, text-to-text, classification, raw features | CPU or GPU |
| `vllm` | vLLM | vLLM engine | Text generation | GPU |
| `tensorrt-llm` | Python | TensorRT-LLM LLM API with PyTorch backend | Text generation | GPU |

The API parameter `runtime` selects the builder adapter. The Triton backend and
model execution framework are separate layers.

Recognized multimodal checkpoints and custom model Python are outside the current
adapter scope. See [runtime limitations](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/docs/serving.md#runtimes-and-request-contracts).

### Manufacturer compatibility catalog

<!-- compatibility:begin -->

Architecture support reported by the backend projects; individual models may require additional configuration.

| Triton image | Backend | Supported architectures |
| --- | --- | --- |
| `26.09-vllm-python-py3` | vLLM | [118 candidates](https://github.com/vllm-project/vllm/blob/98dff2a81d747d1dba01a47f939f48c3526d4206/docs/models/supported_models.md) † |
| `26.06-vllm-python-py3` | vLLM | [129 candidates](https://github.com/vllm-project/vllm/blob/0decac0d96c42b49572498019f0a0e3600f50398/docs/models/supported_models.md) † |
| `26.07-trtllm-python-py3` | TensorRT-LLM | [18 candidates](https://github.com/NVIDIA/TensorRT-LLM/blob/376f7e1bd8ed543f75014309e3fd4b237e9b0e73/docs/source/models/supported-models.md) |
| `26.06-trtllm-python-py3` | TensorRT-LLM | [18 candidates](https://github.com/NVIDIA/TensorRT-LLM/blob/376f7e1bd8ed543f75014309e3fd4b237e9b0e73/docs/source/models/supported-models.md) |

Source: [NVIDIA image matrix](https://github.com/triton-inference-server/server/blob/eb74dea5d400b92f54eacbd5f3f84903c786e4cd/docs/introduction/compatibility.md). † Upstream base version; NVIDIA build differs.

[Full catalog](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/compatibility.json): architecture names, exact engine versions, CUDA and reported driver versions. See the linked backend documentation for quantization, GPU and feature requirements.

<!-- compatibility:end -->

The catalog tracks 26.06 and the latest documented image per runtime. A weekly
workflow proposes updates as PRs. [Maintenance and selection policy](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/scripts/README.md#compatibility-catalog-maintenance).

## Try it with Docker

Run a complete CPU serving example from a repository checkout:

```bash
./examples/docker/quickstart.sh
```

The launcher downloads a small pinned checkpoint, generates its model repository,
builds the serving image, waits for readiness and sends an inference request.
No GPU or local Python installation is needed. The first run downloads large
images and dependencies; later runs reuse cached files and build layers.
See the [Docker quickstart](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/examples/docker/README.md)
for requirements, requests and cleanup.

## Development and releases

See [CONTRIBUTING.md](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/CONTRIBUTING.md) for local checks and
[RELEASING.md](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/RELEASING.md) for package publication and code-server integration.
Real GPU serving tests are opt-in; follow the [test guide](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/scripts/README.md).

[Apache-2.0 license](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/LICENSE) · [Licensing and test models](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/docs/licensing.md) ·
[Security policy](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/SECURITY.md) · [Code of Conduct](https://github.com/ai-lab-tech/triton-hf-builder/blob/main/CODE_OF_CONDUCT.md)
Model weights and serving engines retain their own licenses.
