Metadata-Version: 2.3
Name: bentoml-unsloth
Version: 0.1.2
Summary: BentoML: The easiest way to serve AI apps and models
Project-URL: Homepage, https://bentoml.com
Project-URL: Documentation, https://docs.bentoml.com
Project-URL: GitHub, https://github.com/bentoml/bentoml
Project-URL: Twitter, https://twitter.com/bentomlai
Project-URL: Tracker, https://github.com/bentoml/BentoML/issues
Project-URL: Slack, https://l.bentoml.com/join-slack
Project-URL: Blog, https://bentoml.com/blog
Author-email: BentoML Team <contact@bentoml.com>
License: Apache-2.0
Keywords: BentoML,Compound AI Systems,LLMOps,MLOps,Model Deployment,Model Inference,Model Serving
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.9
Requires-Dist: bentoml>=1.3.4
Requires-Dist: fastapi>=0.114.0
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: unsloth[huggingface]>=2024.9
Requires-Dist: vllm>=0.6.0
Description-Content-Type: text/markdown

## BentoML Unsloth integrations

## Installation

```bash
pip install "bentoml[unsloth]"
```

## Examples.

See [train.py](https://github.com/bentoml/bentoml-unsloth/blob/main/train.py)

## API

To use this integration, one can use `bentoml.unsloth.build_bento`:

```python
bentoml.unsloth.build_bento(model, tokenizer)
```

If you model is continued froma fine-tuned checkpoint, then `model_name` must be passed as well:

```python
bentoml.unsloth.build_bento(model, tokenizer, model_name="llama-3-continued-from-checkpoint")
```

> [!important]
>
> Make sure to save the chat templates to tokenizer instance to make sure generations are correct based on how you setup your data pipeline.
> See [example](https://github.com/bentoml/bentoml-unsloth/blob/da52d51366ea3217a3ee644f80042b1f425e00c6/train.py#L42) and [documentation](https://huggingface.co/docs/transformers/main/en/chat_templating#advanced-adding-and-editing-chat-templates) for more information.
