Metadata-Version: 2.1
Name: cartesia-mlx
Version: 0.0.1rc1
Summary: The official Cartesia MLX library.
Home-page: https://github.com/cartesia-ai/edge
Author: Cartesia, Inc.
Author-email: support@cartesia.ai
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8.0
Description-Content-Type: text/markdown
Requires-Dist: mlx (==0.16.1)
Requires-Dist: transformers
Requires-Dist: setuptools (>=42)
Requires-Dist: ai2-olmo
Requires-Dist: datasets
Requires-Dist: torch
Requires-Dist: numpy (<2)
Requires-Dist: requests
Requires-Dist: huggingface-hub


# Cartesia MLX

This package contains implementations for fast on-device SSM inference on Apple silicon. 

## Installation
This package requires the `cartesia-metal` package.
To install this package, first install Xcode which can be downloaded from https://developer.apple.com/xcode/.
Accept the license agreement with:

```shell 
sudo xcodebuild -license
```

Install metal extension:
```shell 
pip install cartesia-metal
```

Install the package:
```shell 
pip install cartesia-mlx
```

## Models

### Language Models
- `cartesia-ai/Rene-v0.1-1.3b-4bit-mlx` 
- `cartesia-ai/mamba2-130m-8bit-mlx` 
- `cartesia-ai/mamba2-130m-mlx` 
- `cartesia-ai/mamba2-370m-8bit-mlx` 
- `cartesia-ai/mamba2-780m-8bit-mlx` 
- `cartesia-ai/mamba2-1.3b-4bit-mlx` 
- `cartesia-ai/mamba2-2.7b-4bit-mlx` 


## Usage:
```python 
import mlx.core as mx
import cartesia_mlx as cmx

model = cmx.from_pretrained('cartesia-ai/mamba2-130m-8bit-mlx')
model.set_dtype(mx.float32)   

prompt = "Rene Descartes was"

print(prompt, end="", flush=True)
for text in model.generate(
    prompt,
    max_tokens=500,
    eval_every_n=5,
    verbose=True,
    top_p=0.99,
    temperature=0.85,
):
    print(text, end="", flush=True)
```
