Metadata-Version: 2.4
Name: kitten-tts-cpp
Version: 0.1.0
Summary: CPU KittenTTS2 inference with native C++ bindings
Home-page: https://github.com/KittenML/kitten-tts-2-cpp
Author: KittenML
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS
Requires-Dist: numpy>=1.26
Requires-Dist: torch==2.13.0
Requires-Dist: soundfile>=0.12
Requires-Dist: certifi>=2024.7.4
Dynamic: author
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# KittenTTS2 C++

CPU-focused C++ inference for KittenTTS2, with built-in voices, expression controls, and prepared reference voices. The runtime downloads prepared model assets and runs synthesis natively in C++. Users do not need to run the Python exporter.

The speech language model runs through llama.cpp/GGML, text normalization uses the [kitten-text-processing](https://github.com/KittenML/kitten-text-processing) submodule, and the S3 waveform decoder uses CPU LibTorch. Only KittenTTS2 checkpoints are supported.

## Features

- Built-in voices, including Bruno, Bella, Jasper, Luna, Rosie, Hugo, Kiki, and Leo.
- Emotion tags, vocal events, and emphasis: `[joyful]`, `<laugh>`, and `(((really)))`.
- Stable and expressive sampling presets, sentence-aware chunking, and audio joins.
- A 1.03 GB GGUF with lossless packing of the checkpoint's ternary weights, plus an FP16 reference export.
- Default S3 and optional `student_w4` / `student_w8` decoder exports.
- Mono float32 WAV output at 24 kHz.

## Using Python

The `kitten_tts_cpp` package provides a KittenTTS-style Python API backed by the native CPU engine. Models and decoders stay loaded between calls, and prepared assets download automatically on first use. No model export is needed.

### Install

```sh
pip install kitten_tts_cpp
```

### Generate speech

```python
from kitten_tts_cpp import KittenTTS

model = KittenTTS("KittenML/kitten-tts-2", decoder="student_w4", threads=8)
model.generate_to_file("Hello from Kitten T T S!", "hello.wav", voice="Bruno")
print(model.last_report)  # Generation timings and token IDs.
```

Pass the Hugging Face model ID as the first argument, or use `model_name="KittenML/kitten-tts-2"`. Reuse the model for repeated generations and call `model.close()` when finished to release its memory.

This writes a 24 kHz mono WAV file. Choose `decoder="default"`, `"student_w4"`, or `"student_w8"` when loading the model. Use `threads` to control CPU inference threads; `decoder_threads` can set a separate decoder thread count. For private model repositories, set `HF_TOKEN` in your environment or pass `hf_token=` to the constructor.

To get a NumPy array, select a voice, and control sampling:

```python
import soundfile as sf
from kitten_tts_cpp import KittenTTS

model = KittenTTS("KittenML/kitten-tts-2", decoder="student_w4", threads=8)
print(model.available_voices)
print(model.available_decoders)
audio = model.generate(
    "[joyful] Hello again!",
    voice="Bruno",
    preset="expressive",
    advanced={"seed": 1234},
)
sf.write("expressive.wav", audio, model.sample_rate)
```

`generate()` returns a mono float32 NumPy array. Use `preset="stable"` or `"expressive"`; generation also accepts `temperature`, `top_k`, `top_p`, `min_p`, and `max_new_tokens`. `KittenTTS2` and `get_model` are also exported.

### Stream sentence chunks

```python
import soundfile as sf
from kitten_tts_cpp import KittenTTS

model = KittenTTS("KittenML/kitten-tts-2", decoder="student_w4")
for index, chunk in enumerate(model.generate_stream("A longer passage goes here.", voice="Bruno")):
    sf.write(f"chunk-{index}.wav", chunk, model.sample_rate)
```

Streaming yields untrimmed sentence chunks as they are decoded. Close the iterator when stopping early. Use `generate()` or `generate_to_file()` for the normal chunk joins in one complete recording.

The constructor also accepts `cache_dir`, `revision`, `offline`, `repack`, `model_path` (a local GGUF override), `data_dir`, and `ca_file` (a custom HTTPS trust bundle). A local prepared asset directory can replace the repository ID. Native inference releases the GIL, but calls are serialized because the TorchScript decoder uses process-wide random and thread settings.

Only KittenTTS2 CPU inference and `weights="packed"` are supported. Raw-audio `reference=` cloning is not implemented; prepare a reference with [prepare_reference.py](tools/kitten-tts/prepare_reference.py) and select its voice name. Matching seeds do not guarantee identical sampled output to the original PyTorch backend.

### Build the Python package from source

For a build optimized for your local CPU:

```sh
git clone --branch py-interface --recurse-submodules https://github.com/KittenML/kitten-tts-2-cpp.git
cd kitten-tts-2-cpp
python -m pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cpu
python -m pip install setuptools wheel cmake numpy soundfile certifi
python -m pip install . --no-build-isolation
```

`setup.py` invokes CMake during installation and bundles the normalizer grammar data. A C++20 compiler and OpenSSL development headers are required (`libssl-dev` on Debian/Ubuntu). Set `CMAKE_BUILD_PARALLEL_LEVEL=8` to control build parallelism. `CMAKE_ARGS` accepts extra CMake flags, such as `-DOPENSSL_ROOT_DIR=/path/to/openssl`.

Source builds optimize for the local CPU and require the exact PyTorch release used at build time and the linked system OpenSSL libraries. The portable wheels use baseline CPU kernels and statically linked LibreSSL. See the [portable build instructions](packaging/README.md#building) when distributing wheels to other machines.

For development, `python setup.py build_ext --inplace` builds the extension; set `PYTHONPATH=python` to import it directly. `python setup.py bdist_wheel` produces an installable wheel, and `python setup.py sdist` includes the native sources and normalizer submodule contents.

## Build

Requires a C++20 compiler, CMake, and CPU LibTorch. A CPU PyTorch installation can provide LibTorch's headers and libraries. Use the same LibTorch version for export and inference, with compatible torch/torchaudio versions for asset preparation.

```sh
git submodule update --init vendor/kitten-text-processing

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_STANDARD=20 \
  -DCMAKE_CXX_STANDARD_REQUIRED=ON \
  -DGGML_CUDA=OFF -DGGML_METAL=OFF \
  -DLLAMA_BUILD_KITTEN_TTS=ON \
  -DLLAMA_BUILD_SERVER=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DCMAKE_PREFIX_PATH="$(python -c 'import torch; print(torch.utils.cmake_prefix_path)')"

cmake --build build --target kitten-tts -j 8
```

The normalizer prepares its grammar data using Python at build time. To use existing grammar data, configure with `-DKITTEN_DATA_DIR=/path/to/data`.

## Download and run

The runtime fetches `config.json` from `KittenML/kitten-tts-2` first, then downloads its GGUF, selected TorchScript decoder, and matching voice conditioning. Assets are cached under `$XDG_CACHE_HOME/kitten-tts` (or `~/.cache/kitten-tts`).

```sh
build/bin/kitten-tts --text 'Hello from Kitten T T S.' --output hello.wav
build/bin/kitten-tts --decoder student_w4 --text 'Hello again.' --output w4.wav
```

Use `--repo OWNER/REPO` to select another KittenTTS2 repository, `--revision COMMIT` to pin a model version, `--cache-dir DIR` to change the cache location, and `--offline` to use cached files without network access. `HF_TOKEN` supports private repositories. `--download-only --decoder student_w4` prepares the cache without synthesis. Only assets needed for the selected mode are downloaded.

The model repository must publish a `cpp` manifest and prepared files as described in the [publisher guide](tools/kitten-tts/README.md#publish-native-assets). Local exports and model repository directories remain supported through `--assets DIR`; `--model FILE` overrides the GGUF.

The default GGUF uses this fork's `TQ2_1` format: 256 ternary weights packed into 64 bytes, plus two FP16 scales for the checkpoint's 128-weight groups. The exporter requires exact reconstruction. This reduces the model from 1.45 GB with Q4_0 to 1.03 GB. Embeddings remain FP16 and one-dimensional tensors remain FP32. GGML's quantized activation arithmetic can still introduce small logit differences.

`TQ2_1` requires this fork's CPU runtime; it is not standard `TQ2_0` and stock llama.cpp cannot load it. Use `--outtype ternary-q4_0` for the previous standard Q4_0 storage format, or `--outtype f16` for the reference export.

On supported Intel CPUs, the runtime losslessly repacks TQ2_1 weights into the existing AMX Q4_0 layout at load time. The GGUF stays 1.03 GB, but runtime transformer-weight storage grows from about 357 MiB to 756 MiB. Use `--no-repack` to keep the compact CPU representation. CPUs without AMX use the existing compact kernels.

See the [detailed guide](tools/kitten-tts/README.md) for FP16 exports, student decoders, reference voice preparation, and the normalizer submodule setup.

## Generate speech

```sh
build/bin/kitten-tts \
  --text 'Hello there. This is Kitten T T S running on the CPU.' \
  --voice Bruno --threads 8 --seed 1234 \
  --output hello.wav --report hello.json
```

For expressive speech:

```sh
build/bin/kitten-tts \
  --text '[joyful] We won the grant <laugh> I can (((hardly))) believe it!' \
  --voice Kiki --preset expressive \
  --output expressive.wav
```

Use `--help` for sampling, repetition penalties, chunking, and decoder thread settings. `--tokens-only` skips waveform decoding; `--repeat 3` benchmarks repeated synthesis with the model and decoder kept loaded.

## Python parity and CPU performance

The implementation is checked against `kittenml.kittentts2`:

| Check | Recorded result |
| --- | --- |
| Text normalization and chunking | 228 cases passed |
| Sampling rules | 12 cases passed |
| Audio joins | 3 cases passed |
| Packed weights | All 310 tensors reconstruct the FP16 reference exactly |
| Native waveform decoding | Sample-for-sample agreement with Python in tested cases for all three decoders |
| Packed LM logits | Close numerical agreement with the Python reference |

On a Xeon Platinum 8462Y+ with eight pinned CPU cores, the AMX path reduced a 256-token forced-sequence workload from **6.40 to 5.60 seconds** (about **14% higher throughput**). Prompt evaluation fell from **1.09 to 0.58 seconds**. TQ2_1 reached **45.7 generated tokens/second**, comparable to Q4_0 at **44.3** in the same test. These are medians of three warmed runs, including prompt evaluation and sampling but excluding model loading. Performance varies with hardware and workload.

Run the differential checks with:

```sh
python tools/kitten-tts/check_parity.py \
  --binary build/bin/kitten-tts \
  --python-package ../KittenTTS \
  --repo ../kitten-tts-2 --assets models/kitten2 \
  --model models/kitten2/model-tq2_1.gguf \
  --lm --decoder
```

Measurements and source revisions are recorded in [validation.json](tools/kitten-tts/validation.json).

## Current scope

Sampled output is not bit-identical to Python: BF16 arithmetic, GGML activation quantization, and sampling RNGs differ. Matching seeds do not guarantee matching generated tokens across runtimes.

Voice cloning requires an offline Python preparation step and a reference transcript. The Python interface supports sentence-chunk streaming; sample-by-sample streaming is not implemented. Waveform decoding currently depends on CPU LibTorch.

## Credits

KittenTTS2 and kitten-text-processing are from KittenML. See the [implementation guide](tools/kitten-tts/README.md#attribution) for decoder attribution and license details.

Built on [DeepGrove's llama.cpp fork](https://github.com/deepgrove-ai/llama.cpp).
