Metadata-Version: 2.5
Name: pipecat-hecttor
Version: 0.2.0
Summary: Hecttor speech enhancement audio filter for Pipecat
Project-URL: Homepage, https://hecttor.ai
Project-URL: Repository, https://github.com/Saima-AI/pipecat-hecttor
Project-URL: Issues, https://github.com/Saima-AI/pipecat-hecttor/issues
Project-URL: Changelog, https://github.com/Saima-AI/pipecat-hecttor/blob/main/CHANGELOG.md
License-Expression: BSD-2-Clause
License-File: LICENSE
Keywords: audio,denoising,hecttor,noise-cancellation,pipecat,speech-enhancement
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: loguru~=0.7.3
Requires-Dist: numpy<3,>=1.26.4
Requires-Dist: pipecat-ai<2,>=1.5.0
Description-Content-Type: text/markdown

# pipecat-hecttor

[Hecttor](https://hecttor.ai) speech enhancement for [Pipecat](https://github.com/pipecat-ai/pipecat).

Two integrations are provided:

- `HecttorFilter` — an input audio filter that removes background noise from the user's audio in real time using the Hecttor SDK's ASR-optimized speech enhancer. It runs before audio reaches the STT service, improving transcription accuracy in noisy environments. You can choose among several enhancement models and blend the enhanced output with the original audio.
- `HecttorAudioProcessor` — a frame processor that enhances the input audio with **two different blend factors**: one for the STT/agent path and one for VAD and turn-taking (TT) models, since the optimal blend for transcription is usually not the optimal blend for endpointing.

> This integration is maintained by [Saima AI](https://hecttor.ai), the company behind Hecttor.

## Installation

Install the package:

```bash
uv add pipecat-hecttor
```

The filter requires the `hecttor_sdk` Python package, which is **not published to PyPI**. Contact [Hecttor](https://hecttor.ai) for SDK access and an API key — you'll receive a wheel for your platform and Python version:

```bash
pip install hecttor_sdk-<version>-<python>-<platform>.whl
```

Set your API key:

```bash
export HECTTOR_API_KEY=your_api_key_here
```

## Usage

Add the filter to any Pipecat transport via the `audio_in_filter` parameter:

```python
from pipecat_hecttor import HecttorFilter
from pipecat.transports.base_transport import TransportParams

params = TransportParams(
    audio_in_enabled=True,
    audio_in_filter=HecttorFilter(),
    audio_out_enabled=True,
)
```

### Configuration

| Parameter | Default | Description |
| --- | --- | --- |
| `api_key` | `None` | Hecttor API key. Falls back to the `HECTTOR_API_KEY` environment variable. |
| `model_name` | `"coda-vi-1.0"` | ASR enhancement model: `crest-1.0`, `crest-2.0`, `mist-1.0`, `coda-1.0`, or `coda-vi-1.0`. |
| `chunk_size_ms` | `20` | Chunk size in milliseconds, `16` or `20`. `crest-2.0`, `coda-1.0`, and `coda-vi-1.0` require `20`. |
| `enhancer_weight` | `None` | Blend factor `[0.0, 1.0]` between original (`0.0`) and enhanced (`1.0`) audio. `None` uses the model's default. |

## Per-consumer blends: `HecttorAudioProcessor`

When you want STT and the VAD/turn-taking models to hear differently blended audio, use `HecttorAudioProcessor` instead of the transport filter (never both at once). It exploits Pipecat's pipeline ordering: STT consumes audio before the user context aggregator, which hosts the VAD and turn analyzers. Stage 1 sits after the transport input and rewrites frames with the ASR blend; stage 2 (`vad_tt_stage()`) sits after STT and swaps in the VAD/TT blend:

```python
from pipecat_hecttor import HecttorAudioProcessor

hecttor = HecttorAudioProcessor(asr_weight=1.0, vad_tt_weight=0.5)

pipeline = Pipeline(
    [
        transport.input(),       # no audio_in_filter
        hecttor,                 # stage 1: frames now carry the ASR blend
        stt,                     # hears the ASR blend
        hecttor.vad_tt_stage(),  # stage 2: swaps in the VAD/TT blend
        user_aggregator,         # VAD + turn analyzers hear the VAD/TT blend
        llm,
        tts,
        transport.output(),
        assistant_aggregator,
    ]
)
```

`HecttorAudioProcessor` accepts the same `api_key`, `model_name`, and `chunk_size_ms` parameters as `HecttorFilter`, plus `asr_weight` and `vad_tt_weight` blend factors in `[0.0, 1.0]` (`None` uses the model's default weight).

Notes:

- The current implementation runs **two enhancer sessions**, one per weight, doubling enhancement compute and SDK usage accounting. A single-pass multi-weight SDK API may replace this later.
- Anything placed downstream of `vad_tt_stage()` (e.g. audio recorders) sees the VAD/TT blend.

### Runtime toggle

Disable and re-enable denoising at runtime with Pipecat's `FilterEnableFrame`:

```python
from pipecat.frames.frames import FilterEnableFrame

await worker.queue_frame(FilterEnableFrame(False))  # disable
await worker.queue_frame(FilterEnableFrame(True))  # re-enable
```

## Running the example

[`examples/voice-hecttor.py`](examples/voice-hecttor.py) is a complete voice bot with Hecttor denoising, Deepgram STT, OpenAI LLM, and Cartesia TTS.

1. Install the example's dependencies:

   ```bash
   uv add pipecat-hecttor "pipecat-ai[deepgram,cartesia,openai,silero,webrtc,runner]"
   ```

2. Install the `hecttor_sdk` wheel (see [Installation](#installation)).

3. Create a `.env` file with your keys:

   ```
   HECTTOR_API_KEY=...
   DEEPGRAM_API_KEY=...
   OPENAI_API_KEY=...
   CARTESIA_API_KEY=...
   ```

4. Run the bot and open the printed URL in your browser:

   ```bash
   python examples/voice-hecttor.py
   ```

## Testing the filter offline

[`scripts/test_hecttor_filter_audiofile.py`](scripts/test_hecttor_filter_audiofile.py) runs the filter over a pre-recorded audio file so you can compare original and enhanced audio:

```bash
uv add soundfile
python scripts/test_hecttor_filter_audiofile.py input.wav output.wav
```

It reports the realtime factor and before/after audio statistics.

## Testing the processor offline

[`scripts/test_hecttor_processor_audiofile.py`](scripts/test_hecttor_processor_audiofile.py) runs the full two-stage `HecttorAudioProcessor` pipeline over a pre-recorded audio file — including a real Silero VAD on the swapped stream when `pipecat-ai[silero]` is installed — and verifies the processor's invariants, exiting non-zero on any violation:

```bash
uv add soundfile
python scripts/test_hecttor_processor_audiofile.py input.wav --asr-weight 1.0 --vad-tt-weight 0.3
```

It saves both blends (`asr_blend.wav`, `vad_tt_blend.wav`) so you can compare them by ear, and reports frame counts, VAD events, and the realtime factor.

## Compatibility

Tested with Pipecat v1.7.0.

Pipecat evolves rapidly; if you hit a compatibility issue with a newer release, please [open an issue](https://github.com/Saima-AI/pipecat-hecttor/issues).

## License

BSD 2-Clause — see [LICENSE](LICENSE).
