Documentation

Welcome to the Subvocal SDK documentation. Hardware-agnostic middleware connecting surface electromyography (sEMG) bio-sensors directly to LLM-driven AI agents — offline, auditable, and open vocabulary.

Traditional silent speech locks developers to proprietary neckbands or 5–8 word vocabularies. Subvocal provides the software rails — signal conditioning, training skeletons, articulatory shorthand simulators, and context-aware decoders — for high-accuracy, low-latency control.

Setting up

Get your pipeline or evaluation harness running in minutes.

Make it yours

Customize silent speech input to your hardware, models, and intent rules.

Repository Structure

subvocal/
├── src/subvocal/           # pip install subvocal
│   ├── core/               # pipeline, models, interfaces, LLM providers, security
│   ├── hardware/           # HAL drivers + dataset loaders (Ninapro/PutEMG/CSL-HDEMG/Gaddy/MetaEMG)
│   ├── emg_core/           # dsp (filters/handcrafted 112/spd), ml (adaptor/spd_gru/mona/lisa/speechnet), foundation (tinymyo/aemg_tokenizer/spectre), adaptation (sal_lbn/cpep/variance_transfer), benchmarks (emgbench)
│   ├── shorthand/          # vocab, simulator, hybrid decoder (asymmetric Levenshtein)
│   ├── context/            # schemas + phonetic matching
│   ├── mcp/                # stdio Model Context Protocol server
│   └── tts/                # multi-backend TTS
├── tests/                  # pytest (65% floor)
├── benchmarks/             # 50-case eval harness + EMGBench LOSO
├── tools/                  # build_site.py / build_api_page.py
└── docs/                   # GitHub Pages — mercor-inspired black/white

Quickstart Guide

Get a complete silent speech pipeline running offline in three steps.

Step 1: Install the SDK

Base install is lightweight (pydantic + numpy) and covers pipeline, HAL, shorthand, context, and MCP server:

pip install subvocal

Optional extras:

pip install "subvocal[ml]"        # classifier training & inference (torch, scikit-learn)
pip install "subvocal[hardware]"  # dataset drivers (Ninapro, PutEMG, CSL-HDEMG)
pip install "subvocal[tts]"       # audio outside macOS
pip install "subvocal[all]"       # everything

Step 2: Run an end-to-end pipeline

Full pipeline — synthetic sEMG source, classification, intent reconstruction, action execution — with no hardware or API keys:

from subvocal import SubvocalPipeline
from subvocal.core.testing import MockActionExecutor, MockContextProvider, MockLLMProvider
from subvocal.hardware.drivers import SyntheticSignalGenerator
from subvocal.core.models import CommandToken
import time

hardware = SyntheticSignalGenerator(fs=1000.0, num_channels=8)

def classify(frame):
    arr = frame.to_numpy()
    if abs(arr).max() > 1.0:
        return CommandToken(text="gt", confidence=0.95, timestamp=time.time())
    return None

pipeline = SubvocalPipeline(
    hardware=hardware,
    classify_fn=classify,
    llm_provider=MockLLMProvider(),
    context_provider=MockContextProvider(),
    executor=MockActionExecutor(),
    phrase_timeout_seconds=0.5,
)

hardware.start()
hardware.trigger_command("gt", duration_ms=120)
for _ in range(30):
    action = pipeline.step(window_ms=50)
    if action:
        print("Executed:", action.action_type, action.params)
        break
    time.sleep(0.05)

Prefer pipeline.inject_token(token) for live injection — thread-safe (deque + lock) with backpressure.

Step 3: Execute target reconstruction benchmarks

git clone https://github.com/PranavKalkunte/subvocal.git
cd subvocal
pip install -e ".[all,dev]"
python benchmarks/eval_runner.py

Local Development & Integration

Configure DSP, train models on your biopotentials, and customize shorthand alignment.

1. Custom Signal Preprocessing (DSP)

Raw signals ingest at 250 Hz. Conditioning in subvocal.emg_core.dsp.filters:

• 1.3–50.0 Hz bandpass (AlterEgo-inspired) for slow articulatory gestures.
• 60 Hz notch (configurable to 50 Hz) for AC line noise — notch_freq correctly plumbed.
• TD10 extraction in subvocal.emg_core.dsp.features → 840-d per segment window.

Research DSP: dsp/handcrafted 112-d (temporal 11 + stats 7 + spectral 10 → 112 for 4ch), dsp/spd manifold logm (50 ms / 20 ms windows).

2. Training Custom Classifiers

Skeletons in subvocal.emg_core.ml.train (requires [ml]):

• Random Forest — lightweight heuristic baseline.
• 1D CNN — 1.66 ms, 102k params, 409.7 KB — temporal conv over multi-channel traces.
• GRU — 8.27 ms, 105k params — sequence model over feature frames.
• Transformer + SpeechNet 15k — attention encoder; SpeechNet ~15k params, 63.9 µJ on GAP9.

# Train a 1D CNN on calibration data
from subvocal.emg_core.ml.train import train_model
from subvocal.emg_core.ml.config_schema import TrainingConfig

config = TrainingConfig(model_type="cnn", epochs=20, batch_size=16, lr=1e-3, test_size=0.2)
metrics = train_model("your_user_id", model_type="cnn", config_obj=config)
print(metrics["accuracy"])

3. Live Inference & Cooldown Gating

InferenceEngine in subvocal.emg_core.ml.infer applies confidence thresholds + adaptive cooldown (ignore within 500 ms) to prevent duplicates. Pipeline uses bounded OpsQueue(maxsize) with is_full/qsize for backpressure.

4. Articulatory Shorthand Decoder

Compressed consonant shorthand (e.g., g gl for Google). Decoder in subvocal.shorthand.decoder aligns via custom dynamic programming (Asymmetric Levenshtein) with confusion groups in subvocal.shorthand.spec:

• Labial: p, b, m, f, v
• Alveolar: t, d, s, z, n, l
• Velar: k, g, ng
• Rhotic: r

Update vocab in subvocal.shorthand.vocab.

5. Zero-Dependency TTS Engine

subvocal.tts.engine defaults to macOS say / afplay — offline audio confirmations with flag-injection sanitization.

MCP Integration

Stdio Model Context Protocol server so Claude Desktop (or any MCP client) can ingest subvocal commands as tools. Installs as:

subvocal-mcp

Claude Desktop configuration

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "subvocal": { "command": "subvocal-mcp" }
  }
}

Exposed tools

• get_pipeline_status — hardware, classifier, buffer state.
• get_token_buffer — accumulated tokens awaiting phrase reconstruction.
• inject_token — thread-safe pipeline.inject_token() (session lock) for testing without hardware.
• process_phrase — force immediate intent reconstruction + dispatch.
• trigger_calibration — per-user calibration run (requires [ml]).

Resources

Server exposes subvocal://intent/history JSON log of recent intents.