Metadata-Version: 2.4
Name: edgespeak
Version: 0.6.8
Summary: Python client for the local EdgeSpeak CLI runtime
Project-URL: Documentation, https://edgespeak.com/docs/python-sdk
Project-URL: Source, https://github.com/lattifai/EdgeSpeak/tree/main/sdk/python
Project-URL: Issues, https://github.com/lattifai/EdgeSpeak/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: httpx<1,>=0.27
Provides-Extra: numpy
Requires-Dist: numpy>=1.23.5; extra == "numpy"
Provides-Extra: dev
Requires-Dist: mypy==1.20.2; extra == "dev"
Requires-Dist: typing_extensions>=4.6; extra == "dev"
Requires-Dist: numpy>=1.23.5; extra == "dev"

# EdgeSpeak Python SDK

Transcribe, align, segment text, identify speaker turns, and compare voice embeddings
through the local EdgeSpeak CLI. Audio is processed by the connected runtime;
the SDK does not bundle engines or models.

[简体中文](https://github.com/lattifai/EdgeSpeak/blob/main/sdk/python/README.zh-CN.md) · [Documentation](https://edgespeak.com/docs/python-sdk) · [Examples](https://github.com/lattifai/EdgeSpeak/tree/main/sdk/python/examples)

## Installation

Requires Python 3.10+. One pure-Python wheel serves all supported Python versions
and platforms. Install from PyPI:

```bash
python -m pip install edgespeak
```

The SDK discovers an installed CLI and automatically
installs the latest official stable CLI if none meets its requirements.
Automatic installation supports macOS Apple Silicon, Linux x86_64, and Windows x64
and requires network access. Models load on demand; first use may download a model.

## Create and configure an API KEY

1. Sign in to [EdgeSpeak → API KEY](https://edgespeak.com/account/api-keys).
2. Create an API KEY, choose an expiry (including never expires) and usage limit,
   and copy the full key shown at creation.
3. Set `EDGESPEAK_API_KEY` in the environment running Python:

```bash
export EDGESPEAK_API_KEY="your API KEY"
```

In Windows PowerShell, use `$env:EDGESPEAK_API_KEY="your API KEY"`.
`EdgeSpeak()` reads this variable automatically; you can also pass `EdgeSpeak(api_key=...)`.
An API KEY authorizes programmatic access and is different from a product activation
key. Keep it out of source control. An already activated local device can also
reuse its existing authorization.

API KEY access requires online authorization and shares the account subscription
allowance and billing period with desktop usage. Creating a key does not reset usage.
Never expires only removes the expiry date; entitlement, usage limits, and revocation
still apply.

## Quick start

```python
from edgespeak import EdgeSpeak

with EdgeSpeak() as client:
    transcript = client.transcribe("meeting.wav", language="en", diarize=True)
    print(transcript["text"])
    aligned = client.align("speech.wav", text="The exact words spoken.")
    speakers = client.diarize("meeting.wav", num_speakers=2)
```

Reuse a client and use `with` or `close()` to release its owned service.
Results are usually dictionaries with timestamps in seconds;
`response_format="text"` returns a string. Alignment needs the spoken words in
the original language. It also accepts `text_file="captions.srt"`, supporting
TXT, JSON, SRT, ASS, and SSA; existing subtitle timing is recomputed from the audio.

| Method | Purpose |
| --- | --- |
| `transcribe(audio, ...)` | Transcription, optional speaker labels and word timestamps |
| `align(audio, text=... / text_file=..., ...)` | Align a transcript or subtitles to audio |
| `segment(text, ...)` / `segment(segments=..., ...)` | Segment text or resegment timed captions |
| `diarize(audio, ...)` | Independent speaker diarization |
| `speaker_embedding(audio, ...)` | Single or batch voice embeddings |
| `speaker_similarity(a, b, ...)` | Compare two embeddings from the same model |

Typed options include `timestamp_granularities=["word", "segment"]` and
`search_effort="auto"`. Use `help(EdgeSpeak.transcribe)` or `help(EdgeSpeak.align)`
for parameter details. Alignment does not perform diarization; choose a speaker
count with the independent `diarize()` method.

## In-memory audio

`transcribe`, `align`, `diarize`, `diff_align`, `speaker_embedding` and `upsample` accept paths, complete encoded media `bytes`, NumPy arrays or `AudioSamples`.
For arrays install `python -m pip install "edgespeak[numpy]"`.

```python
from pathlib import Path
import numpy as np

with EdgeSpeak() as client:
    client.transcribe(Path("speech.mp3").read_bytes())  # complete encoded file
    audio = np.zeros((16000, 2), dtype=np.float32)      # (frames, channels)
    client.transcribe(audio, sample_rate=16000)
    client.align(audio, sample_rate=16000, text="Hello", audio_channel=1)
```

Arrays require their actual input `sample_rate` in Hz (1..384000); this never
resamples the data. Shape `(frames,)` is mono, `(frames, channels)` supports 1..8
channels in their original order. Channel-first arrays must be transposed explicitly.
Supported dtypes: int16, or finite float32/float64 within [-1, 1]. Floats are
rounded using `rint(x * 32767)` into PCM16 WAV; int16 samples are preserved.
No automatic mixing or speaker-position mapping occurs during conversion.
Transcription retains the gateway's default channel mixing; alignment can select
a zero-based `audio_channel` or `"mix"`. Array `audio_track` must be None or 0.

Encoded bytes carry their own rate/channel metadata; do not pass `sample_rate`
with bytes or paths. Raw PCM bytes, bytearray, memoryview, file objects and URLs
are not accepted. Convert raw samples to ndarray first. In-memory inputs are
limited to 256 MiB of encoded data by default (configurable); server upload/duration limits also apply.
Do not mutate an array while a call is in progress.
NumPy subclasses (such as masked arrays) must be explicitly converted to a plain ndarray.


Use `AudioSamples(array, sample_rate)` to carry an input rate per clip:

```python
from edgespeak import AudioSamples

client.speaker_embedding([AudioSamples(a, 16000), AudioSamples(b, 24000), encoded_wav])
client.diarize(a, sample_rate=16000)
client.upsample(a, input_sample_rate=16000, sample_rate=24000)
```

For `upsample`, `sample_rate` always means the **output rate**; bare arrays require
`input_sample_rate`. Other audio methods use `sample_rate` for the input rate.
Do not supply a duplicate input rate with `AudioSamples`. In embedding batches,
the top-level rate applies only to bare arrays; wrappers carry their own rates,
and encoded files/bytes use their headers. Response `data` order and `index`
match the input. The SDK does not split requests, retry or change billing rules.

### Memory limits

Both clients accept these byte-valued options:

| Parameter | Default | Meaning |
| --- | --- | --- |
| `max_memory_audio_bytes` | 256 MiB | Encoded size limit for each bytes/array input |
| `max_in_flight_audio_bytes` | 512 MiB | Per-client admitted memory-origin input budget, summed over a batch |
| `audio_spool_threshold_bytes` | `None` | In-memory by default; opt in to private temporary files for arrays above a threshold |

`None` disables either SDK size cap. Integers must be positive, excluding bool.
A single item or batch exceeding the total budget fails immediately; requests
wait when other uploads temporarily occupy it, including concurrent sync callers.
The budget and input resources are released when the upload body finishes, before
response-stream consumption. Client close wakes budget waiters. Paths are exempt;
spooled arrays still count. Files are cleaned up on failure/cancellation.
This is not an RSS limit: caller arrays, queued input, response data and model
memory are excluded. Producers still need bounded queues. Gateway limits remain;
`upsample` separately enforces 32 MiB, 120 seconds and 1..8 channels.

## Async usage

Runnable example: [async_transcribe.py](examples/async_transcribe.py) shows concurrent transcription, streaming alignment, and automatic cleanup.

```python
from edgespeak import AsyncEdgeSpeak

async with AsyncEdgeSpeak(max_concurrency=4) as client:
    result = await client.transcribe("speech.wav")
    async with client.align("speech.wav", text="The exact words spoken.", progress=True) as events:
        async for event in events:
            print(event)
```

Method names, parameters and result schemas match the sync client. Await results;
use `async with` directly for streaming. Explicitly awaiting a stream request is
also supported, but then use `async with stream` or `await stream.aclose()`.
Each stream request is single-use. Close streams when stopping early. Response
streams retain an HTTP slot, so nested calls need spare `max_concurrency` capacity.
Budget waiters do not hold HTTP slots; uploads release their budget before events.

Construction performs no CLI installation/startup; first awaited use/context entry
starts an owned service. Use `async with` or `await client.aclose()` for cleanup.
Existing-gateway authentication, billing and shutdown rules match the sync client:
use `gateway_key`; account API KEY environment variables are not read in attached
mode, and closing does not stop that gateway. `transport` must be async, as must
HTTPX `event_hooks`; the SDK owns and closes the supplied transport.
Network I/O is native async; encoding, file I/O and CLI installation/startup use
bounded worker threads. Cancellation waits for active local work to clean up;
it is not guaranteed to return instantly or cancel/refund remote inference.
Clients bind to their first event loop and process; create a new client after
fork or for another loop.


## Progress

```python
with EdgeSpeak() as client:
    with client.align("speech.wav", text="The exact words spoken.", progress=True) as events:
        for event in events:
            print(event)
```

Progress events precede the final result. Close the stream when stopping early;
closing does not guarantee cancellation of inference already running.
`transcribe(stream=True)` currently requires an attached desktop gateway and
does not support diarization or timestamp granularities.

## Compute device

Set the device when the SDK starts its own CLI service:

```python
with EdgeSpeak(device="cpu") as client:
    transcript = client.transcribe("meeting.wav")
# Linux/Windows: device="cuda" or "cuda:1" to select a GPU index.
# macOS: device="metal" (alias "mps").
```

Names are case-insensitive. Omit `device` or use `"auto"` to preserve the CLI's
default/environment policy. CUDA requires a compatible CLI/runtime and GPU;
Metal requires macOS. Hardware availability and model backend support are checked
by the runtime. CUDA indices follow CUDA device enumeration, not necessarily
system monitor numbering. This is a service-wide preference; each capability
retains its supported backends (CPU-only processing remains CPU-only).

An explicit `device`, including `"auto"`, cannot be combined with `base_url`:
the SDK raises `ValueError` before connecting. Existing gateways manage their
own devices; configure the gateway itself instead.

## Connect to an existing gateway

Set `base_url` to use an already running EdgeSpeak desktop or CLI gateway instead
of starting another CLI service:

```python
import os
from edgespeak import EdgeSpeak

with EdgeSpeak(
    base_url="http://127.0.0.1:1118/v1",
    gateway_key=os.environ["EDGESPEAK_GATEWAY_KEY"],
) as client:
    result = client.transcribe("meeting.wav")
```

`EDGESPEAK_GATEWAY_KEY` is an example variable for storing the gateway token; the
SDK does not read it automatically. Obtain the token from the gateway administrator.
Omit `gateway_key` if the local gateway has token authentication disabled.

- **The client does not need an account API KEY.** Do not pass `api_key` together
  with `base_url`; the SDK rejects that combination. Attached mode ignores the
  client's `EDGESPEAK_API_KEY`, does not send or modify it, and does not change
  the existing gateway's authorization.
- **The gateway still needs its own valid authorization.** Requests use the
  account or device authorization configured in that runtime. Metered usage is
  accounted for under its plan and usage rules. A gateway token controls access;
  it does not select a billing account. The SDK adds no separate charge and does
  not bypass quotas. For a gateway operated by someone else, confirm any fees
  with its operator.
- **Connections to remote gateways require HTTPS.** HTTP is allowed for loopback
  hosts (`localhost`, `127.0.0.1`, `::1`); LAN IPs and other non-loopback gateways
  also require HTTPS. Audio is uploaded to the remote gateway server.
- Closing the client does not stop an existing gateway. Environment proxies are
  ignored; create a separate client in each process.

When starting a CLI gateway directly, the CLI's `EDGESPEAK_API_KEY` means a
**gateway token**. In SDK-owned mode, the same variable means an **account API KEY**.
Do not interchange them. Changing the client's environment does not reconfigure
an already running gateway, but a CLI started later from that environment inherits it.

## Errors

`EdgeSpeakError` exposes `code`, `status_code`, `reason`, and `resets_at`.
Its `reason` distinguishes `api_key_expired`, `api_key_revoked`, and `usage_exhausted`.
Changing keys does not restore account allowance. Invalid arguments raise `ValueError`,
file errors raise `OSError`, and CLI startup failures raise `RuntimeStartError`.

`timeout` defaults to 600 seconds per HTTP I/O phase, not a total task deadline.
`startup_timeout` defaults to 30 seconds and excludes CLI installation time.

### Audio upsampling

```python
from pathlib import Path

with EdgeSpeak() as client:
    Path("output.wav").write_bytes(client.upsample("input.wav", sample_rate=24_000))
```

`sample_rate` defaults to 48,000 Hz and accepts positive integer rates up to
48,000 Hz. Higher-than-source rates use AudioVAE at 48 kHz, followed by
band-limited downsampling when needed. Lower-than-source rates only downsample;
equal rates preserve the WAV bytes. Independent calls consume no account usage.
Each channel is processed independently; channel count, order and valid WAV layout are preserved.
Input supports the formats listed below, with 1–8 channels, at most 32 MiB and 120 seconds. First
upsampling may download the AudioVAE model. The result is WAV bytes; the caller
chooses whether and where to save them. Requires a CLI/gateway build containing
`POST /v1/audio/upsamplings` (CLI 0.6.8 or newer).

CLI equivalent: `edgespeak-cli upsample input.wav --output output.wav --sample-rate 24000`.

## Dual-input alignment (CLI 0.6.8+)

The SDK exposes `diff_align(audio, reference_segments=[{"text": "Official words"}], transcript_segments=[{"text": "ASR words"}], offset=0.0)`. Both arrays accept nonempty `text` and optional `speaker`; native acoustic diff chooses the output. This requires `/v1/health.capabilities` to include `diff_alignment`; it is not available in older released gateways and never falls back to `align()`.

Each request is limited to 300 seconds including padding (413 on excess). Long recordings must be windowed by the consumer while preserving both texts. Result times include `offset`; `original_index` is local to the request reference array, or -1. Speaker labels come from input text, not voice identification. Normalization, extended search and streaming options are not supported here.

### Audio formats and encoding options (CLI 0.6.8+)

`upsample` accepts WAV (PCM16/24/32, Float32), FLAC, MP3, M4A/AAC, ADTS AAC, Ogg/Vorbis and Ogg/Opus files, detected by content. It returns WAV bytes by default. `response_format` selects `wav`, `flac`, `mp3`, `m4a`, `aac`, `ogg` (Vorbis), or `opus` (Ogg/Opus).

```python
Path("output.mp3").write_bytes(client.upsample("input.m4a",
    sample_rate=48_000, response_format="mp3", bitrate=192_000))
Path("output.wav").write_bytes(client.upsample("input.flac",
    sample_rate=24_000, pcm_format="s24"))
```

`bitrate` is the total target bit/s for lossy output only; WAV/FLAC reject it. `pcm_format` accepts WAV `s16/s24/s32/f32` or FLAC `s16/s24`; lossy formats reject it. Opus output requires 48000 Hz; MP3 supports mono/stereo. Unsupported rate, layout or encoding combinations fail explicitly. Codec availability depends on the gateway machine; missing codecs cause an error. The SDK installs no extra audio libraries and never silently downmixes.

Limits remain 32 MiB, 120 seconds and 1–8 channels. Speaker-position semantics and synchronization are preserved; storage order may change according to the target container standard. Lossy codecs may add delay/padding; prefer WAV/FLAC for sample-accurate timelines. Results are bytes in the requested format, never automatically saved, and consume no account usage.

## Text, catalog and usage

The following additions require a matching SDK build and gateway capabilities.
New method names follow HTTP paths; existing methods keep their names.

```python
from edgespeak import slice_utf8

with EdgeSpeak() as client:
    result = client.text_normalizations("The fee is $25.", language="en", top_k=1)
    print(result["alternatives"][0]["text"])  # converted text; result["text"] is original
    for span in result["alternatives"][0]["spans"]:
        print(slice_utf8(result["text"], span["start_byte"], span["end_byte"]))
    sentences = client.segment("First sentence. Second sentence.")
    catalog = client.models()
    totals = client.usage(refresh=True)
```

Use `mode="itn"` for spoken → written text. Normalization and segmentation use
text allowance; they do not consume audio-duration allowance. UTF-8 byte offsets
must not be used as Python string indices. `slice_utf8` validates byte boundaries.
`usage()` preserves `null` values for unlimited allowances.

`health()`, `models()`, `models_running()` and `models_downloads()` return gateway
JSON without starting inference. `models_load(model)`, `models_unload(model)`,
`models_download(model)` and `models_download_cancel(model)` are explicit actions;
a download call starts a job and does not wait for completion. Poll `models_downloads()`.

## Text generation

```python
with EdgeSpeak() as client:
    with client.responses("Explain this sentence.", model="your-text-model", stream=True) as events:
        for event in events:
            print(event)
```

`chat_completions(messages, model=...)`, `responses(input, model=...)` and
`messages(messages, model=..., max_tokens=...)` follow their respective API schemas.
They return JSON by default or `EventStream` with `stream=True`. Structured inputs
can include images when the selected model supports them; these APIs generate
text, not images. `options` forwards protocol extensions without overriding named
parameters, including omitted ones. `messages()` sends `anthropic-version` (default
`2023-06-01`); use `anthropic_version`/`anthropic_beta` for protocol headers.
`tokenize(content, model=...)` returns a JSON object or a raw integer token array;
for another provider schema, omit content and pass its fields in `options`.

Streams use their own protocol terminators: Chat `[DONE]`, Responses
`response.completed`, Messages `message_stop`.
Failures, incomplete Responses and EOF without a terminator raise `EdgeSpeakError`.
Keep already-yielded events if partial output matters; non-stream incomplete Responses
raise without returning their partial result.
Use a `with` block or `close()` when exiting early. Closing a stream does not
promise to cancel started inference or erase already consumed usage.

Local generation uses the gateway account's allowance. Calls forwarded to a
configured external provider follow that provider's billing; the SDK adds no
second charge. Model/download/health queries and tokenization consume no
EdgeSpeak inference allowance; external providers may charge separately. Realtime and MCP remain separate gateway protocols; use
their documented clients rather than synchronous SDK methods.

## Types and HTTP integration

`Transcription`, `Alignment`, `Segmentation`, `Diarization`, `Embeddings`,
`Similarity`, `Usage`, `Models` and `GatewayEvent` provide field hints. Results
remain ordinary dictionaries with gateway extension fields preserved. Optional
fields depend on the format/runtime; timestamps are seconds. Base-install typing
works without NumPy; arrays still require `edgespeak[numpy]`.

`EdgeSpeak(transport=..., limits=..., event_hooks=...)` accepts a synchronous
HTTPX transport, connection limits, and request/response hooks. The SDK closes
the supplied transport on `close()`; do not share it with another client. Hooks
also apply to streaming requests. Keep credentials out of logs and do not read
streaming bodies in response hooks. HTTP failures expose `error.request_id`
when the gateway supplies a valid `x-request-id`/`request-id` header.
Requests are not automatically retried, to avoid repeating inference or charges.

Sentence thresholds must be finite in `[0, 1]`, character limits integers in
`0..65535`, margins finite non-negative seconds, and boolean flags actual
booleans. Invalid values fail before uploads. `Device` and `AudioChannel` offer
completion hints; indexed CUDA selectors and numeric channel strings are checked
at runtime.

### Unified concurrency

Both clients accept `max_concurrency: int | None = None` (1–64). An explicit value caps client requests and, for an owned service, passes the same ceiling to `serve --max-concurrency`. Attached clients only cap their own requests; they do not reconfigure the gateway. An unsupported CLI fails with a capability error; update to a CLI supporting this flag.

`None` preserves serve defaults; async client requests default to 4, while sync clients retain HTTPX pool behavior. Existing model/route/device limits can be lower, so this is a ceiling, not a promise of N simultaneous inferences. It does not tune chunk parallelism or realtime. Streams hold client slots until closed; GET queries and model lifecycle controls bypass the client gate. Server execution slots are acquired after upload and held through execution/stream completion; cancellation releases queued admissions. Authentication and usage billing are unchanged.

## Runnable examples

The SDK requires CLI **0.6.8 or newer** with the SDK protocol. In owned mode, a missing or incompatible CLI triggers installation of the latest stable version. Attaching to an existing gateway does not install or upgrade its CLI.

Install with `python -m pip install edgespeak`, then download a script below or
run it from this checkout. Each script is independent and shows client creation,
SDK calls, output and cleanup directly; no shared example helper is required.
Run `python sdk/python/examples/<script>.py --help` for its options.

| Task | Synchronous | Asynchronous |
| --- | --- | --- |
| Transcription | [transcribe.py](examples/transcribe.py) | [async_transcribe.py](examples/async_transcribe.py) |
| Alignment, including progress | [align.py](examples/align.py) | [async_align.py](examples/async_align.py) |
| Text and timed segmentation | [segment.py](examples/segment.py) | [async_segment.py](examples/async_segment.py) |
| Diarization, embeddings and similarity | [speakers.py](examples/speakers.py) | [async_speakers.py](examples/async_speakers.py) |

For example, run `python sdk/python/examples/async_align.py speech.wav --text "Hello." --progress`.
Without `--base-url`, set your account `EDGESPEAK_API_KEY`; the SDK starts and
cleans up its own CLI. To attach, add `--base-url http://127.0.0.1:1118/v1` and
optionally set `EDGESPEAK_GATEWAY_KEY` to the gateway token. The client ignores
its account API KEY in attached mode and leaves the existing gateway running.
Remote gateway URLs require HTTPS; audio is uploaded to that gateway.
