Metadata-Version: 2.3
Name: ai_api_unified
Version: 2.33.0
Summary: Unified access layer for AI completions (LLM), embedding (semantic), image, video, and voice services
License: MIT
Keywords: llm,openai,anthropic,claude,bedrock,gemini,vertex,embeddings,completion,video,voice,api
Author: Dave Thomas
Author-email: davidcthomas@gmail.com
Requires-Python: >=3.11,<3.14
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Provides-Extra: anthropic
Provides-Extra: azure-tts
Provides-Extra: bedrock
Provides-Extra: dev
Provides-Extra: elevenlabs
Provides-Extra: google-gemini
Provides-Extra: middleware-pii-redaction
Provides-Extra: middleware-pii-redaction-large
Provides-Extra: middleware-pii-redaction-small
Provides-Extra: openai
Provides-Extra: similarity-score
Provides-Extra: video-frames
Provides-Extra: voice
Provides-Extra: voyage
Requires-Dist: Pillow (>=11.0.0,<12.0.0) ; extra == "video-frames"
Requires-Dist: PyYAML (>=6.0.1,<7.0.0)
Requires-Dist: anthropic (>=0.116.0,<1.0.0) ; extra == "anthropic"
Requires-Dist: audioop-lts (>=0.2.0) ; (python_version >= "3.13") and (extra == "azure-tts")
Requires-Dist: audioop-lts (>=0.2.0) ; (python_version >= "3.13") and (extra == "elevenlabs")
Requires-Dist: audioop-lts (>=0.2.0) ; (python_version >= "3.13") and (extra == "voice")
Requires-Dist: azure-cognitiveservices-speech (>=1.37.0,<2.0.0) ; extra == "azure-tts"
Requires-Dist: black (>=24.4.2,<25.0.0) ; extra == "dev"
Requires-Dist: boto3 (>=1.34.0,<2.0.0) ; extra == "bedrock"
Requires-Dist: elevenlabs (>=2.3.0,<3.0.0) ; extra == "elevenlabs"
Requires-Dist: google-cloud-speech (>=2.25.0,<3.0.0) ; extra == "google-gemini"
Requires-Dist: google-cloud-texttospeech (>=2.29.0) ; extra == "google-gemini"
Requires-Dist: google-genai (>=2.10.0,<3.0.0) ; extra == "google-gemini"
Requires-Dist: httpx (>=0.27.0,<1.0.0)
Requires-Dist: imageio (>=2.37.0,<3.0.0) ; extra == "video-frames"
Requires-Dist: imageio-ffmpeg (>=0.6.0,<1.0.0) ; extra == "video-frames"
Requires-Dist: numpy (==2.2.6) ; extra == "similarity-score"
Requires-Dist: openai (>=2.0.0,<3.0.0) ; extra == "openai"
Requires-Dist: presidio-analyzer (>=2.2.35) ; extra == "middleware-pii-redaction"
Requires-Dist: presidio-analyzer (>=2.2.35) ; extra == "middleware-pii-redaction-large"
Requires-Dist: presidio-analyzer (>=2.2.35) ; extra == "middleware-pii-redaction-small"
Requires-Dist: presidio-anonymizer (>=2.2.35) ; extra == "middleware-pii-redaction"
Requires-Dist: presidio-anonymizer (>=2.2.35) ; extra == "middleware-pii-redaction-large"
Requires-Dist: presidio-anonymizer (>=2.2.35) ; extra == "middleware-pii-redaction-small"
Requires-Dist: pydantic (>=2,<3)
Requires-Dist: pydantic-settings (>=2,<3)
Requires-Dist: pydub (>=0.25.1,<0.26.0) ; extra == "azure-tts"
Requires-Dist: pydub (>=0.25.1,<0.26.0) ; extra == "elevenlabs"
Requires-Dist: pydub (>=0.25.1,<0.26.0) ; extra == "voice"
Requires-Dist: pytest (>=8.4.0,<9.0.0) ; extra == "dev"
Requires-Dist: pytest-asyncio (>=0.24.0,<1.0.0) ; extra == "dev"
Requires-Dist: ruff (>=0.4.4,<1.0.0) ; extra == "dev"
Requires-Dist: spacy (>=3.4.4,!=3.7.0,<4.0.0) ; extra == "middleware-pii-redaction"
Requires-Dist: spacy (>=3.4.4,!=3.7.0,<4.0.0) ; extra == "middleware-pii-redaction-large"
Requires-Dist: spacy (>=3.4.4,!=3.7.0,<4.0.0) ; extra == "middleware-pii-redaction-small"
Requires-Dist: usaddress (>=0.5.16,<0.6.0) ; extra == "middleware-pii-redaction"
Requires-Dist: usaddress (>=0.5.16,<0.6.0) ; extra == "middleware-pii-redaction-large"
Requires-Dist: usaddress (>=0.5.16,<0.6.0) ; extra == "middleware-pii-redaction-small"
Requires-Dist: voyageai (>=0.3.2,<1.0.0) ; extra == "voyage"
Project-URL: Changelog, https://github.com/davecthomas/ai-api-unified/blob/main/CHANGELOG.md
Project-URL: Homepage, https://github.com/davecthomas/ai-api-unified
Project-URL: Issues, https://github.com/davecthomas/ai-api-unified/issues
Project-URL: Repository, https://github.com/davecthomas/ai-api-unified
Description-Content-Type: text/markdown

# ai-api-unified 2.33.0

[![CI](https://github.com/davecthomas/ai-api-unified/actions/workflows/ci.yml/badge.svg)](https://github.com/davecthomas/ai-api-unified/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/ai-api-unified.svg)](https://pypi.org/project/ai-api-unified/)
[![Python](https://img.shields.io/pypi/pyversions/ai-api-unified.svg)](https://pypi.org/project/ai-api-unified/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

`ai-api-unified` is a unified Python library for AI completions, embeddings, image generation, video generation, and voice. Application code targets stable base interfaces and factory entry points while environment configuration selects the concrete providers at runtime.

Because every provider sits behind the same interface, the library can also
keep your application running when a provider cannot: configure a
[fallback chain](#model-fallback) and a request that fails because a model is
overloaded, rate limited, or out of quota is retried on the next model you
listed, on the same provider or a different one, with no change to the
calling code.

**Production status.** This library is in production use, serving live traffic
across several systems. It is published on PyPI on a regular release cadence,
follows [semantic versioning](https://semver.org/), and every release is gated
on the full mocked regression suite. Breaking changes land only in a major
version, and each release is recorded in [CHANGELOG.md](CHANGELOG.md).

Author: Dave Thomas  
Install name: `ai-api-unified`  
Import path: `ai_api_unified`  
Python: `>=3.11,<3.14`  
License: MIT

The current package architecture is registry-backed and lazy-loaded:

- provider SDKs are optional extras, not base dependencies
- providers are resolved only when a factory selects them
- package `__init__` modules export stable interfaces only
- missing provider selectors are configuration errors, not implicit fallbacks

## Supported models

More than 100 models from 10 model makers (OpenAI, Anthropic, Google,
Amazon, DeepSeek, Alibaba Qwen, Z.ai, Voyage AI, ElevenLabs, and Microsoft)
behind one interface, plus any server that speaks the OpenAI API. Switching
models is a configuration change: set the capability's engine (for example
`COMPLETIONS_ENGINE`) and model (`COMPLETIONS_MODEL_NAME`).

<!-- supported-models:start -->
| Capability | Provider | Engine (`*_ENGINE`) | Models |
| --- | --- | --- | --- |
| Completions | OpenAI | `openai`, `openai-responses` | `gpt-6-astra`, `gpt-6-sol`, `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna` (default), `gpt-5.5`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.2`, `gpt-5.1-codex-max`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano` ⚠ until 2026-10-23, `o4-mini` ⚠ until 2026-10-23, `gpt-4o`, `gpt-4o-mini` |
| Completions | Anthropic | `claude` | `claude-fable-5-1`, `claude-opus-5-5`, `claude-fable-5`, `claude-opus-5` (default), `claude-sonnet-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5` |
| Completions | Google | `google-gemini` | `gemini-3.8-flash`, `gemini-3.7-flash` (default), `gemini-3.6-flash`, `gemini-3.5-flash`, `gemini-3.5-flash-lite`, `gemini-3.1-flash-lite`, `gemini-3.1-pro-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
| Completions | Amazon Bedrock: Amazon Nova | `bedrock`, `nova` | `us.amazon.nova-2-lite-v1:0`, `amazon.nova-micro-v1:0`, `amazon.nova-lite-v1:0` (default), `amazon.nova-pro-v1:0`, `amazon.nova-premier-v1:0` |
| Completions | Amazon Bedrock: Anthropic Claude | `bedrock`, `anthropic` | `us.anthropic.claude-fable-5-1`, `us.anthropic.claude-opus-5-5`, `us.anthropic.claude-opus-5`, `us.anthropic.claude-sonnet-5`, `us.anthropic.claude-3-5-haiku-20241022-v1:0` |
| Completions | Amazon Bedrock: DeepSeek | `bedrock` | `deepseek.v3.2`, `us.deepseek.r1-v1:0` |
| Completions | Amazon Bedrock: Qwen (Alibaba) | `bedrock` | `qwen.qwen3-next-80b-a3b`, `qwen.qwen3-235b-a22b-2507-v1:0`, `qwen.qwen3-coder-next`, `qwen.qwen3-32b-v1:0` |
| Completions | Amazon Bedrock: GLM (Z.ai) | `bedrock` | `zai.glm-5`, `zai.glm-4.7`, `zai.glm-4.7-flash` |
| Completions | Any OpenAI-compatible server (vLLM, Ollama, LiteLLM, ...) | `openai-compatible` | any model the server hosts (`COMPLETIONS_MODEL_NAME`) |
| Embeddings | OpenAI | `openai` | `text-embedding-3-small` (default), `text-embedding-3-large`, `text-embedding-ada-002` |
| Embeddings | Google | `google-gemini` | `gemini-embedding-001` (default), `gemini-embedding-2` (multimodal) |
| Embeddings | Voyage AI | `voyage` | `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-code-4`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-3-lite`, `voyage-3` (default), `voyage-3-large`, `voyage-code-3`, `voyage-finance-2`, `voyage-law-2` |
| Embeddings | Amazon Bedrock: Titan | `titan` | `amazon.titan-embed-text-v2:0` (default), `amazon.titan-embed-text-v1` |
| Images | OpenAI | `openai` | `gpt-image-2.5-flare`, `gpt-image-2.5-sunburst`, `gpt-image-2` (default), `gpt-image-1.5` ⚠ until 2026-12-01, `gpt-image-1-mini` ⚠ until 2026-12-01, `gpt-image-1` ⚠ until 2026-10-23 |
| Images | Google | `google-gemini` | `gemini-3.1-flash-image` (default), `gemini-3.1-flash-lite-image`, `gemini-3-pro-image`, `gemini-2.5-flash-image` ⚠ until 2026-10-02 |
| Images | Amazon Bedrock: Nova Canvas | `nova-canvas`, `bedrock`, `nova` | `amazon.nova-canvas-v1:0` (default) |
| Video | Google | `google-gemini` | `veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview`, `veo-3.1-lite-generate-preview` (default) |
| Video | Amazon Bedrock: Nova Reel | `nova-reel`, `bedrock`, `nova` | `amazon.nova-reel-v1:1` (default) |
| Video | OpenAI | `openai` | none: `sora-2` and `sora-2-pro` retired 2026-09-24 |
| Text to speech | OpenAI | `openai` | `tts-1-hd` (default), `tts-1`, `gpt-4o-mini-tts` |
| Text to speech | Google | `google` | `gemini-2.5-pro-tts` (default), `gemini-2.5-flash-tts`, `gemini-3.1-flash-tts-preview` |
| Text to speech | ElevenLabs | `elevenlabs` | `eleven_multilingual_v2` (default), `eleven_v3`, `eleven_flash_v2_5` |
| Text to speech | Microsoft Azure | `azure` | `neural`, `neural_hd`, `openai_neural`, `openai_hd` voice families |
| Speech to text | OpenAI | `openai` | `gpt-transcribe` |
| Speech to text | Google | `google` | Google Cloud Speech-to-Text |
| Speech to text | ElevenLabs | `elevenlabs` | `scribe_v1` |
<!-- supported-models:end -->

(default) marks the model an engine uses when no model is configured. ⚠
marks a deprecated model that still works until the date shown. Bedrock
models run on AWS under your AWS account; AWS serves Qwen3 235B only in
us-east-2 and us-west-2. The list matches each provider's live model list as
of 2026-09-25 (Bedrock: the AWS model cards), and each model carries pricing
in the [cost registry](#model-pricing).

## Overview

Use this library when you want one consistent interface across multiple AI providers without binding application code to a single SDK. The library currently covers:

- text completions
- embeddings
- image generation
- video generation
- text-to-speech and selected speech-to-text flows

Across those capabilities the library gives you:

- one interface per capability, so swapping providers is a configuration change
- automatic [model fallback](#model-fallback) for completions, so an outage,
  rate limit, or exhausted quota on one model is served by the next
- cost tracking, prompt caching, PII redaction, and observability middleware
  that work the same way on every engine

The public entry points are the stable base interfaces and factories:

- `AIFactory.get_ai_completions_client()`
- `AIFactory.get_ai_embedding_client()`
- `AIFactory.get_ai_images_client()`
- `AIFactory.get_ai_video_client()`
- `AIFactory.get_ai_voice_client()` (or `AIVoiceFactory.create()`)

## Capabilities

| Capability  | Stable interface    | Engines                                                                                                                       | Required extra(s)                                    |
| ----------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| Completions | `AIBaseCompletions` | `openai`, `openai-responses`, `openai-compatible`, `claude`, `google-gemini`, Bedrock-routed aliases such as `nova`, `anthropic`, `llama`, `mistral`, `cohere`, `ai21`, `rerank` | `openai`, `anthropic`, `google_gemini`, `bedrock`    |
| Embeddings  | `AIBaseEmbeddings`  | `openai`, `titan`, `google-gemini`, `voyage`                                                                                  | `openai`, `bedrock`, `google_gemini`, `voyage`       |
| Images      | `AIBaseImages`      | `openai`, `google-gemini`, `nova-canvas` and Bedrock image aliases                                                            | `openai`, `google_gemini`, `bedrock`                 |
| Videos      | `AIBaseVideos`      | `openai`, `google-gemini`, `nova-reel` and Bedrock video aliases                                                              | `openai`, `google_gemini`, `bedrock`                 |
| Voice TTS   | `AIVoiceBase`       | `openai`, `google`, `azure`, `elevenlabs`                                                                                     | `openai`, `google_gemini`, `azure_tts`, `elevenlabs` |
| Voice STT   | `AIVoiceBase`       | provider-specific support such as Google and ElevenLabs                                                                       | `google_gemini`, `elevenlabs`                        |

Default model guidance in the checked-in OSS env files:

- Anthropic completions: `claude-opus-5`
- Google completions: `gemini-3.7-flash`
- Google embeddings: `gemini-embedding-001` (text-only) or `gemini-embedding-2` (multimodal)
- Google images: `gemini-3.1-flash-image`
- Google videos: `veo-3.1-lite-generate-preview`
- Google voice: `gemini-2.5-pro-tts`

## Installation

### Python Requirements

This package requires Python `>=3.11,<3.14`.

### Install as a Dependency

Base package only:

```bash
poetry add ai-api-unified
```

Install with one or more provider extras:

```bash
poetry add 'ai-api-unified[google_gemini]'
poetry add 'ai-api-unified[openai]'
poetry add 'ai-api-unified[anthropic]'
poetry add 'ai-api-unified[bedrock,google_gemini]'
poetry add 'ai-api-unified[google_gemini,video_frames]'
poetry add 'ai-api-unified[openai,video_frames]'
poetry add 'ai-api-unified[google_gemini,voice]'   # Google voice/TTS/STT
poetry add 'ai-api-unified[openai,voice]'          # OpenAI voice
```

Audio dependencies (pydub and, on Python 3.13+, audioop-lts) ship in the
`voice` extra as of 2.16.0, so text-only installs such as
`ai-api-unified[anthropic]` carry no audio packages and importing the library
never triggers pydub's warnings. The `azure_tts` and `elevenlabs` extras
include the audio dependencies; Google and OpenAI voice pair their provider
extra with `voice`. Using a voice feature without the audio dependencies
raises `AiProviderDependencyUnavailableError` naming the extra to install.

The `google_gemini` extra states no protobuf range of its own as of 2.26.1.
Releases through 2.26.0 capped it below 5.0, which made the extra uninstallable
in any application already on protobuf 5 or later. protobuf is still required
here, but only transitively: `google-api-core`, `google-cloud-speech`,
`google-cloud-texttospeech` and `proto-plus` all depend on it, and they enforce
their own ranges. This library never imports protobuf, so any range it declared
could only conflict with theirs. Installs are verified against protobuf 4.25.8,
5.29.5, 6.33.6 and 7.36.1.

### Install in a Local Clone

Base install:

```bash
poetry install
```

Common local development installs:

```bash
poetry install --with dev
poetry install --extras "google_gemini"
poetry install --extras "openai"
poetry install --extras "google_gemini" --extras "video_frames" --with dev
poetry install --extras "openai" --extras "video_frames" --with dev
poetry install --all-extras --with dev
```

### Optional Extras

| Extra                            | Installs                                                               |
| -------------------------------- | ---------------------------------------------------------------------- |
| `openai`                         | OpenAI completions, embeddings, images, and voice                      |
| `anthropic`                      | Anthropic Claude completions via the native Anthropic API (`claude` engine) |
| `google_gemini`                  | Google Gemini completions, embeddings, images, and Google voice        |
| `bedrock`                        | AWS Bedrock completions, Titan embeddings, Bedrock image providers, and Bedrock video providers |
| `voyage`                         | Voyage AI embeddings (the `voyage` embeddings engine; `VOYAGE_API_KEY`) |
| `video_frames`                   | Optional frame extraction helpers backed by ImageIO + Pillow           |
| `azure_tts`                      | Azure Cognitive Services TTS (includes the `voice` audio deps)         |
| `elevenlabs`                     | ElevenLabs TTS and STT (includes the `voice` audio deps)               |
| `voice`                          | Shared audio processing (pydub; audioop-lts on Python 3.13+) for voice/TTS/STT; pair with a provider extra, e.g. `[google_gemini,voice]` or `[openai,voice]` |
| `middleware-pii-redaction`       | Presidio + spaCy + `usaddress`; install the required spaCy model separately |
| `middleware-pii-redaction-small` | Compatibility alias for PII redaction deps; pair with separate `en_core_web_sm` install |
| `middleware-pii-redaction-large` | Compatibility alias for PII redaction deps; pair with separate `en_core_web_lg` install |
| `similarity_score`               | NumPy-based similarity helpers                                         |
| `dev`                            | Optional dev dependencies from `[project.optional-dependencies]`       |

### Environment File

Copy [`env_template`](env_template) to `.env` and fill in only the providers you use.

The OSS template defaults to Google API-key auth:

```dotenv
COMPLETIONS_ENGINE=google-gemini
EMBEDDING_ENGINE=google-gemini
IMAGE_ENGINE=google-gemini
VIDEO_ENGINE=google-gemini
AI_VOICE_ENGINE=google

GOOGLE_GEMINI_API_KEY=...
GOOGLE_AUTH_METHOD=api_key

COMPLETIONS_MODEL_NAME=gemini-3.7-flash
EMBEDDING_MODEL_NAME=gemini-embedding-001
IMAGE_MODEL_NAME=gemini-3.1-flash-image
VIDEO_MODEL_NAME=veo-3.1-lite-generate-preview
DEFAULT_GEMINI_TTS_MODEL=gemini-2.5-pro-tts
```

Leave `EMBEDDING_DIMENSIONS` unset unless you deliberately want a provider-specific override. The library preserves provider defaults.

### Smoke Test

```bash
python -c "import ai_api_unified; print(ai_api_unified.__version__)"
```

## Quickstart

The examples below assume the Google API-key-first OSS defaults shown above. The same APIs work with OpenAI, Bedrock, Azure, or ElevenLabs by changing env selectors and installing the matching extras.

### Completions

```python
from ai_api_unified import AIFactory, AIBaseCompletions

client: AIBaseCompletions = AIFactory.get_ai_completions_client()
response: str = client.send_prompt("Say hello in one short sentence.")
print(response)
```

### Completions with automatic fallback

Add one setting and the same client keeps serving when the primary model
cannot. The chain below runs Claude first, then OpenAI, then Gemini:

```bash
COMPLETIONS_ENGINE=claude
COMPLETIONS_MODEL_NAME=claude-opus-5
COMPLETIONS_FALLBACKS=openai:gpt-5.6-luna,google-gemini:gemini-3.7-flash
```

```python
from ai_api_unified import AIFactory

client = AIFactory.get_ai_completions_client()   # an AiFallbackCompletions
response: str = client.send_prompt("Say hello in one short sentence.")
print(response, "served by", client.last_route.label)
```

Nothing else changes: `send_prompt`, `send_conversation`,
`send_structured_output`, streaming, and the async variants all work as
before. See [Model fallback](#model-fallback) for what triggers a failover,
what does not, and how to see which model answered.

### Streaming Completions

Every completions client exposes a `capabilities` descriptor with a
`supports_streaming` flag set per model. Models without streaming support raise
`AiProviderCapabilityUnsupportedError` from `send_prompt_streaming`. OpenAI,
Anthropic, Google Gemini, and Bedrock chat models all stream:

```python
from ai_api_unified import AIFactory

client = AIFactory.get_ai_completions_client()

if client.capabilities.supports_streaming:
    for chunk in client.send_prompt_streaming("Tell me a short story."):
        print(chunk, end="", flush=True)
```

Streaming is unavailable while the PII redaction middleware is enabled
(`AiProviderConfigurationError`): redaction cannot be guaranteed across chunk
boundaries, so use `send_prompt` in PII-redacting deployments.

### Token Counting

Providers whose capabilities include `supports_token_counting` can return a
provider-counted input token total without running inference. Bedrock supports
this via its `CountTokens` operation for Nova and Claude (not for the DeepSeek,
Qwen, and GLM models it hosts), and the native Anthropic API via its
`count_tokens` endpoint; other providers raise
`AiProviderCapabilityUnsupportedError`.

```python
client = AIFactory.get_ai_completions_client(completions_engine="claude")

if client.capabilities.supports_token_counting:
    print(client.count_tokens("How many tokens is this prompt?"))
```

### OpenAI Responses engine

OpenAI exposes two completions engines. The default `openai` engine uses Chat
Completions; the `openai-responses` engine (`COMPLETIONS_ENGINE=openai-responses`)
uses the Responses API, OpenAI's successor to Chat Completions. Both implement
`send_prompt`, `strict_schema_prompt`, and
`send_prompt_streaming`. The Responses engine is text-only for now; use the
`openai` engine for image inputs.

### OpenAI-compatible servers

The `openai-compatible` engine drives any server that speaks the OpenAI Chat
Completions protocol (vLLM, Ollama, LiteLLM, hosted vendors with an
OpenAI-compatible endpoint) through the `openai` extra. It supports the same
surface as the `openai` engine: `send_prompt`, streaming,
`send_conversation` with tools, `send_structured_output`,
`strict_schema_prompt`, and the async variants.

```dotenv
COMPLETIONS_ENGINE=openai-compatible
OPENAI_COMPATIBLE_BASE_URL=http://localhost:8000/v1   # required
COMPLETIONS_MODEL_NAME=Qwen/Qwen3-32B                 # required
OPENAI_COMPATIBLE_API_KEY=...                         # optional for local servers
OPENAI_COMPATIBLE_STRUCTURED_OUTPUT=json_schema       # or json_object
OPENAI_COMPATIBLE_CONTEXT_WINDOW=131072               # optional context guard
```

The base URL must be https:// unless it targets a loopback host. Requests
send `max_tokens` rather than OpenAI's `max_completion_tokens`, which most
compatible servers do not accept. Set `OPENAI_COMPATIBLE_STRUCTURED_OUTPUT`
to `json_object` for servers that guarantee valid JSON but do not enforce a
schema; the schema is then placed in the system prompt. The generic engine
has no pricing, reports no organization identity, and labels observability
events `openai-compatible`.

Vendor engines subclass `AiOpenAICompatibleCompletions` and set class
attributes (API key and base-URL settings, default endpoint, model
catalog, context windows, image and reasoning models, structured-output
mode, and the pricing-registry label) instead of reading them from the
environment.

### Anthropic Claude engines

Claude models are reachable through two completions engines:

- `claude` — the native Anthropic API (api.anthropic.com) via the official
  `anthropic` SDK. Requires the `anthropic` extra and `ANTHROPIC_API_KEY`.
- `anthropic` — Claude on Amazon Bedrock via the Converse API. Requires the
  `bedrock` extra and AWS credentials.

The two engines expose the same core caller-facing API (`send_prompt`,
`strict_schema_prompt`, `send_prompt_streaming`, `count_tokens`) plus the
capability-gated conversation/structured-output surface (see the support
matrix above); switching between them is a configuration change. The `claude`
engine additionally implements the async variants and batch completions. Use
`claude` for the current model lineup on Anthropic's own endpoint; use
`anthropic` when your Claude access is provisioned through AWS.

```dotenv
COMPLETIONS_ENGINE=claude
COMPLETIONS_MODEL_NAME=claude-opus-5
ANTHROPIC_API_KEY=...
```

Models cataloged for the `claude` engine (alias model IDs):
`claude-fable-5-1`, `claude-opus-5-5`, `claude-fable-5`, `claude-opus-5`
(default), `claude-sonnet-5`, `claude-opus-4-8`, `claude-opus-4-7`,
`claude-opus-4-6`, `claude-sonnet-4-6`, and `claude-haiku-4-5`. Capabilities per model include the context window (1M
tokens except `claude-haiku-4-5` at 200K), streaming, provider-side token
counting, image inputs, and registry pricing. Structured output uses the Messages API JSON-schema response format,
so `strict_schema_prompt` works on every cataloged model. On
`claude-fable-5`, `claude-fable-5-1`, and `claude-opus-5-5`, whose thinking
is always on and counts against `max_tokens`, pass a `max_response_tokens`
well above the 2048 default so the budget covers thinking plus the JSON body.
`claude-fable-5-1` and `claude-opus-5-5` reject a forced tool choice, so
`send_conversation(..., tool_choice=...)` raises `ValueError` on those models
before any request is sent; leave `tool_choice` unset and name the tool in
the prompt. Image attachments are capped at Anthropic's 5MB
per-image limit.

### Send prompt options

`send_prompt` accepts three optional parameters. Omitting them leaves prior
behavior unchanged.

```python
from ai_api_unified import AIFactory

client = AIFactory.get_ai_completions_client()
text = client.send_prompt(
    "Generate a workflow document from this instruction: ...",
    system_prompt="You write workflow documents.",
    max_response_tokens=9000,
    request_timeout_seconds=30.0,
)
```

All three parameters map to native provider fields on the `claude`,
`openai`, `openai-responses`, `openai-compatible`, and `google-gemini`
engines. Bedrock-routed
engines map `system_prompt` and `max_response_tokens` but raise
`AiProviderCapabilityUnsupportedError` for `request_timeout_seconds` (boto3
has no per-call timeout). See the support matrix below.

#### Feature support by engine

| Feature | `claude` | `openai` | `openai-responses` | `openai-compatible` | `google-gemini` | Bedrock-routed |
|---|---|---|---|---|---|---|
| `send_prompt` extended params | yes | yes | yes | yes | yes | partial (no per-call timeout) |
| `send_structured_output` | yes | yes | yes | yes (`json_schema` or `json_object` mode) | yes | per-model: Claude 4.5 and Opus 4.6, DeepSeek V3.2, Qwen3, GLM |
| `send_conversation` tool loop | yes | yes | yes | yes | yes | Nova, Claude, DeepSeek V3.2, Qwen3, GLM (not DeepSeek R1) |
| Async variants (`asend_*`) | yes | yes | yes | yes | yes | no (boto3 has no official async client) |
| `retry_policy` / `AiProviderRequestError` | yes | yes | yes | yes | yes | yes (engine loop; SDK retries via `AWS_MAX_ATTEMPTS`) |

Unsupported combinations raise the typed `AiProviderCapabilityUnsupportedError`
and each engine's `client.capabilities` flags report support at runtime.

### Structured output extraction (send_structured_output)

`send_structured_output` is the single-shot extraction call: prose in, a parsed
JSON object out. It accepts either an `AIStructuredPrompt` subclass
(`response_model`) or a raw JSON Schema (`response_schema`) — for example a
hand-written schema with `anyOf` variants per node type. Engines declare
support via `capabilities.supports_structured_output`. Provider mappings:
`claude` uses the Messages API JSON-schema response format (and streams and
accumulates internally above its non-streaming budget); `openai` and
`openai-responses` use the `json_schema` response format in schema-guided
mode; `openai-compatible` uses `json_schema`, or `json_object` with the
schema in the system prompt; `google-gemini` uses `response_json_schema`;
Bedrock uses Converse `outputConfig` on the models AWS supports (Claude 4.5
and Opus 4.6, DeepSeek V3.2, Qwen3, and GLM).

```python
result = client.send_structured_output(
    "Compile this prose into a workflow graph: ...",
    response_schema=GRAPH_SCHEMA,          # raw JSON Schema owned by the caller
    system_prompt="You are a workflow compiler.",
    max_response_tokens=32_000,            # up to the model context limit
    request_timeout_seconds=120.0,
)
if result.finish_reason == "complete":
    graph = result.data                     # parsed dict
elif result.finish_reason == "length":
    ...                                     # truncated: retry with a larger budget
elif result.finish_reason == "refusal":
    ...                                     # model declined: abort
print(result.usage.input_tokens, result.usage.output_tokens)
```

Every result carries a normalized `finish_reason`
(`complete | length | tool_use | refusal`, one enum across engines) and token
usage, so truncation and refusal are distinguishable in code. `data` is `None`
on `length` and `refusal`. Multi-turn correction is supported through
`messages`: replay a prior model output plus feedback as extra
`{role, content}` turns, and `prompt` may then be omitted.

```python
result = client.send_structured_output(
    "The kind 'bogus' is invalid; use 'task' or 'gate'.",
    response_schema=GRAPH_SCHEMA,
    messages=[
        {"role": "user", "content": original_prompt},
        {"role": "assistant", "content": previous_bad_output},
    ],
)
```

Budgets above the engine's non-streaming request limit are handled inside the
engine (the `claude` engine streams and accumulates); the caller sees one
blocking call either way. `provider_options` is a per-engine escape hatch
merged into the underlying request; the `claude` engine honors the reserved key
`retry_policy` and merges every other key verbatim into the Messages request.

### Tool-use conversations (send_conversation)

`send_conversation` sends one conversation turn and returns the model's turn;
the caller owns the tool loop. Engines declare support via
`capabilities.supports_tool_use`.

```python
from ai_api_unified import AITool

tools = [
    AITool(
        name="lookup_ticket",
        description="Fetch a ticket by id.",
        input_schema={
            "type": "object",
            "properties": {"ticket_id": {"type": "string"}},
            "required": ["ticket_id"],
        },
        strict=True,               # schema-exact tool inputs where supported
    )
]

messages = [{"role": "user", "content": "Summarize ticket VL-123."}]
for _ in range(MAX_ITERATIONS):
    turn = client.send_conversation(
        "You are a support agent.",
        messages,
        tools=tools,
        # tool_choice="lookup_ticket",   # force a named tool for this turn
        max_response_tokens=4096,
        request_timeout_seconds=60.0,
    )
    if turn.finish_reason != "tool_use":
        break
    client.extend_messages_with_turn(messages, turn)   # engine-shaped replay
    for tool_call in turn.tool_calls:
        output = execute_tool(tool_call.name, tool_call.input)   # caller-side (e.g. MCP)
        messages.append(
            client.build_tool_result_message(
                tool_call_id=tool_call.id, result=output, is_error=False
            )
        )
```

Each `AITurnResult` carries `text`, `tool_calls` (`id`, `name`, `input`), the
normalized `finish_reason`, `usage` for that turn, and `raw_content` —
engine-specific content replayable as the next assistant turn. Because the
assistant-turn wire shape differs per engine, `extend_messages_with_turn`
appends it for you and `build_tool_result_message` produces the engine's
tool-result shape, so the loop above runs unchanged on `claude`, `openai`,
`openai-responses`, `openai-compatible`, `google-gemini`, and tool-capable
Bedrock models (Nova, Claude, DeepSeek V3.2, Qwen3, and GLM). On Gemini, tool calls carry no provider ids, so
`AIToolCall.id` is the function name.

### Async variants

Engines whose SDK has an async client expose `a`-prefixed variants with the
same signatures: `asend_prompt`, `asend_structured_output`, and
`asend_conversation`. Support is declared via `capabilities.supports_async`:
`claude` (lazy `AsyncAnthropic`), `openai`, `openai-responses`, and
`openai-compatible` (lazy `AsyncOpenAI`), and `google-gemini` (`client.aio`)
implement all three;
Bedrock does not (boto3 has no official async client). Gemini async calls
run a single attempt — the engine backoff loop is synchronous — so pair them
with caller-owned backoff on `AiProviderRequestError`. Sync methods are
unchanged.

```python
turn = await client.asend_conversation("system", messages, tools=tools)
```

### Prompt caching

Providers bill repeated prompt prefixes at a discounted cached-input rate. Some
cache every long prefix on their own; others cache only what the request marks.
`AIPromptCacheHint` asks for caching in one provider-neutral way: it marks the
system prompt and tool definitions as a stable prefix. Pass it on
`other_params.prompt_cache` (`send_prompt`, `asend_prompt`,
`send_prompt_streaming`, `strict_schema_prompt`), as `prompt_cache=` on
`send_conversation`, `send_structured_output`, and their async variants, or on
`AIBatchRequestItem.prompt_cache` for Anthropic batches.

```python
from ai_api_unified import AIPromptCacheHint, AIPromptCacheRetention

hint = AIPromptCacheHint(
    retention=AIPromptCacheRetention.EXTENDED,  # DEFAULT (about 5 min) or EXTENDED
    key="tenant-42",                            # optional; OpenAI cache routing
)
turn = client.send_conversation(
    LONG_STABLE_SYSTEM_PROMPT, messages, tools=tools, prompt_cache=hint
)
```

| Engine | Without a hint | With a hint |
| ------ | -------------- | ----------- |
| `claude` | No caching | `cache_control` breakpoint on the system block; conversations also cache the growing history. EXTENDED = 1-hour TTL |
| Bedrock (Claude, Nova) | Implicit, best effort | `cachePoint` after the system block. EXTENDED = 1-hour TTL on Claude 4.5 and later; Nova stays at 5 minutes |
| `openai`, `openai-responses` | Implicit on prefixes of 1,024+ tokens | Adds `prompt_cache_key`; EXTENDED sends `prompt_cache_retention="24h"` on models that offer it |
| `google-gemini` | Implicit on Gemini 2.5+; durable (up to 24 hours, no storage fee) on Gemini 3.x from 2026-10-15 | Ignored, since there is nothing to add |
| `openai-compatible`, other Bedrock models, and `claude` or `openai` behind a `base_url` gateway | Provider dependent | Ignored, since a gateway may reject the fields |

`capabilities.implicit_prompt_caching` and `capabilities.supports_prompt_cache_hint`
report both columns per model. Caching changes cost and latency but never the
response, so an engine that cannot honor the hint ignores it instead of raising.
The hint is not free on every provider: Anthropic and Bedrock bill cache
writes at 1.25x base input (2x for the 1-hour TTL), so it pays off only when
requests sharing the prefix repeat within the TTL. A hit needs a byte-identical
prefix, so keep timestamps and request IDs out of the system prompt. Cache
reads and writes show up in `AITurnResult.usage` and in the cost-tracking
events described under [Cost tracking](#cost-tracking-financial-ops).
Batch requests run in any order over minutes to hours, so batch cache hits are
best effort; EXTENDED retention usually suits them better than DEFAULT.

### Retry policy and typed request errors

Engine retry behavior:

- `claude` / `openai` / `openai-responses` — the provider SDK retries
  transient failures (408, 409, 429, 5xx) twice by default with exponential
  backoff; `retry_policy="none"` constructs the client with `max_retries=0`.
- `google-gemini` — the engine's exponential-backoff loop (5 attempts);
  `retry_policy="none"` collapses it to a single attempt.
- Bedrock-routed engines — the engine's backoff schedule;
  `retry_policy="none"` collapses it to a single attempt. botocore's own
  client-level retries are configured separately via the standard
  `AWS_MAX_ATTEMPTS` environment variable.

Every engine accepts `retry_policy` at the constructor, through configuration
(`COMPLETIONS_RETRY_POLICY=none`), or per call
(`provider_options={"retry_policy": "none"}`; on Bedrock the per-call
override collapses that call's schedule). HTTP-level failures raise
`AiProviderRequestError`, whose `status_code` attribute lets caller backoff
classify 429/5xx/529 uniformly across engines; it is `None` when the failure
happened before a status was available (connection error or client-side
timeout).

The error also carries `fallback_reason`, an `AiFallbackReason` that says
whether a different model might serve the request. Each engine sets it from
the provider's own error codes, since a status alone cannot separate a rate
limit from an exhausted quota (OpenAI returns 429 for both, and Anthropic
reports a low credit balance or a reached spend cap as a 400):

| Reason | Meaning | Examples |
| ------ | ------- | -------- |
| `UNAVAILABLE` | The provider or model cannot serve right now | 5xx, Anthropic 529 overloaded, Bedrock `ServiceUnavailableException` / `ModelNotReadyException`, a failed connection |
| `RATE_LIMITED` | A transient 429; the engine's own backoff ran first | OpenAI `rate_limit_exceeded`, Bedrock `ThrottlingException`, Gemini per-minute quota |
| `QUOTA_EXHAUSTED` | Billing, credit, quota, or a spend cap is used up; retrying the same model is pointless | OpenAI `insufficient_quota` / `billing_hard_limit_reached`, Anthropic "credit balance is too low" or a reached workspace spend cap ("API usage limits"), Bedrock `ServiceQuotaExceededException`, Gemini plan or daily quota |
| `AUTH_REJECTED` | The provider rejected the credentials or the account; retrying the same provider is pointless | 401/403, Anthropic `authentication_error` / `permission_error` (including a disabled organization), Gemini "API key not valid" (a 400), Bedrock `AccessDeniedException` / `UnrecognizedClientException` / `ExpiredTokenException` |
| `MODEL_UNAVAILABLE` | The model id is unknown, retired, or not offered in the region | 404, Bedrock `ResourceNotFoundException` or an invalid model identifier |
| `None` | Another model would not help | Validation errors, missing local credentials, a client-side timeout, a refusal |

`is_transient` is True for `UNAVAILABLE` and `RATE_LIMITED`. Engines that run
their own retry schedule (Bedrock, Gemini, and the OpenAI `strict_schema_prompt`
loop) stop retrying as soon as an error classifies as `QUOTA_EXHAUSTED`,
`AUTH_REJECTED`, or `MODEL_UNAVAILABLE`, instead of sleeping through the schedule. Gemini reports
a per-minute limit and a daily quota with the same 429 text, so the split
reads the quota id in the message (`PerMinute` versus `PerDay`).

```python
from ai_api_unified import AiFallbackReason, AiProviderRequestError

try:
    turn = client.send_conversation("system", messages, tools=tools)
except AiProviderRequestError as error:
    if error.fallback_reason in (
        AiFallbackReason.UNAVAILABLE,
        AiFallbackReason.QUOTA_EXHAUSTED,
    ):
        turn = backup_client.send_conversation("system", messages, tools=tools)
    elif error.is_transient:
        backoff_and_retry()
    else:
        raise
```

You do not have to write that loop yourself: a [fallback chain](#model-fallback)
does it inside the client.

### Batch completions (Anthropic)

The `claude` engine can process many prompts as one asynchronous batch through
Anthropic's Message Batches API. Batches run in the background (most finish well
under an hour, up to a 24-hour ceiling) at roughly half the per-token cost of
individual calls — use them for bulk work that can wait.

Batch support is capability-gated like streaming and token counting: check
`capabilities.supports_batch` before calling. Every cataloged `claude` model
supports it; other engines raise `AiProviderCapabilityUnsupportedError`.

Each request carries a `custom_id` you choose. Results come back in arbitrary
order, keyed by that `custom_id`.

The blocking convenience path submits, polls, and returns results in one call:

```python
from ai_api_unified import AIBatchRequestItem, AIFactory

client = AIFactory.get_ai_completions_client(completions_engine="claude")

requests = [
    AIBatchRequestItem(custom_id="a", prompt="Summarize: the cat sat on the mat."),
    AIBatchRequestItem(custom_id="b", prompt="Translate to French: good morning."),
]

if client.capabilities.supports_batch:
    results = client.run_batch(requests, poll_interval_seconds=30)
    for item in results:
        print(item.custom_id, item.status, item.text)
```

For explicit control, drive the lifecycle
directly — submit, poll status, then fetch results once the batch has ended:

```python
job = client.submit_batch(requests)
print(job.batch_id, job.status)          # AIBatchStatus.IN_PROGRESS

job = client.get_batch(job)               # refresh status + per-state counts
if job.is_terminal:                        # ENDED, FAILED, EXPIRED, or CANCELED
    for item in client.get_batch_results(job):
        if item.status.value == "succeeded":
            print(item.custom_id, item.text)
        else:
            print(item.custom_id, "failed:", item.error_message)

# client.cancel_batch(job)                 # request cancellation while in progress
```

Request prompts are PII-redacted (when the redaction middleware is enabled)
before submission, exactly as `send_prompt` redacts. Each `AIBatchRequestItem`
also accepts an optional `system_prompt` and `max_response_tokens`. Successful
result items carry the per-request `provider_prompt_tokens` /
`provider_completion_tokens` for cost attribution.

### Model pricing

Each model's rates are exposed through `capabilities.pricing` as a structured
`AIModelPricing`: separate per-1M input, output, and cached-input rates (a
`Decimal`), with an effective date, source, and confidence. Compute the cost of
a call from the token counts the provider reports:

```python
client = AIFactory.get_ai_completions_client(model_name="gpt-5.4")
pricing = client.capabilities.pricing
print(pricing.token_rates.input_per_1m, pricing.token_rates.output_per_1m)

usd = client.compute_completion_cost(input_tokens=1200, output_tokens=800)

# Cache-aware. Pass the NON-CACHED remainder as input_tokens: of 1200 prompt
# tokens with 400 served from cache, 800 bill at the input rate and 400 at the
# cached rate. Cache writes are counted separately by the provider, so they add
# on top rather than coming out of the prompt count.
usd = client.compute_completion_cost(
    input_tokens=800,          # 1200 prompt tokens minus the 400 cache reads
    output_tokens=800,
    cached_input_tokens=400,
    cache_write_5m_tokens=2000,
    cache_write_1h_tokens=1000,
)
```

Note the difference from the cost event, where `input_tokens` *includes* the
cached subset because that is what providers report. This API takes the split
already applied.

Embeddings clients expose `compute_embedding_cost(input_tokens=...)`. The rate
tables live in a single pricing registry (`ai_api_unified.pricing`) keyed by
`(provider, model)`; see `docs/pricing_research.md` for the full table and
sources. The blended `price_per_1k_tokens` and `calculate_cost` are deprecated
shims over the split rates.

### Deprecated and retired models

The pricing registry also carries model lifecycle status. Requesting a
**retired** model (one the provider no longer serves) raises
`AiProviderConfigurationError` at construction and names a replacement,
surfacing the problem at setup. Requesting a **deprecated** model logs a
warning and emits a `DeprecationWarning` once per process, naming the sunset
date and replacement, then proceeds. Set `AI_STRICT_DEPRECATIONS=1` to escalate
deprecated models to the same construction-time error (useful in CI).

A retired model keeps whatever pricing it carried while it was active, so cost
enrichment can still price usage recorded before the withdrawal date. As of
2.22.0, `claude-opus-4-1` is retired (withdrawn 2026-08-05); use
`claude-opus-5`. As of 2.28.0, OpenAI `sora-2` / `sora-2-pro` (the Videos
API shut down 2026-09-24 with no replacement), `dall-e-2` / `dall-e-3`,
Google Imagen 4, and Veo 2.0 / 3.0 are retired. `o4-mini`, `gpt-4.1-nano`,
and `gpt-image-1` are deprecated (shutdown 2026-10-23), as are
`gpt-image-1-mini` and `gpt-image-1.5` (2026-12-01) and
`gemini-2.5-flash-image` (2026-10-02).

### Embeddings

Providers are interchangeable: the code below is identical for every
embeddings engine — only the engine name (and the vector dimensionality,
reported by `capabilities.default_dimensions`) differs.

```python
from ai_api_unified import AIFactory, AIBaseEmbeddings

client: AIBaseEmbeddings = AIFactory.get_ai_embedding_client()  # engine from env
result: dict[str, object] = client.generate_embeddings("hello world")
embedding = result.get("embedding")
print(len(embedding) if embedding else None)

# Same call, explicit engines — identical signatures and return shapes:
openai_client = AIFactory.get_ai_embedding_client(embedding_engine="openai")
voyage_client = AIFactory.get_ai_embedding_client(embedding_engine="voyage")
for client in (openai_client, voyage_client):
    result = client.generate_embeddings("hello world", input_type="document")
    print(client.model_name, result["dimensions"])
```

The `voyage` engine (extra: `voyage`, auth: `VOYAGE_API_KEY`) serves Voyage
AI's embeddings specialist models — `voyage-4-large`, `voyage-4`,
`voyage-4-lite`, `voyage-code-4`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-3`
(default), `voyage-3-lite`, `voyage-3-large`, and the domain variants
`voyage-code-3`, `voyage-finance-2`, `voyage-law-2` — with registry pricing so cost events
work like completions. Anthropic recommends Voyage for embeddings since
Anthropic serves none.

- `input_type` ("query" | "document") is a provider-neutral retrieval hint:
  the `voyage` engine forwards it to the API; other engines ignore it.
- Batching: `generate_embeddings_batch(texts)` accepts any list length; the
  `voyage` engine chunks internally at the provider's 128-text request cap.
- Async: `agenerate_embeddings` / `agenerate_embeddings_batch` on engines
  whose capabilities declare `supports_async` (currently `voyage`); others
  raise `AiProviderCapabilityUnsupportedError`.
- The `voyage` engine honors `retry_policy="none"` (constructor or
  `COMPLETIONS_RETRY_POLICY=none`) and wraps provider failures in
  `AiProviderRequestError` carrying `status_code`, matching the completions
  clients.

### Multimodal Embeddings

Every embeddings client exposes a `capabilities` descriptor stating which input
types the configured model supports. Text-only models raise
`AiProviderCapabilityUnsupportedError` from `generate_embeddings_multimodal`.
Google `gemini-embedding-2` embeds interleaved text, images, video, audio, and
PDFs into one vector space (set `EMBEDDING_MODEL_NAME=gemini-embedding-2`):

```python
from ai_api_unified import (
    AIEmbeddingsMultimodalParams,
    AIFactory,
    SupportedDataType,
)

client = AIFactory.get_ai_embedding_client()  # EMBEDDING_MODEL_NAME=gemini-embedding-2

if SupportedDataType.IMAGE in client.capabilities.supported_data_types:
    with open("bicycle.png", "rb") as file_image:
        params = AIEmbeddingsMultimodalParams(
            text="a red bicycle",
            included_types=[SupportedDataType.IMAGE],
            included_data=[file_image.read()],
            included_mime_types=["image/png"],
        )
    result = client.generate_embeddings_multimodal(params)
    print(result["dimensions"])
```

Media attachments are capped at 20MB per attachment and per request. They also
require API-key auth (`GOOGLE_AUTH_METHOD=api_key`): the google-genai SDK sends
only text parts to the Vertex embedding endpoint, so in service-account mode
the client rejects media attachments up front.

### Image Generation

```python
from ai_api_unified import AIFactory, AIBaseImageProperties, AIBaseImages

client: AIBaseImages = AIFactory.get_ai_images_client()
images: list[bytes] = client.generate_images(
    "A watercolor skyline at sunrise.",
    AIBaseImageProperties(width=1024, height=1024, format="png", num_images=1),
)

with open("generated_image.png", "wb") as generated_file:
    generated_file.write(images[0])
```

### Video Generation

The blocking convenience path is:

```python
from pathlib import Path

from ai_api_unified import AIFactory, AIBaseVideoProperties, AIBaseVideos

client: AIBaseVideos = AIFactory.get_ai_video_client()
result = client.generate_video(
    "A cinematic tracking shot of a neon train crossing the desert at dusk.",
    AIBaseVideoProperties(output_dir=Path("./generated_videos")),
)

video_bytes: bytes = result.artifacts[0].read_bytes()
frames: list[bytes] = AIBaseVideos.extract_image_frames_from_video_buffer(
    video_bytes,
    time_offsets_seconds=[0.0, 1.0],
)
AIBaseVideos.save_image_buffers_as_files(
    frames,
    output_dir=Path("./generated_frames"),
)
```

For explicit job control:

```python
from ai_api_unified import AIFactory, AIBaseVideos

client: AIBaseVideos = AIFactory.get_ai_video_client()
job = client.submit_video_generation(
    "A stop-motion paper city waking up at sunrise."
)
job = client.wait_for_video_generation(job)
result = client.download_video_result(job)
print(result.job.status, result.artifacts[0].file_path)
```

#### Kick Off Google Gemini Video Generation

Environment:

```dotenv
VIDEO_ENGINE=google-gemini
VIDEO_MODEL_NAME=veo-3.1-lite-generate-preview
GOOGLE_GEMINI_API_KEY=...
GOOGLE_AUTH_METHOD=api_key
```

Code:

```python
from pathlib import Path

from ai_api_unified import AIFactory, AIBaseVideoProperties, AIBaseVideos

client: AIBaseVideos = AIFactory.get_ai_video_client()
result = client.generate_video(
    "A cinematic dolly shot of a red vintage train moving through a desert at sunset.",
    AIBaseVideoProperties(
        output_dir=Path("./generated_videos/google"),
        timeout_seconds=1200,
        poll_interval_seconds=10,
    ),
)

print(result.artifacts[0].file_path)
```

#### Kick Off OpenAI Video Generation

OpenAI shut down Sora 2 and the Videos API on 2026-09-24 with no
replacement. Both `sora-2` models are now retired, so constructing this
client raises `AiProviderConfigurationError`. The example below is kept
for reference only; use the `google-gemini` or `nova-reel` video engine.

Environment:

```dotenv
VIDEO_ENGINE=openai
VIDEO_MODEL_NAME=sora-2
OPENAI_API_KEY=...
```

Code:

```python
from pathlib import Path

from ai_api_unified import AIFactory, AIBaseVideoProperties, AIBaseVideos

client: AIBaseVideos = AIFactory.get_ai_video_client()
result = client.generate_video(
    "A wide cinematic shot of gentle ocean waves meeting a rocky coastline at golden hour.",
    AIBaseVideoProperties(
        output_dir=Path("./generated_videos/openai"),
        timeout_seconds=1200,
        poll_interval_seconds=10,
    ),
)

print(result.artifacts[0].file_path)
```

Frame extraction requires the optional `video_frames` extra.

### Voice

```python
from ai_api_unified import AIFactory, AIVoiceBase

voice: AIVoiceBase = AIFactory.get_ai_voice_client()
audio_bytes: bytes = voice.text_to_voice(
    text_to_convert="Hello from ai-api-unified",
    voice=voice.get_default_voice(),
    audio_format=voice.default_audio_format,
)

with open(f"out{voice.default_audio_format.file_extension}", "wb") as output_file:
    output_file.write(audio_bytes)
```

`text_to_voice` is keyword-only, and `voice` and `audio_format` are required;
`get_default_voice()` and `default_audio_format` supply the provider's own
defaults. Use `stream_audio(text)` for the streaming variant.

`AIFactory.get_ai_voice_client()` reads `AI_VOICE_ENGINE` and accepts an
explicit `voice_engine` override, matching the other capability constructors.
It delegates to `AIVoiceFactory.create()`, which remains supported and callable
with no arguments.

Calls are attributed to `OPENAI_USER` on the OpenAI engine, falling back to the
`default_user` sentinel when it is unset — the same behavior the OpenAI
completions, embeddings, images, and videos clients already have. The other
voice engines expose no vendor user setting, so their calls carry no
originating caller id unless the application sets one through the
observability context.

The OpenAI voice client shares `AIOpenAIBase` with the other OpenAI-backed
capabilities, so it honors `COMPLETIONS_RETRY_POLICY` and `OPENAI_BASE_URL_OVERRIDE`,
and accepts the same `retry_policy` and `base_url` constructor arguments:

```python
voice = AIFactory.get_ai_voice_client(
    voice_engine="openai",
    base_url="https://gateway.example.com/v1",
    retry_policy="none",
)
```

Only the `openai` voice engine accepts these; passing them to `google`,
`azure`, or `elevenlabs` raises `AiProviderCapabilityUnsupportedError` rather
than ignoring them.

A blank retry-policy setting (`COMPLETIONS_RETRY_POLICY=`) is treated as
unconfigured and falls back to `default` across every OpenAI-backed capability.

## Model fallback

A fallback chain keeps completions flowing when a model cannot serve a
request. You list the models to try, in order; the factory returns a client
with the same interface as a single engine; and a request that fails on one
candidate because that model is overloaded, rate limited, or out of quota is
retried on the next. The candidates can be on the same provider
(`claude-opus-5` then `claude-sonnet-5`) or on different ones
(`claude` then `openai` then `google-gemini`).

### Why it exists

Provider incidents, per-minute rate limits, and exhausted monthly quotas all
look the same to an application: a request that raises. Before this feature
you caught `AiProviderRequestError`, built a second client, and re-sent the
request by hand, in every code path. The chain moves that logic into the
client, and it uses the classification every engine already puts on the
error (`fallback_reason`, see
[Retry policy and typed request errors](#retry-policy-and-typed-request-errors))
so it acts only on failures another model could serve.

### Setting up a chain

From configuration, which is the usual way:

```bash
# The primary stays where it always was.
COMPLETIONS_ENGINE=claude
COMPLETIONS_MODEL_NAME=claude-opus-5

# Ordered engine:model pairs to try after the primary fails.
COMPLETIONS_FALLBACKS=openai:gpt-5.6-luna,google-gemini:gemini-3.7-flash

# Optional. Which failures trigger a move; this is the default.
COMPLETIONS_FALLBACK_ON=unavailable,rate_limited,quota_exhausted,auth_rejected
```

Or in code, which overrides the settings:

```python
from ai_api_unified import AIFactory, AIFallbackCandidate, AiFallbackReason

client = AIFactory.get_ai_completions_client(
    completions_engine="claude",
    model_name="claude-opus-5",
    fallbacks=[
        ("openai", "gpt-5.6-luna"),
        AIFallbackCandidate(engine="google-gemini", model="gemini-3.7-flash"),
    ],
    fallback_on={AiFallbackReason.UNAVAILABLE, AiFallbackReason.QUOTA_EXHAUSTED},
)
```

Rules for the chain:

- The split in `COMPLETIONS_FALLBACKS` is on the first colon only, so Bedrock
  ids keep theirs: `bedrock:amazon.nova-lite-v1:0`. A pair with no model
  (`google-gemini`) uses that engine's default model.
- Each fallback engine needs its own credentials and optional extra, exactly
  as it would if it were the primary.
- The factory checks every fallback engine token against the provider
  registry when it builds the client, so a typo fails at startup, long
  before any outage.
- Passing `fallbacks=[]` in code turns the chain off even when the setting
  is present.

### What happens on a request

1. The request goes to the primary. The engine's own retry schedule runs
   first, so a rate limit fails over only after backoff runs out.
2. If the engine raises `AiProviderRequestError` with a `fallback_reason` in
   the configured set, the client logs a warning naming both candidates and
   the reason, and sends the request to the next candidate.
3. The client builds a fallback the first time a request needs it. When a
   build fails (missing extra, bad credentials, unsupported option), the
   client logs it once and skips that candidate for the rest of the
   process, so a broken fallback never breaks startup or blocks the rest
   of the chain.
4. The client skips, with a warning, a fallback that lacks a capability the
   call needs (structured output, tool use, async, streaming).
5. After a `QUOTA_EXHAUSTED` or `AUTH_REJECTED` failure, the client skips
   the call's later candidates on the same account (same engine and
   `base_url`), with a warning, since they would fail the same way. Gemini
   quotas and Bedrock `ServiceQuotaExceededException` / `AccessDeniedException`
   apply to one model, so after those the client still tries the account's other models.
6. When every candidate has failed, the client raises the last failure.

| Reason | Fails over by default | Typical causes |
| ------ | --------------------- | -------------- |
| `UNAVAILABLE` | yes | 5xx, Anthropic 529 overloaded, Bedrock `ServiceUnavailableException`, a failed connection |
| `RATE_LIMITED` | yes | 429 after the engine's backoff is exhausted |
| `QUOTA_EXHAUSTED` | yes | OpenAI `insufficient_quota`, Anthropic low credit balance or workspace spend cap, Bedrock `ServiceQuotaExceededException`, a Gemini daily quota |
| `AUTH_REJECTED` | yes | 401/403, a revoked or invalid API key, a disabled organization, expired AWS credentials |
| `MODEL_UNAVAILABLE` | no | 404, an unknown or retired model id, a model not offered in the region |
| `None` | never | validation errors, a client-side timeout, a safety refusal |

`MODEL_UNAVAILABLE` is off by default because an unknown model id usually
means a configuration typo, and a working fallback would hide it. Turn it on
with `fallback_on` or `COMPLETIONS_FALLBACK_ON` when you want an unknown or
retired model to fail over.

`QUOTA_EXHAUSTED` and `AUTH_REJECTED` are usually account-level failures:
every model on that provider's account fails the same way; the client skips
the account's other candidates (step 5), so a chain needs a candidate on a
different provider to recover from them. `AiProviderRequestError.account_wide`
reports which case a given error is. A rejected key is
often a configuration error too; the `failover` and `served_by_fallback` log
events (below) record it while traffic keeps flowing. Remove
`auth_rejected` from `fallback_on` to make it fail loudly instead.

A safety refusal is never a trigger: retrying a
refused request on another model is a policy decision, not a reliability one,
and it belongs in your code if you want it.

### Seeing which model answered

Every failover is logged, on the logger
`ai_api_unified.completions.ai_fallback_completions`, and each record carries
an `ai_fallback_event` field plus the candidates and reason involved, so a
log processor can count and alert on them without parsing the message:

| Level | `ai_fallback_event` | When |
| ----- | ------------------- | ---- |
| ERROR | `failover` | A candidate failed with a trigger reason and the request is moving to the next one. Fields: `operation`, `fallback_from`, `fallback_to`, `fallback_reason`, `status_code`. |
| WARNING | `served_by_fallback` | A fallback answered a request the primary could not. Once per request that moved. Fields: `operation`, `fallback_from`, `fallback_to`, `fallback_reason`. |
| ERROR | `chain_exhausted` | Every candidate failed; the last error is about to be raised. |
| ERROR | `candidate_unbuildable` | A fallback could not be constructed and is out of the chain for this process. |
| WARNING | `candidate_skipped` | A fallback lacks a capability the call needs, or its account already failed the call with `QUOTA_EXHAUSTED` or `AUTH_REJECTED`. |

The messages start with `FALLBACK`, `FALLBACK SERVED`, or `FALLBACK
EXHAUSTED`, so they stand out in plain log output too. A request the primary
serves logs nothing. In code:

```python
turn = client.send_conversation("system", messages, tools=tools)
turn.provider_engine, turn.model_name     # the candidate that served the turn

result = client.send_structured_output(prompt, response_model=Report)
result.provider_engine, result.model_name

text = client.send_prompt("...")
client.last_route                          # AIFallbackCandidate for the last call
client.last_route.label                    # "openai:gpt-5.6-luna"
```

Cost follows the route with no extra configuration: each engine emits its own
observability and cost events, so a call served by a fallback is billed to
that engine and model. `client.compute_completion_cost(...)` and
`price_per_1k_tokens()` price at the candidate that served the most recent
call. `client.capabilities` reports the primary's capabilities.

### Conversations, streaming, and batches

The engines shape their requests differently, and three consequences follow.

**Conversations fail over on the first turn, then stay put.** A history that
holds engine-shaped entries (replayed `raw_content`, tool results) cannot be
sent to another engine. While the history is plain user, assistant, and
system text, a turn can fail over. After that the client reads which engine
family shaped the history and sends the turn to a candidate of that family,
with fallback off; a failure there propagates. Because routing is read from
the history, one client can serve conversations on different engines at the
same time.

```python
messages = [{"role": "user", "content": "Summarize ticket VL-123."}]
turn = client.send_conversation("You are a support agent.", messages, tools=tools)
client.extend_messages_with_turn(messages, turn)          # shape of the engine that answered
for call in turn.tool_calls:
    messages.append(
        client.build_tool_result_message(
            tool_call_id=call.id,
            result=run_tool(call),
            messages=messages,                             # lets it match the history's shape
        )
    )
turn = client.send_conversation("You are a support agent.", messages, tools=tools)
```

Pass `messages=` to `build_tool_result_message` whenever one client serves
more than one conversation; without it the result takes the shape of the
engine that served the most recent call.

**Streams fail over only before the first chunk.** Once a candidate has
yielded output the caller already holds partial text, so a later error
propagates instead of restarting on another model.

**Batches and token counts always use the primary.** Batch jobs are
provider-specific, and token counts are model-specific.

### Per-call control

`provider_options={"fallback": "none"}` keeps one request on the primary,
for a call where a different model would be the wrong answer even during an
outage. The client removes the key before the options reach an engine, so
the same call works on a plain engine client too.

The prompt cache hint (`AIPromptCacheHint`) is provider-neutral and carries
across the chain.

### Configuration reference

| Setting | Meaning |
| ------- | ------- |
| `COMPLETIONS_FALLBACKS` | Ordered `engine:model` pairs tried after the primary fails. Blank or unset means no chain. |
| `COMPLETIONS_FALLBACK_ON` | Comma-separated `AiFallbackReason` values that trigger a move. Default `unavailable,rate_limited,quota_exhausted,auth_rejected`. |
| `fallbacks=` (factory argument) | Same as the setting, as `(engine, model)` tuples or `AIFallbackCandidate` objects; overrides the setting. `[]` turns the chain off. |
| `fallback_on=` (factory argument) | Same as the setting, as reasons or their values; overrides the setting. |

Exports: `AiFallbackCompletions`, `AIFallbackCandidate`,
`DEFAULT_FALLBACK_REASONS`, `AiFallbackReason`.

### Testing a chain against live providers

A live test runs Claude as the primary with an unknown model id and OpenAI
as the fallback, so a real 404 drives a real second request. It also covers
a healthy primary that never builds its fallback and a first conversation
turn that fails over:

```bash
poetry run pytest -m nonmock tests/test_model_fallback_nonmock.py -q
```

It needs `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` in `.env` and skips without
them.

## Configuration

### Required Engine Selectors

There is no implicit default provider. Set the selector for each capability you use.

| Environment variable | Valid values                                                                                                                  |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `COMPLETIONS_ENGINE` | `openai`, `openai-responses`, `openai-compatible`, `claude`, `google-gemini`, `bedrock`, Bedrock-routed aliases such as `nova`, `anthropic`, `llama`, `mistral`, `cohere`, `ai21`, `rerank` |
| `EMBEDDING_ENGINE`   | `openai`, `titan`, `google-gemini`                                                                                            |
| `IMAGE_ENGINE`       | `openai`, `google-gemini`, `nova-canvas`, `bedrock`, `nova`                                                                   |
| `VIDEO_ENGINE`       | `openai`, `google-gemini`, `bedrock`, `nova`, `nova-reel`                                                                     |
| `AI_VOICE_ENGINE`    | `openai`, `google`, `azure`, `elevenlabs`                                                                                     |

### Common Model Settings

| Environment variable        | Notes                                                                                       |
| --------------------------- | ------------------------------------------------------------------------------------------- |
| `COMPLETIONS_MODEL_NAME`    | Optional completions model override                                                         |
| `COMPLETIONS_FALLBACKS`     | Optional ordered `engine:model` pairs tried after the primary fails (see Model fallback)     |
| `COMPLETIONS_FALLBACK_ON`   | Optional comma-separated `AiFallbackReason` values that trigger a fallback; default `unavailable,rate_limited,quota_exhausted,auth_rejected` |
| `EMBEDDING_MODEL_NAME`      | Optional embeddings model override. `gemini-embedding-2` enables multimodal embeddings.     |
| `IMAGE_MODEL_NAME`          | Optional image model override                                                               |
| `VIDEO_MODEL_NAME`          | Optional video model override                                                               |
| `VIDEO_OUTPUT_DIR`          | Optional local output directory for materialized video artifacts                            |
| `VIDEO_POLL_INTERVAL_SECONDS` | Optional default poll interval for video job waits                                        |
| `VIDEO_TIMEOUT_SECONDS`     | Optional default timeout for blocking video generation                                      |
| `BEDROCK_VIDEO_OUTPUT_S3_URI` | Required for Nova Reel unless provided per request                                        |
| `DEFAULT_GEMINI_TTS_MODEL`  | Optional Google voice model override                                                        |
| `EMBEDDING_DIMENSIONS`      | Optional embeddings dimension override. Leave unset for provider defaults.                  |
| `AI_API_GEO_RESIDENCY`      | Optional geo hint. `US`, `USA`, or `United States` normalize to US routing where supported. |
| `AI_MIDDLEWARE_CONFIG_PATH` | Optional YAML config path for observability and PII middleware                              |
| `ANTHROPIC_BASE_URL_OVERRIDE` | Optional API base-URL override for the `claude` engine (https required) |
| `OPENAI_BASE_URL_OVERRIDE`  | Optional API base-URL override for the OpenAI engines (https required) |
| `GOOGLE_GEMINI_BASE_URL_OVERRIDE` | Optional API base-URL override for `google-gemini` (https required) |
| `ANTHROPIC_ADMIN_BASE_URL_OVERRIDE` | Optional separate override for the Anthropic Admin API lookup; the admin key does not follow the inference override |

### Cataloged Completions Models

The full list of cataloged models for every capability is in
[Supported models](#supported-models). Engine defaults sit one generation
behind the newest cataloged model. Gemini images generate through
`generate_content` because Imagen 4 is retired, and `person_generation`
on Gemini images applies only in Vertex AI mode; the Gemini Developer API
rejects it.

An uncataloged model name passes through to the provider on the OpenAI and
Claude engines (with a conservative default context window); the
`google-gemini` engine falls back to its default model and logs a warning.
Deprecated models warn once per process with a sunset date and replacement;
retired models fail fast (see the pricing registry).

### API Base URL Overrides

The `claude`, `openai`, `openai-responses`, `openai-compatible`, and
`google-gemini` engines accept a base-URL override, so provider traffic can
route through an LLM gateway, a corporate egress proxy, a recording proxy in
tests, or any OpenAI-compatible server:

| Engine | Setting |
| --- | --- |
| `claude` | `ANTHROPIC_BASE_URL_OVERRIDE` |
| `openai`, `openai-responses` | `OPENAI_BASE_URL_OVERRIDE` |
| `openai-compatible` | `OPENAI_COMPATIBLE_BASE_URL` (required) |
| `google-gemini` | `GOOGLE_GEMINI_BASE_URL_OVERRIDE` |

```dotenv
# Route Claude traffic through a gateway
ANTHROPIC_BASE_URL_OVERRIDE=https://llm-gateway.internal/anthropic

# Point the openai engine at a local OpenAI-compatible server
OPENAI_BASE_URL_OVERRIDE=http://localhost:11434/v1
```

Per client, when one process needs several destinations:

```python
client = AIFactory.get_ai_completions_client(
    completions_engine="claude",
    base_url="https://llm-gateway.internal/anthropic",
)
```

Rules and rationale:

- **https is required.** An override redirects your API keys to that host, so
  plain `http://` is rejected with `AiProviderConfigurationError` unless the
  host is `localhost`, `127.0.0.1`, or `::1` (local gateways and test
  servers).
- **The `_OVERRIDE` suffix is deliberate.** The OpenAI, Anthropic, and Google
  SDKs read their own `OPENAI_BASE_URL`, `ANTHROPIC_BASE_URL`, and
  `GOOGLE_GEMINI_BASE_URL` variables. Using distinct names, and passing the
  resolved value to the SDK explicitly, keeps a stray native variable from
  taking effect without the https check. (On `google-gemini` with no override
  set, the SDK still resolves its own default, including those native
  variables, which this library does not validate.)
- **Precedence:** the `base_url` argument, then `<ENGINE>_BASE_URL_OVERRIDE`,
  then (OpenAI only) the deprecated `OPENAI_BASE_URL` and geo-residency
  routing, then the provider default.
- **Organization lookups follow the override**, so finops attribution does not
  bypass your gateway — with one deliberate exception: the Anthropic **Admin
  API key** grants org-wide read/write, far beyond the inference key, so it
  does not follow `ANTHROPIC_BASE_URL_OVERRIDE`. Routing inference through a
  gateway never silently hands that gateway an administration credential. Set
  `ANTHROPIC_ADMIN_BASE_URL_OVERRIDE` to opt the admin lookup in as well.
- The same rules validate the deprecated `OPENAI_BASE_URL`, since other
  tooling commonly sets that name process-wide.
- Engines without SDK support (Bedrock-routed, `titan`, `voyage`, voice)
  raise `AiProviderCapabilityUnsupportedError` when passed `base_url`, rather
  than ignoring it silently.

### Provider Authentication

#### OpenAI

Required:

- `OPENAI_API_KEY`

Common optional settings:

- `OPENAI_BASE_URL`
- `COMPLETIONS_MODEL_NAME`
- `EMBEDDING_MODEL_NAME`
- `IMAGE_MODEL_NAME`
- `VIDEO_MODEL_NAME`
- `EMBEDDING_DIMENSIONS`
- `AI_API_GEO_RESIDENCY`

#### Anthropic (native Claude API)

Required for the `claude` completions engine:

- `ANTHROPIC_API_KEY`

Common optional settings:

- `COMPLETIONS_MODEL_NAME` (defaults to `claude-opus-5`)

Claude via Amazon Bedrock (the `anthropic` engine) does not use
`ANTHROPIC_API_KEY`; it authenticates with AWS credentials like the other
Bedrock engines below.

For current model IDs and pricing, use the Anthropic documentation:

- [Claude model overview](https://platform.claude.com/docs/en/about-claude/models/overview)
- [Claude API pricing](https://platform.claude.com/docs/en/pricing)

#### AWS Bedrock and Titan

Required:

- `AWS_REGION`
- standard AWS credentials in environment or runtime IAM context

Common optional settings:

- `COMPLETIONS_MODEL_NAME`
- `EMBEDDING_MODEL_NAME`
- `IMAGE_MODEL_NAME`
- `VIDEO_MODEL_NAME`
- `BEDROCK_VIDEO_OUTPUT_S3_URI`
- `EMBEDDING_DIMENSIONS`
- `AI_API_GEO_RESIDENCY`

If your AWS profile uses `aws login` (a `login_session` profile), install
`botocore[crt]` as well; botocore needs it to read those credentials.

##### DeepSeek, Qwen, and GLM on Bedrock

AWS hosts these models itself, so requests go to AWS under your AWS account
and never reach DeepSeek, Alibaba, or Z.ai.

```dotenv
COMPLETIONS_ENGINE=bedrock
COMPLETIONS_MODEL_NAME=deepseek.v3.2
AWS_REGION=us-east-1
AWS_PROFILE=<your profile>
```

| Model | Context | Tool calls and structured output | Notes |
| --- | ---: | --- | --- |
| `deepseek.v3.2` | 164K | yes | |
| `us.deepseek.r1-v1:0` | 128K | no | reasoning model; `strict_schema_prompt` extracts JSON from a plain reply |
| `qwen.qwen3-next-80b-a3b` | 256K | yes | reasoning |
| `qwen.qwen3-235b-a22b-2507-v1:0` | 256K | yes | us-east-2 and us-west-2 only |
| `qwen.qwen3-coder-next` | 256K | yes | coding |
| `qwen.qwen3-32b-v1:0` | 32K | yes | reasoning |
| `zai.glm-5` | 200K | yes | |
| `zai.glm-4.7` | 203K | yes | |
| `zai.glm-4.7-flash` | 203K | yes | lowest cost |

All are text-only, stream, and are priced in the registry. None supports
`count_tokens`. Enable each model under **Model access** in the Bedrock
console first. For a model outside your default region, set `AWS_REGION` or
pass `region=` to `AiBedrockCompletions`.

For current Bedrock model IDs, use the AWS documentation:

- [Supported foundation models in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html)
- [Amazon Bedrock foundation model reference](https://docs.aws.amazon.com/bedrock/latest/userguide/foundation-models-reference.html)

#### Google-backed Providers

Required for the default OSS path:

- `GOOGLE_GEMINI_API_KEY`

Default auth mode:

- `GOOGLE_AUTH_METHOD=api_key`

Optional service-account mode:

- `GOOGLE_AUTH_METHOD=service_account`
- `GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account.json`
- `GOOGLE_PROJECT_ID=<gcp-project-id>`
- `GOOGLE_LOCATION=us-central1`

This auth policy applies across Google completions, embeddings, images, videos, and voice.

##### Google auth modes per capability

The two `GOOGLE_AUTH_METHOD` values select different google-genai clients:
`api_key` creates the Gemini Developer client
(`genai.Client(api_key=...)`, generativelanguage endpoint) and
`service_account` creates the Vertex client
(`genai.Client(vertexai=True, project=..., location=...)`). Some capabilities
work in only one client:

| Google capability | `api_key` (Developer) | `service_account` (Vertex) | Notes |
| --- | :---: | :---: | --- |
| Completions (incl. streaming) | ✅ | ✅ | |
| Text embeddings | ✅ | ✅ | |
| Image generation (Gemini image models) | ✅ | ✅ | `person_generation` is sent only in Vertex mode |
| Multimodal (media) embeddings | ✅ | ❌ | the SDK sends only text parts to the Vertex embedding endpoint, so the library raises `NotImplementedError` when media is attached in Vertex mode |
| Text-to-video with local download | ✅ | ❌ | downloaded via the Files API (`client.files.download`), which the SDK supports only in the Developer client; under Vertex set `download_outputs=False` to receive a remote `gs://` URI instead |
| `source_video` (video continuation) | ❌ | ✅ | the library raises `NotImplementedError` in `api_key` mode; this path requires Vertex |
| Voice (Gemini TTS / STT) | ⚠️ | ✅ | the library wires API keys into the Cloud TTS/STT clients. Google may reject a key not enabled for those APIs, so `service_account` is the reliable mode |

Pick the mode for what you need: multimodal-media embeddings and
text-to-video-with-download require `api_key`, and `source_video` requires
`service_account`. Running both families means switching `GOOGLE_AUTH_METHOD`
between runs or constructing two clients. For a single call, `get_client(...,
use_api_key=True)` on the shared Google base forces the Developer client
without changing the environment.

#### Azure TTS

Required:

- `MICROSOFT_COGNITIVE_SERVICES_API_KEY`
- `MICROSOFT_COGNITIVE_SERVICES_REGION`

Optional:

- `MICROSOFT_COGNITIVE_SERVICES_ENDPOINT`
- `AI_VOICE_LANGUAGE`

#### ElevenLabs

Required:

- `ELEVEN_LABS_API_KEY`

## Lazy Loading and Imports

Prefer the stable interfaces and factories exported from the root package:

```python
from ai_api_unified import (
    AIFactory,
    AIVoiceFactory,
    AIBaseCompletions,
    AIBaseEmbeddings,
    AIBaseImages,
    AIBaseVideos,
    AIVoiceBase,
    AIEmbeddingsCapabilitiesBase,
    AIEmbeddingsMultimodalParams,
    AIIncludedMediaParamsBase,
    AiProviderCapabilityUnsupportedError,
    SupportedDataType,
)
```

Factory entry points:

- `AIFactory.get_ai_completions_client()`
- `AIFactory.get_ai_embedding_client()`
- `AIFactory.get_ai_images_client()`
- `AIFactory.get_ai_video_client()`
- `AIFactory.get_ai_voice_client()` (or `AIVoiceFactory.create()`)

Concrete providers are not re-exported from package `__init__.py` modules. If you need a concrete class directly, import it from the module that defines it, for example:

```python
from ai_api_unified.completions.ai_google_gemini_completions import (
    GoogleGeminiCompletions,
)
```

Typical factory failure modes:

- unsupported engine selector: `ValueError`
- selected provider extra is not installed: `AiProviderDependencyUnavailableError`
- provider load/runtime failure: `AiProviderRuntimeError`
- input modality unsupported by the configured embedding model: `AiProviderCapabilityUnsupportedError`
- retired model requested (or deprecated model under `AI_STRICT_DEPRECATIONS`): `AiProviderConfigurationError`

## Middleware

A YAML file named by `AI_MIDDLEWARE_CONFIG_PATH` configures middleware.

### Observability

Observability middleware is built into the base package and does not require a separate extra.

Features:

- metadata-only input, output, and error events
- request-scoped correlation via `set_observability_context(...)`
- coverage for completions, embeddings, images, videos, and text-to-speech

Minimal config:

```yaml
middleware:
  - name: 'observability'
    enabled: true
```

#### Request-scoped context and tags

`set_observability_context` stores request-scoped correlation values in a
`contextvars.ContextVar`, so they apply to every library call made in that
context (including across `await` boundaries) without threading parameters
through application code. It accepts:

- `caller_id` — stable caller identifier; emitted as `originating_caller_id`
  on lifecycle events and `caller_id` on cost events
- `session_id` / `workflow_id` — emitted in event metadata when set
- `tags` — arbitrary string-to-string mapping (for example `run_id`,
  `node_id`, or a workflow name); each tag is emitted as a `tag_<name>` field
  on every event, including cost-topic events

The function returns a token; pass it to `reset_observability_context` to
restore the prior context (for example in a request-handler `finally` block).
Blank keys and values are dropped.

```python
from ai_api_unified.middleware import (
    set_observability_context,
    reset_observability_context,
)

token = set_observability_context(
    caller_id="workflow-service",
    tags={"run_id": run_id, "node_id": node_id, "workflow": workflow_name},
)
try:
    turn = client.send_conversation(system_prompt, messages, tools=tools)
finally:
    reset_observability_context(token)
```

Token usage is also available on every result object — `AITurnResult.usage`
per conversation turn and `AIStructuredOutputResult.usage` per structured call
— so billing attribution does not require parsing logs.

See [`docs/observability_middleware_example.yaml`](docs/observability_middleware_example.yaml) and [`docs/observability_middleware_design.md`](docs/observability_middleware_design.md).

#### Cost tracking (financial-ops)

Set `emit_cost: true` on the observability settings to attach a USD cost to each
call. Cost is computed from the provider-reported token counts and the model's
registry pricing (`capabilities.pricing`). The result is emitted as a structured
event on a dedicated cost topic logger (`ai_api_unified.observability.cost`),
separate from the observability event stream so handlers can route it
independently. Each event carries the pricing provenance (effective date,
source, confidence) so the cost is auditable. Unpriced models are skipped. This
is observe-only — it never affects program flow.

Prompt-cache reads are captured per provider (OpenAI, OpenAI Responses,
Anthropic, Bedrock, and Google Gemini) and billed at the model's cached-input
rate: the event carries `cached_input_tokens`, and the cached subset of
`input_tokens` is priced at the cached rate while the remainder bills at the
full input rate. Providers that report cache reads separately from the input
count (Anthropic, Bedrock) are normalized so `input_tokens` includes the cached
subset, keeping the cost split consistent across providers.

Prompt-cache **writes** are billed too. Priming a cache costs more than base
input, and the premium depends on how long the cache lives, so each rate is
stored per lifetime and each event carries its own count:

| Field | Meaning |
| --- | --- |
| `cache_write_5m_tokens` | Tokens written to a 5-minute-TTL cache (Anthropic: 1.25x base input) |
| `cache_write_1h_tokens` | Tokens written to a 1-hour-TTL cache (Anthropic: 2x base input) |

Unlike cache reads, writes are **not** a subset of `input_tokens` — the provider
reports them separately, so they add to the cost rather than re-slicing the
prompt. They are likewise excluded from the provider's own total token count, so
`usd_cost / provider_total_tokens` is not a valid effective rate on a
cache-priming call; include the cache-write counts in the denominator. Anthropic supplies the per-TTL split; Bedrock reports one
`cacheWriteInputTokens` that is attributed to the 5-minute tier, the only
lifetime it exposes. OpenAI charges nothing to write a cache, and Google bills
explicit-cache storage per hour rather than per written token, so both carry an
explicit zero write rate. The two states are recorded distinctly: a rate of
zero means the provider charges nothing to populate a cache, and writes bill as
free. An absent rate means the write is charged but not yet rated — it falls
back to the base input rate, under-reporting the premium rather than dropping
the cost. Five Bedrock entries are in that second state today: the hosted Claude
model and the four Amazon Nova models, all partner-priced with no
authoritative AWS cache-write rate sourced.

```yaml
middleware:
  - name: 'observability'
    enabled: true
    settings:
      emit_cost: true
      # emit_cost_topic: 'my.cost.logger'   # optional logger-name override
```

Cost enrichment fires whenever `emit_cost` is on, even when output events are
disabled by `direction`. See [`docs/finops_middleware_design.md`](docs/finops_middleware_design.md).

Cost events also carry organization identity for billing attribution
(`org_id` / `org_name`), resolved per provider to what each platform
supports, cached per client, and failing open (cost events omit the fields
when identity is unavailable):

| Provider | `org_id` | `org_name` | How |
|---|---|---|---|
| `claude` (native Anthropic) | always | with `ANTHROPIC_ADMIN_KEY` | Admin API; else a free `count_tokens` response header |
| `openai` / `openai-responses` | always | always | account API (`/v1/me`) with the regular key; header-probe fallback |
| Bedrock-routed | always | with `iam:ListAccountAliases` | STS caller identity; IAM account alias |
| `google-gemini` | with `GOOGLE_PROJECT_ID` | unsupported | configured GCP project id |
| `voyage`, `titan` | unsupported | unsupported | the platforms expose no caller identity |

Callers can also fetch the identity on demand from any client:

```python
info = client.get_org_info()   # AIProviderOrgInfoBase
print(info.org_id, info.org_name, info.source)
# source: "admin_api" (id + name) | "response_header" (id only) | "none"
```

Unlike enrichment, the explicit call raises `AiProviderRequestError` (with
`status_code`) when resolution fails — a rejected `ANTHROPIC_ADMIN_KEY`
surfaces as a 401. Providers subclass the returned model
(`AIProviderOrgInfoAnthropic`, `AIProviderOrgInfoOpenAI`, ...) for
provider-native fields, and `client.get_org_info_capability()` declares up
front what the configured engine can resolve:

```python
capability = client.get_org_info_capability()
capability.supports_org_id, capability.supports_org_name, capability.requirement
# e.g. on Bedrock: (True, True, "org_name is the AWS account alias and needs
#                    iam:ListAccountAliases; identity falls back to id-only without it")
```

### PII Redaction

PII redaction is optional and requires one of:

- `middleware-pii-redaction`
- `middleware-pii-redaction-small`
- `middleware-pii-redaction-large`

Minimal config:

```yaml
middleware:
  - name: 'pii_redaction'
    enabled: true
    settings:
      direction: 'input_only'
```

Notes:

- `strict_mode: true` enables fail-closed behavior
- `balanced`, `high_accuracy`, and `low_memory` detection profiles are supported
- install a matching spaCy model separately, for example `poetry run python -m spacy download en_core_web_sm` for `balanced` or `poetry run python -m spacy download en_core_web_lg` for `high_accuracy`
- `middleware-pii-redaction-small` and `middleware-pii-redaction-large` are compatibility aliases for the same Python dependency set; the spaCy model is still installed as a separate build/runtime asset
- for no-egress images and Lambda-style deployments, install the spaCy model wheel into the build artifact (runtime model downloads need egress)
- recognizer customization is configured in YAML

See [`docs/pii_redaction_design.md`](docs/pii_redaction_design.md) for the fuller contract and deployment tradeoffs.

## Structured Responses

Use `AIStructuredPrompt` together with `strict_schema_prompt(...)` when you want schema-validated structured output.

```python
from copy import deepcopy
from typing import Any

from ai_api_unified import (
    AIFactory,
    AIStructuredPrompt,
    StructuredResponseTokenLimitError,
)


class ContactExtraction(AIStructuredPrompt):
    name: str | None = None
    city: str | None = None

    @staticmethod
    def get_prompt() -> str:
        return "Extract the person's name and city from: Alice lives in Paris."

    @classmethod
    def model_json_schema(cls) -> dict[str, Any]:
        schema = deepcopy(super().model_json_schema())
        schema["properties"] = {
            "name": {"type": "string"},
            "city": {"type": "string"},
        }
        schema["required"] = ["name", "city"]
        return schema


client = AIFactory.get_ai_completions_client()

try:
    result = client.strict_schema_prompt(
        prompt=ContactExtraction.get_prompt(),
        response_model=ContactExtraction,
        max_response_tokens=2048,
    )
    print(result.name, result.city)
except StructuredResponseTokenLimitError as exc:
    print(exc)
```

Key behavior:

- structured prompts are provider-agnostic at the call site
- the library validates the response against your output schema
- undersized or truncated structured responses raise `StructuredResponseTokenLimitError`

`AIStructuredPrompt.send_structured_prompt(...)` is also available when you want the prompt to live on the model instance itself.

On Bedrock, `strict_schema_prompt` picks a request each model accepts. By
default it pre-fills the reply with a JSON code fence and stops at the closing
fence. Claude 4.6 and later reject that pre-fill, so Opus 4.6 and the
open-weight models use native structured output, and the rest (Sonnet 4.6,
Claude 4.7 and later, DeepSeek R1) send a plain request and extract the JSON
from the reply. Claude 4.7 and later also get no `temperature`, which they
reject.

## Testing

Regular test run:

```bash
poetry run pytest -m "not nonmock"
```

Live provider tests:

```bash
poetry run pytest -m nonmock -s -vv
```

Notes for live tests:

- install the matching provider extras first
- configure credentials in `.env`
- some tests may skip when a provider account lacks quota, a paid image tier, or an enabled cloud service

## Release and PyPI Publishing

The OSS repository publishes to public PyPI only.

Before publishing:

1. Bump the version in `pyproject.toml`.
2. Bump the version in `src/ai_api_unified/__version__.py`.
3. Bump the version in the `README.md` title heading (line 1).
4. Run `poetry run pytest tests/test_version_sync.py` to confirm all three agree.
5. Ensure the working tree is clean.
6. Ensure your PyPI token is configured for Poetry.

Publish with the checked-in script:

```bash
./publish.sh
```

The script:

- checks for uncommitted changes
- confirms the version
- removes old build artifacts
- builds the wheel and sdist locally
- fails before upload if built metadata contains direct URL requirements that PyPI rejects
- runs `poetry publish` only after metadata validation passes

After publishing, tag and push the release:

```bash
git tag v<version>
git push origin v<version>
```

## Troubleshooting

- `COMPLETIONS_ENGINE must be configured explicitly` or similar: set the required engine selector for that capability.
- `AiProviderDependencyUnavailableError`: install the extra for the selected provider.
- Google auth errors: the OSS default is `GOOGLE_AUTH_METHOD=api_key`. If you switch to `service_account`, point `GOOGLE_APPLICATION_CREDENTIALS` at a valid local JSON credential file. Set `GOOGLE_PROJECT_ID` and `GOOGLE_LOCATION` when required.
- Unexpected embeddings dimensions: leave `EMBEDDING_DIMENSIONS` unset unless you intentionally want a non-default size.
- Google image generation or TTS failures in live tests often come from account or service state: a disabled cloud API, or missing paid-plan access. Check those before suspecting a library bug.
- `AI_API_GEO_RESIDENCY=US` is a best-effort routing hint. Only providers that expose regional routing controls can honor it directly.

## License

This project is released under the MIT License.

