Metadata-Version: 2.4
Name: model-calling
Version: 0.2.0
Summary: A unified API for calling multiple LLM providers through a consistent, OpenAI-compatible interface
Author: Wes Jackson
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/rdwj/model-calling
Project-URL: Source, https://github.com/rdwj/model-calling
Project-URL: Issues, https://github.com/rdwj/model-calling/issues
Project-URL: Documentation, https://github.com/rdwj/model-calling/tree/main/docs
Keywords: llm,api,ollama,vllm,openai,anthropic,ai,language-model
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: httpx==0.28.1
Requires-Dist: pydantic==2.14.1
Provides-Extra: server
Requires-Dist: fastapi==0.150.0; extra == "server"
Requires-Dist: uvicorn==0.54.0; extra == "server"
Requires-Dist: python-dotenv==1.2.4; extra == "server"
Requires-Dist: PyYAML==6.0.3; extra == "server"
Provides-Extra: moderation
Requires-Dist: torch==2.14.1; extra == "moderation"
Requires-Dist: transformers==5.20.2; extra == "moderation"
Provides-Extra: examples
Requires-Dist: requests==2.34.2; extra == "examples"
Requires-Dist: aiohttp==3.14.5; extra == "examples"
Requires-Dist: numpy==2.4.6; python_version < "3.12" and extra == "examples"
Requires-Dist: numpy==2.5.4; python_version >= "3.12" and extra == "examples"
Requires-Dist: scikit-learn==1.9.1; extra == "examples"
Dynamic: license-file

# model-calling

[![PyPI version](https://img.shields.io/pypi/v/model-calling.svg)](https://pypi.org/project/model-calling/)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

An LLM client library for Python that speaks OpenAI Chat Completions and Anthropic Messages over Ollama, vLLM, OpenAI-compatible, and OpenRouter backends. Use it in-process or as a gateway.

## Install

```bash
pip install model-calling              # library only
pip install 'model-calling[server]'    # adds the gateway (FastAPI + uvicorn)
pip install 'model-calling[moderation]' # adds moderation endpoint (torch + transformers)
```

The library depends on httpx and pydantic, pinned exactly (`httpx==0.28.1`, `pydantic==2.14.1`) so every release is tested against the versions it installs; an app that pins other versions of either will conflict. Importing it never loads FastAPI and never reads `.env`.

## Status

0.2.0 is the first release of the rewritten library. It covers Chat Completions and Messages, sync and async with streaming, over Ollama, vLLM, OpenAI, OpenRouter, and Anthropic, plus the alias-routed gateway. Not yet available: the OpenAI Responses API, the Cohere backend, retries and fallbacks. Upgrading from 0.1.x involves breaking changes; see the [migration guide](https://github.com/rdwj/model-calling/blob/main/docs/migration-guide.md). Remaining work is tracked in [#8](https://github.com/rdwj/model-calling/issues/8).

## Quick start

```python
from model_calling import Client

client = Client()

# Chat Completions (OpenAI format)
response = client.completion(
    "ollama/qwen3:4b",
    [{"role": "user", "content": "What is the capital of France?"}]
)
print(response.choices[0].message.content)

# Anthropic Messages
message = client.messages(
    "vllm/gpt-oss-20b",
    [{"role": "user", "content": "What is the capital of France?"}],
    max_tokens=256
)
# Reasoning models put a thinking block first; print the text.
print(next(b.text for b in message.content if b.type == "text"))

# Embeddings
vectors = client.embedding("ollama/nomic-embed-text", ["hello", "world"])
print(vectors.data[0].embedding[:5])

client.close()
```

### Streaming

```python
from model_calling import Client

client = Client()

# Chat Completions stream
for chunk in client.completion(
    "ollama/qwen3:4b",
    [{"role": "user", "content": "Count to five."}],
    stream=True
):
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

# Messages stream
for event in client.messages(
    "vllm/gpt-oss-20b",
    [{"role": "user", "content": "Count to five."}],
    max_tokens=256,
    stream=True
):
    if event.type == "content_block_delta" and event.delta.type == "text_delta":
        print(event.delta.text, end="", flush=True)

client.close()
```

### Async

```python
from model_calling import Client
import asyncio

async def main():
    async with Client() as client:
        response = await client.acompletion(
            "ollama/qwen3:4b",
            [{"role": "user", "content": "Hi!"}]
        )
        print(response.choices[0].message.content)

asyncio.run(main())
```

Module-level functions are also available:

```python
import model_calling

response = model_calling.completion("ollama/qwen3:4b", [...])
vectors = model_calling.embedding("ollama/nomic-embed-text", ["hello"])
```

## Backends

Model routing uses `prefix/name`. Unprefixed names go to Ollama.

| Prefix | Default base_url | Credentials |
|--------|------------------|-------------|
| `anthropic` | `https://api.anthropic.com/v1` | `ANTHROPIC_API_KEY` |
| `ollama` | `http://localhost:11434` | none |
| `vllm` | `http://localhost:8000/v1` | `api_key` in config if needed |
| `openai` | `https://api.openai.com/v1` | `OPENAI_API_KEY`, optional `OPENAI_ORG_ID` |
| `openrouter` | `https://openrouter.ai/api/v1` | `OPENROUTER_API_KEY` |

The library reads only those environment variables. Override per call with `api_base=` / `api_key=`, or per client via `config`.

The `anthropic` prefix speaks Anthropic Messages to any server that serves it at `{base_url}/messages`: Anthropic, OpenRouter (`https://openrouter.ai/api/v1`), or vLLM (`http://localhost:8000/v1`). A `messages()` call reaches it unchanged. A `completion()` call is translated, and since Messages requires `max_tokens`, one sent without it gets 4096.

The Cohere backend is planned and not available yet, in the library or the gateway.

Both formats work over every backend. You can call `client.messages()` against Ollama, vLLM, OpenAI, or OpenRouter; the library translates to their Chat Completions APIs. Likewise for `client.completion()` against an Anthropic backend.

## Strictness

Strict by default: a parameter the backend cannot honor raises `UnsupportedParamsError` (a `BadRequestError` with a `.params` list) before anything is sent. Pass `drop_params=True` (client-wide or per call) to drop them instead.

Example: Ollama cannot force a tool call (`tool_choice` "required" or `{"type": "any"}`). Chat Completions backends cannot take `thinking.budget_tokens`.

Some parameters are always dropped silently: `cache_control` (a caching hint that changes no output) and `user` for Ollama embeddings.

## Gateway

The gateway is a FastAPI server running on the same library code. Apps talk to it with the official `openai` or `anthropic` SDK and name models by alias; a YAML file decides which backend serves each alias. Moving from Ollama in development to vLLM in production is an edit to that file. The app doesn't change, and the gateway picks up the edit without a restart.

```yaml
# model-calling.yaml (or the file MODEL_CALLING_CONFIG names)
backends:
  prod: {provider: vllm, base_url: https://vllm.example.com/v1, api_key_env: VLLM_API_KEY}
models:
  chat: {backend: ollama, model: qwen3:4b}   # development
  # chat: {backend: prod, model: gpt-oss-20b}   # production
  embed: {backend: ollama, model: nomic-embed-text, dimensions: 768}
```

```bash
model-calling  # or: python -m model_calling
```

Binds `0.0.0.0`, port from `PORT` (default 8000). The gateway reads `.env` (python-dotenv) at startup.

### Endpoints

- `POST /v1/chat/completions` (OpenAI Chat Completions; streaming SSE ending `data: [DONE]`)
- `POST /v1/messages` (Anthropic Messages; streaming SSE with `event:` lines; errors as Anthropic error bodies)
- `POST /v1/messages/count_tokens` (Anthropic token counting, answered with a local estimate since no backend here has a counter)
- `POST /v1/embeddings` (OpenAI Embeddings)
- `GET /v1/models` (the configured aliases), `POST /v1/moderations`, `GET /` health

Point the official SDKs at it:

```python
import openai
import anthropic

openai_client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
anthropic_client = anthropic.Anthropic(base_url="http://localhost:8000", api_key="unused")

reply = openai_client.chat.completions.create(
    model="chat", messages=[{"role": "user", "content": "Hello"}]
)
```

The gateway does not check client keys; it uses its own backend keys from its environment. Any name that isn't an alias, including a backend-prefixed one such as `ollama/qwen3:4b`, gets 404. See [Model Aliases](https://github.com/rdwj/model-calling/blob/main/docs/advanced.md#model-aliases) for the file format and reload behavior.

### Strict mode

The gateway drops unsupported parameters by default. Set `MODEL_CALLING_STRICT_PARAMS=true` (accepts true/false/1/0/yes/no) to turn on strict mode: 400 with message "model '\<alias\>' cannot honor \<param\>".

### Backend URLs

For backends the alias file names without a `base_url`, the gateway takes URLs from its environment: `OLLAMA_BASE_URL` is Ollama's native root (`http://ollama:11434`), while `VLLM_BASE_URL` and `MODEL_CALLING_ANTHROPIC_BASE_URL` are API roots with `/v1`. `VLLM_API_KEY` is sent to vLLM if set.

Environment: `MODEL_CALLING_CONFIG`, `OPENAI_API_KEY`, `OPENAI_ORG_ID`, `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, `OLLAMA_BASE_URL`, `VLLM_BASE_URL`, `VLLM_API_KEY`, `MODEL_CALLING_ANTHROPIC_BASE_URL`, `PORT`, `MODEL_CALLING_STRICT_PARAMS`. The `model-calling` command also reads a `.env` file from the directory it starts in.

## Containers

Build with podman:

```bash
./run_podman.sh         # Containerfile directly
./run_podman_compose.sh # podman-compose.yml
```

UBI 9 Python 3.12 base, runs as non-root. FIPS variant in `Containerfile-FIPS`.

## Testing

```bash
uv sync
uv run pytest                 # offline suite; no network
uv run pytest -m live         # live smoke tests via OpenRouter (OPENROUTER_API_KEY)
```

The offline suite runs the gateway in-process against recorded backend fixtures and validates output with the official `openai` and `anthropic` SDK types. See [tests/README.md](https://github.com/rdwj/model-calling/blob/main/tests/README.md) for the harness and fixture recorder.

## Documentation

The [documentation](https://github.com/rdwj/model-calling/blob/main/docs/README.md) covers the library API, the gateway, model aliases, and migrating from 0.1.x.

## License

Apache-2.0. See [LICENSE](https://github.com/rdwj/model-calling/blob/main/LICENSE) and [NOTICE](https://github.com/rdwj/model-calling/blob/main/NOTICE).

## Acknowledgements

This project was developed with assistance from AI tools. It was inspired by the need for a flexible approach to building LLM applications with locally-hosted and cloud models.
