Metadata-Version: 2.4
Name: dbx-tools-model-proxy
Version: 0.9.32
Summary: LiteLLM proxy backed by Node-owned Databricks authentication and dynamic model discovery
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Dist: filelock>=3.16,<4
Requires-Dist: httpx>=0.28,<1
Requires-Dist: pythonmonkey>=1.3,<2
Requires-Dist: fastapi>=0.116,<1
Requires-Dist: hypercorn>=0.17,<1
Requires-Dist: litellm[proxy]==1.99.0
Requires-Dist: pillow>=11,<13
Requires-Dist: pystray==0.19.5
Requires-Python: >=3.11, <4
Project-URL: Source, https://github.com/reggie-db/dbx-tools/tree/main/packages/py/model-proxy
Description-Content-Type: text/markdown

# `dbx-tools-model-proxy`

OpenAI-compatible LiteLLM proxy backed by Node-owned Databricks authentication,
model discovery, ranking, routing, and metadata.

LiteLLM owns request conversion, streaming, transport, and retry behavior. The
Python package embeds the generated `@dbx-tools/model/python` client under its
private `_generated` submodule, so authentication and model policy remain
implemented once in Node.

## Run

```sh
uv run dbx-model-proxy --port 4000
```

The proxy listens on `127.0.0.1` unless a different host is forwarded to
LiteLLM. Select a Databricks profile explicitly when needed:

```sh
uv run dbx-model-proxy --profile my-workspace --port 4000
```

The host exposes LiteLLM's OpenAI-compatible APIs, a dynamic `GET /v1/models`
catalogue, and `GET /lookup` for Node-ranked fuzzy model search. Model requests
are resolved lazily and receive current Databricks authentication headers from
the generated client.

The standard model response keeps the OpenAI `data` array. Requests whose
`originator` header starts with `codex` also receive a Codex `models` array with
Node-ranked priorities, Databricks AI Gateway model names, reasoning levels,
and build-cached capability metadata. Embedding, retired, and unsupported Codex
families remain available through the OpenAI catalogue but are omitted from the
Codex extension.

## Service

Install and start the proxy as a per-user background service:

```sh
dbx-model-proxy service install -- --profile my-workspace --port 4000
```

The lifecycle includes `start`, `stop`, `restart`, `status`, and `uninstall`,
with `remove` as an uninstall alias. Configuration and logs default to
`~/.dbx-tools/model-proxy`. Uninstall retains them unless `--purge` is supplied.

Service installation uses the current package's exact Python environment and
registers a launchd agent on macOS, a systemd user unit on Linux, or current-user
scheduled tasks on Windows. `--systray auto` installs the tray when its native
backend is available. Use `always` to require it or `never` for a headless
service. The tray uses the dbx-tools icon, opens the local models and API pages,
switches among configured Databricks profiles, and can stop the service.

The proxy writes service output to `service.log` and tray output to `tray.log`.
Lifecycle operations fail inside Databricks Apps because host service management
is unavailable there.

For an isolated test installation, add `--concurrent`:

```sh
dbx-model-proxy service install --concurrent -- --profile my-workspace
```

Concurrent mode defaults the proxy to port `4001` and uses the independent
`model-proxy-python` configuration directory, service registration, logs, and
tray identity. An explicit `--port` after `--` overrides the concurrent default.

## Ownership

- The embedded `@dbx-tools/model/python` client owns authentication, profile
  selection, endpoint discovery, caching, fuzzy matching, protocol selection,
  URLs, and published metadata.
- LiteLLM owns Chat, Responses, embeddings, streaming, parameter conversion,
  provider transport, and retries.
- This package owns the generated Python boundary, FastAPI route installation,
  LiteLLM callback wiring, and command-line startup.

Add proxy-specific controls only where LiteLLM does not already provide
equivalent behavior.
