Metadata-Version: 2.4
Name: servellm
Version: 1.0.0
Summary: Expose a local LLM runtime as a reliable HTTP/WebSocket service.
Author-email: adilatif24 <adilatif24@gmail.com>
License-Expression: MIT
Keywords: llm,ollama,vllm,llama.cpp,openai-api,inference-server
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi
Requires-Dist: uvicorn[standard]
Requires-Dist: httpx
Requires-Dist: pyyaml
Requires-Dist: click
Requires-Dist: psutil
Requires-Dist: prometheus-client
Dynamic: license-file

# ServeLLM

Expose a local LLM runtime (Ollama, vLLM, or llama.cpp) as a reliable, OpenAI-compatible HTTP/WebSocket API.

## Install

```bash
pip install servellm
```

## Quickstart

```bash
servellm init      # writes servellm.yaml with a generated API key
servellm start      # starts the server in the background
servellm models      # list available models
```

## Usage

OpenAI-compatible endpoint, so existing OpenAI clients work by changing `base_url`:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"model": "fast", "messages": [{"role": "user", "content": "hello"}]}'
```

Also available: `GET /v1/models`, `GET /health`, `GET /metrics`, `WS /ws/chat`, and the simpler
`POST /chat` endpoint.

## Configuration

Edit `servellm.yaml` (created by `servellm init`) to point at your runtime, set API keys,
concurrency limits, and model aliases. No Python code required. See the project repository
for the full configuration reference.

## CLI

```
servellm init      create servellm.yaml in the current directory
servellm start      start the server
servellm stop      gracefully stop the server
servellm status      check whether the server is running and healthy
servellm models      list configured/discovered models
```

## License

MIT
