Metadata-Version: 2.1
Name: codemindai-api
Version: 1.0.1
Summary: Run trained CodeMind code-intelligence models - from Python or as an OpenAI-compatible API server
Author: FWKTechnologies
Maintainer: FWKTechnologies
Home-page: https://github.com/FWKTechnologies/CodeMind-Code-Intelligence
Project-URL: Homepage, https://github.com/FWKTechnologies/CodeMind-Code-Intelligence
Project-URL: License, https://github.com/FWKTechnologies/CodeMind-Code-Intelligence/blob/main/LICENSE
Requires-Python: >=3.11,<3.14
License: Proprietary
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Environment :: GPU :: NVIDIA CUDA
Classifier: License :: Other/Proprietary License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Dist: torch>=2.4
Requires-Dist: safetensors>=0.4.5
Requires-Dist: psutil>=5.9
Requires-Dist: tree-sitter-language-pack>=1.0,<2
Requires-Dist: pypdf>=4.0
Provides-Extra: server
Requires-Dist: fastapi>=0.115,<1 ; extra == "server"
Requires-Dist: uvicorn[standard]>=0.30 ; extra == "server"
Requires-Dist: pydantic>=2.7,<3 ; extra == "server"
Requires-Dist: redis>=5.0 ; extra == "server"
Requires-Dist: psycopg[binary]>=3.2 ; extra == "server"
Description-Content-Type: text/markdown

# codemindai-api

Run trained **CodeMind** code-intelligence models on your own machine - from Python, or as an OpenAI-compatible API
server for websites and apps. Windows and Linux, one or many NVIDIA GPUs.

A CodeMind model folder (`Model_CodeMind/`) contains only data: weights (`safetensors`), configuration, tokenizers
and documents. `codemindai-api` is what runs it.

## Requirements

| | |
|---|---|
| OS | Windows 10/11 (64-bit) or Linux (x86-64) |
| Python | 3.11, 3.12 or 3.13 |
| PyTorch | 2.4+ with CUDA (install it first - see below) |
| GPU | NVIDIA, any number; memory per precision is in each model's README |

## Install

```bash
python -m venv .venv
# Linux:   source .venv/bin/activate
# Windows: .venv\Scripts\activate

# 1. PyTorch with CUDA (choose your system at https://pytorch.org/get-started/locally/), e.g.
pip install torch --index-url https://download.pytorch.org/whl/cu128

# 2. codemindai-api - add [server] for the HTTP API server
pip install "codemindai-api[server]"

codemindai-api info --model ./Model_CodeMind      # model, runtime, PyTorch, GPUs
```

## Use it from Python

```python
import codemindai_api

model = codemindai_api.load("./Model_CodeMind")       # first GPU (CPU if none); FP8 where supported

print(model.chat("What is the difference between a process and a thread?"))

ans = model.code("A thread-safe LRU cache", language="python")
print(ans.text, ans.confidence)

print(model.validate("SELECT * FROM users WHERE id = ' + user_id", language="sql")["issues"])

with codemindai_api.load("./Model_CodeMind", precision="bf16", device="cuda:1") as m2:   # one chosen GPU
    for piece in m2.stream("Explain this regex: ^\\d{3}-\\d{4}$"):
        print(piece, end="")

big = codemindai_api.load("./Model_CodeMind", precision="fp8", devices="all")   # one copy on EVERY GPU
answers = big.chat_many([f"Explain error code E{i}" for i in range(100)])  # spread over all of them
```

`load(path, precision="auto"|"fp32"|"bf16"|"fp16"|"fp8"|"fp4", device="auto"|"cuda:N"|"cpu", devices="all"|[...],
split=False, fp8="auto", safety=True, before=None, after=None, verify=False, threads=0, warmup=True)`

`Model.chat(prompt | messages=[...], system="", mode="auto"|"text"|"code", language=None, temperature=0.8, top_p=0.92,
max_tokens=0)` → `Answer(text, mode, uncertain, confidence, reason, questions, language, language_supported, score, raw)`

Also `code()`, `chat_many()`, `stream()`, `validate()`, `ground()` (exact math/physics with units), `count_tokens()`,
`add_hook()`, `info`, `runtime`, `close()`.

### Precision

| `precision` | weights | use it for |
|---|---|---|
| `fp32` | 4 bytes | exact reference |
| `bf16` | 2 bytes | same quality, half the memory (Ampere and newer) |
| `fp16` | 2 bytes | GPUs without BF16 (Turing, Volta) |
| `fp8` | 1 byte | FP8 tensor cores on RTX 40xx / Ada, Hopper, Blackwell (elsewhere FP8 storage, BF16 compute) |
| `fp4` | ~5 bits | the smallest: 4-bit E2M1 values, one scale per 16 |
| `auto` | | FP8 when the GPU passes a numeric self-test, else BF16; FP32 on CPU |

Every choice is checked on the real GPU and falls back safely (FP4 → BF16, FP8 → BF16) instead of failing.

### Your own code around every answer

```python
def before(ctx):                       # system, prompt, mode, language, temperature, top_p, max_tokens
    if "password" in ctx["prompt"].lower():
        raise ValueError("not on this service")       # refuses the request
    return ctx

model = codemindai_api.load("./Model_CodeMind", before=before, after=lambda ctx, a: a.text.strip())
```

The server takes the same idea from a file: `codemindai-api serve --hooks my_hooks.py` with `before_request(ctx)` and
`after_response(ctx, result)`.

## Run the API server

```bash
mkdir my-server && cd my-server
codemindai-api init                      # .env with an admin token + Model_CodeMind/
#   put the model files into Model_CodeMind/
codemindai-api serve --supervise --host 0.0.0.0   # every GPU, auto-restart (default host 127.0.0.1 = local only)
codemindai-api create-key --owner my-app
```

OpenAI-compatible: `POST /v1/chat/completions` (streaming), `/v1/completions`, `/v1/code/validate`, `/v1/ground`,
`/v1/models`, `/v1/me`, `/v1/usage`, `/health`, `/ready`, plus `/admin/*` (keys, limits, usage, audit, workers,
zero-downtime model reload) and `/metrics`. Any OpenAI SDK works:

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="YOUR_KEY")
print(client.chat.completions.create(model="codemind-3.0",
      messages=[{"role": "user", "content": "Hello"}]).choices[0].message.content)
```

The full reference ships with every model: `Model_CodeMind/API_REFERENCE.md`.

## Hardware use

- **Any number of GPUs:** the server starts model workers on every GPU (`--gpus auto|0,1,2,...`), sizes how many copies
  fit per GPU from real memory use (`--workers-per-gpu`), scales up under load and down when idle, and spreads requests
  over all of them. In Python: `devices="all"` + `chat_many()`.
- **Precision:** `--precision` / `precision=` FP32, BF16, FP16, FP8, FP4 - self-tested on each GPU, with fallback.
- **One big 3.x model over GPU pairs:** `codemindai_api.load(path, devices="all", split=True)`.
- **Limits are yours:** queue, connections, output length, requests/minute, parallel requests per key - each can be
  set, or `0` for unlimited (`codemindai-api serve --help`).
- **CPU:** works without a GPU (slow - for testing).

## Security

- Model folders are data only (`safetensors`, no pickle): loading a model never runs code from it.
- Every load checks the model files; `codemindai-api verify` checks each file's SHA-256 against `manifest.json`.
- No internet needed to answer. The code parser downloads each language grammar once and caches it - run
  `codemindai-api prefetch` once before taking a machine offline (`serve` does it at start).
- Server: API keys stored only as HMAC-SHA256 hashes; admin token for `/admin`; per-key rate, token, concurrency,
  IP and website limits; request size limits; content screening of requests and answers; secret redaction; audit log.
- The server listens on this machine only (127.0.0.1) unless you pass `--host 0.0.0.0`.
- Keep `.env` private; use HTTPS in front of a public server.

## Commands

| command | does |
|---|---|
| `codemindai-api init` | prepare the current folder for serving |
| `codemindai-api serve` | run the API server |
| `codemindai-api chat "..."` | one answer from the command line |
| `codemindai-api info` / `verify` / `doctor` / `prefetch` | inspect, check files, full system check, download grammars |
| `codemindai-api create-key` / `list-keys` / `revoke-key` / `gen-token` | API keys and tokens |

## Troubleshooting

| message | fix |
|---|---|
| `PyTorch sees no CUDA GPU` | install the CUDA build of PyTorch (step 1), update the NVIDIA driver |
| `this process already runs the v2 runtime` | 2.x and 3.x models need separate processes |
| `model folder is incomplete` | copy the whole model folder again; `codemindai-api verify` names the bad file |
| out of GPU memory | `fp8="auto"` (default), a smaller `--workers-per-gpu`, or `devices=[...]` for 3.x |

License: proprietary - see the terms that came with the model.
