Metadata-Version: 2.5
Name: cembedding
Version: 0.8.0
Summary: Local-first embedding server: vector generation + index/search over HTTP (ONNX on-device or API providers). The reference /embed server for CPersona.
Project-URL: Homepage, https://github.com/Cloto-dev/CEmbedding
Project-URL: Repository, https://github.com/Cloto-dev/CEmbedding
Project-URL: Issues, https://github.com/Cloto-dev/CEmbedding/issues
Project-URL: Changelog, https://github.com/Cloto-dev/CEmbedding/releases
Project-URL: Documentation, https://github.com/Cloto-dev/CEmbedding#readme
Author-email: ClotoCore Project <ClotoCore@proton.me>
License: MIT
License-File: LICENSE
Keywords: bge-m3,embedding,jina,mcp,onnx,vector-search
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Requires-Dist: aiohttp>=3.9.0
Requires-Dist: aiosqlite>=0.20.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: mcp<1.27.0,>=1.0.0
Requires-Dist: numpy>=1.24.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Requires-Dist: tokenizers>=0.15.0; extra == 'dev'
Provides-Extra: mlx
Requires-Dist: mlx-embeddings>=0.1.0; extra == 'mlx'
Requires-Dist: mlx>=0.18.0; extra == 'mlx'
Provides-Extra: onnx
Requires-Dist: onnxruntime>=1.17.0; extra == 'onnx'
Requires-Dist: tokenizers>=0.15.0; extra == 'onnx'
Provides-Extra: onnx-gpu
Requires-Dist: onnxruntime-gpu>=1.17.0; extra == 'onnx-gpu'
Requires-Dist: tokenizers>=0.15.0; extra == 'onnx-gpu'
Description-Content-Type: text/markdown

<div align="center">

# CEmbedding

### Local-first embedding server

Vector embeddings over a tiny HTTP contract.
On-device ONNX or any OpenAI-compatible API. The reference `/embed` server for [CPersona](https://github.com/Cloto-dev/CPersona).

[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/Cloto-dev/CEmbedding/blob/main/LICENSE)
[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)]()

</div>

---

> **Standalone repository** — extracted from the (now private) `clotohub-servers` monorepo so it can be used on its own. [ClotoCore](https://github.com/Cloto-dev/ClotoCore) users get this through the in-app marketplace ([ClotoHub](https://hub.cloto.dev)); everyone else can run it directly as described below.

## What it is

A small server that turns text into vectors. It speaks a minimal HTTP contract so anything can call it — its primary consumer is [CPersona](https://github.com/Cloto-dev/CPersona), whose hybrid search uses it for the vector-similarity layer. It can run a model **on-device** via ONNX (no API key, no network) or proxy an **OpenAI-compatible API**.

It also exposes an MCP (stdio) surface and an optional persistent vector index (`/index`, `/search`), but the HTTP `/embed` endpoint is all CPersona needs.

## The `/embed` contract

```
POST /embed
Request:  { "texts": ["string", ...] }                 # non-empty array, max 100 per batch
Response: { "embeddings": [[float, ...], ...], "dimensions": <int> }
```

### Where the vector stops seeing a text

A model embeds only the first `window` tokens of a text; the rest is stored by the
caller but invisible to vector search. How many characters that is depends on the
text — for one window it varies by more than a factor of two — so the server reports
it instead of leaving each client to guess.

```
POST /embed          { "texts": [...], "token_info": true }   # adds "token_info" to the response above
POST /count_tokens   { "texts": [...] }                       # the same report, without running the model
Response entry:      { "count": <int>, "window": <int>, "truncated": <bool>, "window_end_char": <int> }
```

- `count` and `window` both include the tokenizer's special tokens.
- `window_end_char` is a character offset: `text[:window_end_char]` is what the vector
  represents, and it equals `len(text)` when nothing was cut.
- `token_info` is `null` when the provider cannot see its own tokens (`api_openai`,
  `mlx_bge_m3`). Read that as unknown, never as "fits".
- Without `"token_info": true`, `/embed` answers exactly as it did before the field existed.
- The counting tokenizer loads on the first request that needs it (about 29 MiB resident
  for jina-v5-nano), so a server that is never asked pays nothing.

Point any client (e.g. CPersona's `CPERSONA_EMBEDDING_URL` / generic `EMBEDDING_HTTP_URL`) at `http://127.0.0.1:8401/embed`.

## Quick Start (on-device ONNX)

**Prerequisites:** Python 3.10+

```bash
# Download a model into ./data/models (jina-v5-nano is what CPersona is tuned for)
uvx --from "cembedding[onnx]" cembedding-download-model --model jina-v5-nano

# Run the server (reads ./data/models from the current directory)
EMBEDDING_PROVIDER=onnx_jina_v5_nano uvx --from "cembedding[onnx]" cembedding
```

Or install it onto your PATH with `pip install "cembedding[onnx]"`, then run
`cembedding-download-model --model jina-v5-nano` and `cembedding`.

From source (development):

```bash
git clone https://github.com/Cloto-dev/CEmbedding.git
cd CEmbedding
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install ".[onnx]"
python -m cembedding.download_model --model jina-v5-nano
EMBEDDING_PROVIDER=onnx_jina_v5_nano python -m cembedding   # or: python server.py
```

You should see `HTTP embedding endpoint started on http://127.0.0.1:8401/embed`. Verify it:

```bash
curl -s http://127.0.0.1:8401/embed \
  -H 'content-type: application/json' \
  -d '{"texts":["hello world"]}' | head -c 200
```

## Run it in a container

No image is published: build it from this repository, at the revision you mean to run.

```bash
docker build -t cembedding .

# Fill the volume once. Left to itself the server downloads the weights on the
# first request instead -- with the port already accepting connections it cannot
# yet answer, and no progress visible to whoever is waiting on it.
docker run --rm -v cembedding-data:/data cembedding \
    cembedding-download-model --model jina-v5-nano

docker run -d --name cembedding -p 8401:8401 -v cembedding-data:/data \
    -e EMBEDDING_PROVIDER=onnx_jina_v5_nano \
    -e CEMBEDDING_AUTH_TOKEN="$(openssl rand -hex 32)" \
    cembedding
```

The image sets `EMBEDDING_HTTP_HOST=0.0.0.0`, because inside a container the
server's default binds the container's own loopback: a published port then
forwards to a socket nothing is listening on, and the connection is refused in a
way that reads like a crash. That is a reachability decision and not a security
one — see [Authentication](#authentication-v062), and set a token whenever the
port is published.

`ONNX_MODEL_DIR` is `/data/model` in the image, so the download command above and
the server look in the same place. One model per volume: the variable names a
directory, not a collection.

The model and the index live on the `/data` volume, which is what survives the
container. A bind-mounted host directory has to be writable by uid 10001 (the
image's user), or the run needs `--user "$(id -u)"`.

## Providers

Set `EMBEDDING_PROVIDER`:

| Value | Model | Notes |
|-------|-------|-------|
| `onnx_jina_v5_nano` | jina-embeddings-v5-text-nano-retrieval (~212M params in the fp32 ONNX graph, 768d) | Local CPU, what CPersona is benchmarked against |
| `onnx_bge_m3` | bge-m3 | Local CPU, larger / multilingual |
| `onnx_miniml` | all-MiniLM-L6-v2 (22M, 384d) | Local CPU, smallest |
| `mlx_bge_m3` | bge-m3 (MLX) | Apple Silicon only — `pip install ".[mlx]"` |
| `auto_bge_m3` | bge-m3 | Auto-selects MLX on Apple Silicon, ONNX elsewhere |
| `api_openai` | provider's model | OpenAI-compatible API; needs `EMBEDDING_API_KEY` (+ optional `EMBEDDING_API_URL`, `EMBEDDING_MODEL`) |

Download a local model with `cembedding-download-model --model {miniml,jina-v5-nano,bge-m3}` (or `python -m cembedding.download_model ...` from a source checkout; fetched from HuggingFace into `./data/models`, not committed to this repo).

## Model precision

The jina-v5-nano repository ships the same graph in several precisions. `EMBEDDING_MODEL_VARIANT` selects one; `cembedding-download-model --model jina-v5-nano --variant <v>` fetches it ahead of time. Measured on CPU execution providers with a mixed English/Japanese corpus (55 texts, 22 queries), against the fp32 vectors as reference:

| variant | download | resident (laptop) | single query (laptop / 4-core x86) | cosine to fp32, median / worst | top-10 agreement |
|---|---:|---:|---:|---|---:|
| `fp32` (default) | 810 MB | 906 MB | 5.8 ms / 71 ms | — | — |
| `fp16` | 405 MB | 933 MB | 6.3 ms / — | 1.0000 / 1.0000 | 1.00 |
| `int8` | 236 MB | 628 MB | 46 ms / 319 ms | 0.9998 / 0.9981 | 0.97 |

- `fp16` produces the same vectors as `fp32`. It halves the download and nothing else: CPU execution providers widen the weights back to fp32 at load.
- `int8` cuts resident memory by about a third and keeps retrieval quality (top-10 agreement 0.97 against fp32, 0.98 when int8 queries run against an fp32-indexed corpus), but single-query latency is 4-8x worse on every CPU measured, because activations are quantized at run time. Choose it when memory is the constraint and latency is not.
- The 4-bit variants in the same repository are not offered: on this corpus their worst-case cosine to fp32 was 0.30.

Vectors already indexed with one precision stay usable with another (the mixed-precision agreement above), so switching does not require re-indexing, though re-indexing removes the residual difference.

## Configuration

| Env var | Default | Description |
|---------|---------|-------------|
| `EMBEDDING_PROVIDER` | `api_openai` | Provider (see table above) |
| `EMBEDDING_HTTP_PORT` | `8401` | HTTP port for `/embed` |
| `EMBEDDING_HTTP_HOST` | `127.0.0.1` | Address `/embed` binds to. Loopback is right for a single host; a container has to bind an address its peers can reach (see [Run it in a container](#run-it-in-a-container)). Moving it decides nothing about who may call — a token does |
| `EMBEDDING_INDEX_ENABLED` | `true` | Enable the persistent vector index endpoints (`/index`, `/search`, `/remove`, `/purge`) |
| `EMBEDDING_INDEX_DB_PATH` | `data/embedding_index.db` | SQLite file backing the vector index |
| `EMBEDDING_SEARCH_BACKEND` | `numpy` | `/search` matmul backend. `numpy` (Accelerate BLAS) or `mlx` (Apple-GPU resident matrix; falls back to numpy when mlx is absent) |
| `EMBEDDING_SIDECAR` | `auto` | Startup source for the resident vectors: `auto` uses the sidecar file (see below), `off` always reads them from SQLite |
| `EMBEDDING_SIDECAR_PATH` | `<index db>.sidecar` | Where the sidecar file lives |
| `EMBEDDING_SIDECAR_MIN_ROWS` | `10000` | Smallest corpus that gets a sidecar; below it the in-memory index is cheap enough that a file adds nothing |
| `EMBEDDING_SIDECAR_MAX_TAIL` | `0.25` | Rebuild when the rows written since the last build exceed this fraction of the rows in the file |
| `ONNX_MODEL_DIR` | (auto) | Override the model directory for ONNX providers |
| `ONNX_EP_PREFERENCE` | (auto) | ONNX execution providers, comma-separated. Empty = auto (CoreML on macOS, DirectML on Windows, else CPU; CPU always ensured) |
| `ONNX_MAX_SEQ_LEN` | `2048` | Max tokenization length (1–8192; MiniLM clamped to 512 internally) |
| `EMBEDDING_MODEL_VARIANT` | `fp32` | Precision of jina-v5-nano to load: `fp32` / `fp16` / `int8` (downloaded on first use). See [Model precision](#model-precision) before changing it |
| `ONNX_INTRA_OP_THREADS` | `0` | ONNX Runtime intra-op threads. `0` = runtime default (physical cores). Set it when the process runs under a CPU quota the runtime cannot see (container limit, shared host) |
| `ONNX_GRAPH_OPT_LEVEL` | `all` | ONNX Runtime graph optimization: `disable` / `basic` / `extended` / `all`. Lower it only to compare against an un-fused graph |
| `EMBEDDING_MAX_BATCH` | `64` | Most texts one model run may carry when concurrent requests are merged into it. Local providers run one pass at a time, so requests that arrive while a pass is in flight share the next one instead of each paying for a pass (nothing waits for a batch to fill, and one request is never split). Results are bit-identical on the CPU provider; on accelerator providers the low bits (about 1e-6) can depend on the batch a text ran in, as they already did for multi-text requests. `0` disables merging |
| `EMBEDDING_API_KEY` | — | Required for `api_openai` |
| `EMBEDDING_API_URL` | `https://api.openai.com/v1/embeddings` | API endpoint for `api_openai` |
| `CEMBEDDING_AUTH_TOKEN` | — | Inbound bearer token. Unset = no authentication (see below) |
| `CEMBEDDING_REQUIRE_AUTH` | `false` | Refuse to start when no token is configured |

## Startup: the sidecar file

Loading the index from SQLite copies every stored blob into a matrix, which
costs time and memory proportional to the corpus at every start. With
`EMBEDDING_SIDECAR=auto` (the default) the same rows are also kept in a file
laid out the way the search matrix already is, and a start maps it instead of
decoding the corpus. At 100,000 vectors of 768 dimensions on an Apple laptop
that is 0.09s to the first search instead of 0.22s, at 403 MiB of peak resident
memory instead of 1.1 GiB.

SQLite stays the durable record and decides every disagreement. The file is
written at startup once the corpus reaches `EMBEDDING_SIDECAR_MIN_ROWS`, and
again when the rows written since the last build outgrow
`EMBEDDING_SIDECAR_MAX_TAIL`; rows written after a build are read from SQLite
and are never invisible. Deleting the file costs the next start a full read and
nothing else, and a file that is truncated, foreign or unreadable is ignored
with the reason logged.

```bash
cembedding-sidecar status --db data/embedding_index.db   # present? usable? how stale?
cembedding-sidecar build  --db data/embedding_index.db   # write one now
```

## Authentication (v0.6.2)

Both HTTP surfaces — the REST endpoints (`/embed`, `/count_tokens`, `/index`, `/search`,
`/remove`, `/purge`) and the Streamable HTTP MCP transport — accept an inbound
bearer token:

```bash
CEMBEDDING_AUTH_TOKEN=$(openssl rand -hex 32)
```

With the token set, every request must carry `Authorization: Bearer <token>`;
a missing header, a wrong scheme and a wrong token are all rejected with `401`.
Comparison is constant-time. With the token unset, requests are served exactly
as in earlier versions and a warning is logged — set `CEMBEDDING_REQUIRE_AUTH=true`
to turn that warning into a startup error instead. Requiring a token is opt-in
in this release so existing deployments keep working; a later release will make
it the default.

**Do not treat the bind address as the security boundary.** The REST surface
binds loopback and the MCP transport binds `0.0.0.0` by default, but a tunnel or
reverse proxy forwards to loopback all the same, so a loopback bind is no
evidence that requests are local. If the process is reachable through a tunnel,
a proxy, or any non-loopback interface, configure a token.

## Use with CPersona

Run this server, then tell CPersona to use it:

```bash
# CPersona MCP config env
CPERSONA_EMBEDDING_MODE=http
CPERSONA_EMBEDDING_URL=http://127.0.0.1:8401/embed
```

Without an embedding server CPersona still works (FTS5 + keyword search); adding one enables the vector-similarity layer.

To serve CPersona's *remote vector search* (`CPERSONA_VECTOR_SEARCH_MODE=remote`),
this server's `/index` + `/search` endpoints hold the vectors. v0.6.0 searches a
per-namespace resident matrix (one matmul per query, ~21x faster than v0.5.0 at
237k x 384: 131 ms -> 6 ms/query), so a full-corpus semantic recall stays fast at
memory-corpus scale. When flipping an existing CPersona deployment to remote mode,
first migrate its already-stored vectors:

```bash
python scripts/backfill_embedding_index.py \
    --cpersona-db ~/.claude/cpersona.db \
    --index-db data/embedding_index.db \
    --expect-dim 768   # your embedding model's dimension
```

then restart this server so it reloads the index. Skipping the backfill silently
drops every pre-flip memory from semantic recall (the remote branch only falls
back to local search on HTTP errors, not on empty results).

## Found a bug, or something the docs do not explain?

Open an issue — [bug report](https://github.com/Cloto-dev/CEmbedding/issues/new?template=bug_report.yml)
or [feature request](https://github.com/Cloto-dev/CEmbedding/issues/new?template=feature_request.yml).

Reports are welcome even when you are not certain it is a bug. If it turns out
to be a configuration problem, that is still useful signal — it means the
documentation was unclear, which is a defect of its own. Security
vulnerabilities are the one exception: please report those privately through
[GitHub Security Advisories](https://github.com/Cloto-dev/CEmbedding/security/advisories/new)
rather than in a public issue.

## License

MIT — see [LICENSE](https://github.com/Cloto-dev/CEmbedding/blob/main/LICENSE).
