Metadata-Version: 2.5
Name: deqio
Version: 0.6.5
Summary: Unified runtime for fast typed AI decisions across multiple decision engines
Project-URL: Repository, https://github.com/ILuce/deqio
Project-URL: Issues, https://github.com/ILuce/deqio/issues
License-Expression: MIT
License-File: LICENSE
Keywords: ai,cuda,decision,gguf,inference,jev,llm,mlx,system-one
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: deqio-client<0.7,>=0.6.5
Requires-Dist: fastapi<1,>=0.110
Requires-Dist: huggingface-hub<2,>=1
Requires-Dist: keyring<26,>=25
Requires-Dist: pydantic<3,>=2
Requires-Dist: uvicorn[standard]<1,>=0.27
Provides-Extra: dev
Requires-Dist: httpx<1,>=0.27; extra == 'dev'
Requires-Dist: pytest<10,>=8; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16; extra == 'dev'
Requires-Dist: tomli<3,>=2; (python_version < '3.11') and extra == 'dev'
Description-Content-Type: text/markdown

# Deqio

**Decision in, probabilities out.**

## What is Deqio

Deqio is a local server for *decision models* — small language models trained to answer a typed question about a
piece of text with a constrained, probabilistic answer instead of free-form prose. You send a **state** (text or
JSON) and a question; you get back a yes/no, exactly one option, an ordered score, a set of labels or a cost-aware
action — always with the probability of every allowed answer.

A generative model might answer:

> "I think the refund should probably be approved."

A decision model answers:

```json
{"choice": "approve", "probabilities": {"approve": 0.91, "review": 0.07, "reject": 0.02}}
```

That makes Deqio useful wherever the allowed answers are known in advance and a large LLM would be slow, expensive or
unpredictable: routing and triage, policy checks and guardrails, agent checkpoints (next step, stop or continue, ask
a human), scoring and classification. Deqio keeps the differences between model families and runtimes behind one
HTTP API, records provenance for every answer, and ships a browser UI, a live request view, a benchmark runner and a
Python client. Everything runs on your machine; the two remote decision APIs are optional catalog entries.

## Install with uv

Install [uv](https://docs.astral.sh/uv/) if you do not have it, then Deqio:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install deqio
```

Pick a workspace directory, install a model and start the server:

```bash
mkdir -p deqio-work && cd deqio-work
deqio models setup          # backend → model → token budget → download, install and probe
deqio serve                 # http://127.0.0.1:8787
```

Then open the dashboard at `http://127.0.0.1:8787/ui`, the live request view at `/ui/watch` and the API
documentation at `/docs`. Deqio binds to `127.0.0.1` and keeps its state in the workspace (`.deqio/`,
`.model-runtimes/`, `models/`, `config.json`).

Supported platforms: macOS on Apple Silicon (MLX and GGUF backends) and Linux (CUDA with an NVIDIA GPU, GGUF).
Python 3.10 and 3.12 are tested in CI. Windows is not supported.

## Models

Every model profile in the Deqio 0.6.5 catalog (31 models). A check mark means Deqio has a profile for that
model/backend combination; availability on your machine still depends on the operating system, accelerator and
memory. `deqio models list` prints the catalog for your host, `deqio models installed` what is ready to use.

| Model | Model ID | MLX | CUDA | GGUF | Remote |
| --- | --- | :---: | :---: | :---: | :---: |
| SemIf / Qwen3.5 4B | `semif-qwen3.5-4b` | ✓ 8-bit | ✓ | — | — |
| Decision 3.0 d3 26B | `d3` | — | ✓ | ✓ Q8_0 | — |
| Decision 3.0 d3-flash 8B | `d3-flash` | — | ✓ | ✓ Q8_0 | — |
| Decision 3.0 d3-mini 4.5B | `d3-mini` | — | ✓ | ✓ Q8_0 | — |
| Decision 3.0 d3-nano 2.2B | `d3-nano` | — | ✓ | ✓ Q8_0 | — |
| Decision 3.0 d3-lite 0.85B | `d3-lite` | — | ✓ | ✓ Q8_0 | — |
| Kev 0.8B | `kev-0.8b` | ✓ | ✓ | ✓ Q8_0 | — |
| Kev 4B | `kev-4b` | ✓ | ✓ | ✓ Q8_0 | — |
| Kev 9B | `kev-9b` | ✓ | ✓ | ✓ Q8_0 | — |
| Kev 27B | `kev-27b` | — | ✓ | — | — |
| JevK5 4B | `jevk5-4b` | — | ✓ | ✓ Q8_0 | — |
| JevK5 9B | `jevk5-9b` | — | ✓ | ✓ Q8_0 | — |
| Open-Jev 2B | `open-jev-2b` | — | ✓ | — | — |
| Open-Jev 9B | `open-jev-9b` | — | ✓ | — | — |
| Open-Jev 27B v1.1 | `open-jev-27b-v1.1` | — | ✓ | — | — |
| CLM 8B | `clm-8b` | — | ✓ | — | — |
| Basal 2.0 Mini — 1.5B | `basal-2.0-mini` | ✓ 8-bit | — | ✓ Q8_0 | — |
| Basal 2.0 — 4.5B | `basal-2.0-main` | ✓ 8-bit | — | ✓ Q8_0 | — |
| Basal 2.0 Max — 11B | `basal-2.0-max` | ✓ 8-bit | — | ✓ Q8_0 | — |
| Clef Flash 9B | `clef-flash` | ✓ 8-bit | — | ✓ Q8_0 | — |
| Clef 27B | `clef` | ✓ 8-bit | — | ✓ Q8_0 | — |
| Decider 0.8B | `decider-0.8b` | — | ✓ | — | — |
| Decider 2B | `decider-2b` | — | ✓ | ✓ Q8_0 | — |
| Decider 4B | `decider-4b` | — | ✓ | ✓ Q8_0 | — |
| Laya English 421M | `laya-english` | ✓ | ✓ | ✓ Q8_0 | — |
| Laya Multilingual 322M | `laya-multilingual` | ✓ | ✓ | — | — |
| Laya Typed Decisions 421M | `laya-typed-decisions` | ✓ | ✓ | — | — |
| Von | `von` | — | ✓ | — | — |
| Bespoke Nimble 9B | `nimble-9b` | ✓ | ✓ | — | — |
| TypeSafe Jev (remote) | `typesafe-jev` | — | — | — | ✓ |
| OpenAI Decisions API (remote) | `openai-decisions` | — | — | — | ✓ |

- `✓ 8-bit` is the author's MLX 8-bit checkpoint, `✓ Q8_0` the official GGUF Q8_0 file through the pinned llama.cpp
  or the engine's own GGUF sidecar, `✓` under `Remote` a remote decision API (no weights; the API key lives in the
  OS keychain).
- Every engine runs in its own isolated environment with every runtime package and model revision pinned; every
  profile is labelled `official` or `community`.
- Capabilities differ by model: Choice, Noul and Score are native everywhere; Multi is native on Basal 2.0 and
  emulated on every other profile (`multi_emulated` — one yes/no branch per label in one engine request); Act, Facts
  and option keys are Basal 2.0 features. Unsupported features are refused explicitly, never silently ignored.
- Backends and model families in detail, remote API setup, token budgets and the `deqio models` commands:
  [docs/MODELS.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/MODELS.md).

## Benchmarks and results

Deqio ships two editable benchmark suites — **ENG Bench** (`deqio-basic-220`: 55 Noul, 55 Choice, 55 Shared and
55 Multi cases) and **PL Bench** (`deqio-pl-120`: 30 of each, Polish-native, not a translation) — covering customer
service, retail, security triage, AI-agent checkpoints, office administration, politics and world news. Run them
against your installed models with:

```bash
deqio benchmark                     # choose a suite and the profiles interactively
deqio benchmark --all               # every installed profile; results under .deqio/benchmarks/<timestamp>
deqio benchmark compare             # two completed runs side by side
```

How to read the tables: `Accuracy` is the share of whole cases answered exactly right; `Decision accuracy` scores
every decision — each Shared decision and each Multi label counts once. A Multi case passes only when all of its 6–8
labels are right, so its case accuracy is roughly the label accuracy to the 7th power; compare models on **Multi
label accuracy**. Latencies are per request on one Apple Silicon laptop with one model resident at a time; smaller
groups ran separately from larger ones. The two runtime errors in the above-3B ENG run are two SemIf cases refused
by its upstream shared-state prefix check, reported as errors, not worked around. Everything else about the suites,
the scoring and the tables is in [docs/BENCHMARKS.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/BENCHMARKS.md); the complete reports with the previous
snapshots are [benchmarks/summary_eng.md](https://github.com/ILuce/deqio/blob/v0.6.5/benchmarks/summary_eng.md) and
[benchmarks/summary_pl.md](https://github.com/ILuce/deqio/blob/v0.6.5/benchmarks/summary_pl.md).

### 0.6.4 snapshot

Three groups measured on PL Bench and ENG Bench with Deqio 0.6.4 on 2026-10-18 (0.6.5 is the same code, published
after the interrupted 0.6.4 upload): models up to 1B (`d3-lite` GGUF,
Kev 0.8B, Laya English), models from 1B to 3B (`d3-nano` GGUF, Decider 2B, Basal 2.0 Mini) and models above 3B
(`d3-mini` GGUF, Decider 4B, Basal 2.0 on GGUF and on MLX, JevK5 4B, Kev 4B, SemIf). The tables are rendered by
`deqio benchmark tables` from the committed `runs/<group>-<lang>/summary.json` files; the remote decision APIs were
last measured on the v1 PL suite (see the PL report).

### Models up to 1B — PL Bench (0.6.4 snapshot)

#### Run

- `20261018T080338Z` · suite `deqio-pl-120` — 120 cases / 344 scored decisions · Deqio 0.6.4 · 2026-10-18
- runtime errors across the listed profiles: 0

#### Quality

`Accuracy` is whole-case accuracy; `Decision accuracy` scores every decision/assertion (a Shared case holds several; a Multi case scores every label as present or absent).

| Model | Backend | Accuracy | Decision accuracy | Noul | Choice | Shared | Multi | Shared decision accuracy | Multi label accuracy |
| --- | :---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-lite 0.85B | GGUF Q8_0 | 68.3% | 82.8% | 83.3% | 80.0% | 76.7% | 33.3% | 91.6% | 79.6% |
| Kev 0.8B | MLX | 52.5% | 78.5% | 66.7% | 66.7% | 56.7% | 20.0% | 83.1% | 80.1% |
| Laya English 421M | MLX | 26.7% | 56.1% | 46.7% | 43.3% | 13.3% | 3.3% | 57.8% | 58.7% |

#### Performance

Median and P95 are per-request latencies; throughput is scored decisions per second over the whole run; cost is the estimated USD per 1 000 scored decisions from the API's reported input tokens and its published input price (remote profiles only; an estimate, not a bill).

| Model | Backend | Median | P95 | Throughput | Cost / 1k decisions |
| --- | :---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-lite 0.85B | GGUF Q8_0 | 135.9 ms | 592.7 ms | 11.60/s | — |
| Kev 0.8B | MLX | 86.2 ms | 171.0 ms | 29.94/s | — |
| Laya English 421M | MLX | 67.8 ms | 285.6 ms | 23.84/s | — |

#### Identity

| Model | Backend | Engine | Runtime | Precision | Source | Model / endpoint | Revision |
| --- | :---: | --- | --- | --- | --- | --- | --- |
| Decision 3.0 d3-lite 0.85B | GGUF Q8_0 | decision3 | llama.cpp-systemone | q8_0 | official | `ggml-org/vllm-d3-lite-GGUF` | a1f9fa12 |
| Kev 0.8B | MLX | kev | — | — | official | `jaredpalmer/kev-0.8b@bf75a6a8848ea6960ff2ed108d9ed44c2941174f` | bf75a6a8 |
| Laya English 421M | MLX | laya | — | — | community | `aac6fef/laya-mlx` | 04767856 |

### Models up to 1B — ENG Bench (0.6.4 snapshot)

#### Run

- `20261018T080133Z` · suite `deqio-basic-220` — 220 cases / 652 scored decisions · Deqio 0.6.4 · 2026-10-18
- runtime errors across the listed profiles: 0

#### Quality

`Accuracy` is whole-case accuracy; `Decision accuracy` scores every decision/assertion (a Shared case holds several; a Multi case scores every label as present or absent).

| Model | Backend | Accuracy | Decision accuracy | Noul | Choice | Shared | Multi | Shared decision accuracy | Multi label accuracy |
| --- | :---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-lite 0.85B | GGUF Q8_0 | 75.0% | 86.2% | 92.7% | 94.5% | 70.9% | 41.8% | 88.4% | 83.1% |
| Kev 0.8B | MLX | 61.8% | 80.8% | 81.8% | 87.3% | 58.2% | 20.0% | 82.9% | 78.8% |
| Laya English 421M | MLX | 45.9% | 73.5% | 63.6% | 65.5% | 27.3% | 27.3% | 65.9% | 79.4% |

#### Performance

Median and P95 are per-request latencies; throughput is scored decisions per second over the whole run; cost is the estimated USD per 1 000 scored decisions from the API's reported input tokens and its published input price (remote profiles only; an estimate, not a bill).

| Model | Backend | Median | P95 | Throughput | Cost / 1k decisions |
| --- | :---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-lite 0.85B | GGUF Q8_0 | 137.1 ms | 546.6 ms | 13.55/s | — |
| Kev 0.8B | MLX | 75.0 ms | 144.4 ms | 36.45/s | — |
| Laya English 421M | MLX | 47.1 ms | 166.7 ms | 41.01/s | — |

#### Identity

| Model | Backend | Engine | Runtime | Precision | Source | Model / endpoint | Revision |
| --- | :---: | --- | --- | --- | --- | --- | --- |
| Decision 3.0 d3-lite 0.85B | GGUF Q8_0 | decision3 | llama.cpp-systemone | q8_0 | official | `ggml-org/vllm-d3-lite-GGUF` | a1f9fa12 |
| Kev 0.8B | MLX | kev | — | — | official | `jaredpalmer/kev-0.8b@bf75a6a8848ea6960ff2ed108d9ed44c2941174f` | bf75a6a8 |
| Laya English 421M | MLX | laya | — | — | community | `aac6fef/laya-mlx` | 04767856 |

### Models from 1B to 3B — PL Bench (0.6.4 snapshot)

#### Run

- `20261018T082457Z` · suite `deqio-pl-120` — 120 cases / 344 scored decisions · Deqio 0.6.4 · 2026-10-18
- runtime errors across the listed profiles: 0

#### Quality

`Accuracy` is whole-case accuracy; `Decision accuracy` scores every decision/assertion (a Shared case holds several; a Multi case scores every label as present or absent).

| Model | Backend | Accuracy | Decision accuracy | Noul | Choice | Shared | Multi | Shared decision accuracy | Multi label accuracy |
| --- | :---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-nano 2.2B | GGUF Q8_0 | 78.3% | 89.0% | 93.3% | 93.3% | 86.7% | 40.0% | 95.2% | 85.1% |
| Basal 2.0 Mini — 1.5B | MLX 8-bit | 75.8% | 87.8% | 83.3% | 90.0% | 86.7% | 43.3% | 95.2% | 85.1% |
| Decider 2B | GGUF Q8_0 | 74.2% | 87.5% | 90.0% | 90.0% | 83.3% | 33.3% | 92.8% | 84.6% |

#### Performance

Median and P95 are per-request latencies; throughput is scored decisions per second over the whole run; cost is the estimated USD per 1 000 scored decisions from the API's reported input tokens and its published input price (remote profiles only; an estimate, not a bill).

| Model | Backend | Median | P95 | Throughput | Cost / 1k decisions |
| --- | :---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-nano 2.2B | GGUF Q8_0 | 320.7 ms | 1,420.1 ms | 4.86/s | — |
| Basal 2.0 Mini — 1.5B | MLX 8-bit | 232.3 ms | 496.2 ms | 10.24/s | — |
| Decider 2B | GGUF Q8_0 | 225.9 ms | 878.9 ms | 7.83/s | — |

#### Identity

| Model | Backend | Engine | Runtime | Precision | Source | Model / endpoint | Revision |
| --- | :---: | --- | --- | --- | --- | --- | --- |
| Decision 3.0 d3-nano 2.2B | GGUF Q8_0 | decision3 | llama.cpp-systemone | q8_0 | official | `ggml-org/vllm-d3-nano-GGUF` | 67baf946 |
| Basal 2.0 Mini — 1.5B | MLX 8-bit | basal | basal-2.0-mlx 2.0.0 | mlx-8bit | official | `Remek/basal-2.0-mini-MLX-8bit` | f26d1e2d |
| Decider 2B | GGUF Q8_0 | decider | decider-gguf-native | q8_0 | official | `Mapika/decider-2b-GGUF` | ff2e5e68 |

### Models from 1B to 3B — ENG Bench (0.6.4 snapshot)

#### Run

- `20261018T081947Z` · suite `deqio-basic-220` — 220 cases / 652 scored decisions · Deqio 0.6.4 · 2026-10-18
- runtime errors across the listed profiles: 0

#### Quality

`Accuracy` is whole-case accuracy; `Decision accuracy` scores every decision/assertion (a Shared case holds several; a Multi case scores every label as present or absent).

| Model | Backend | Accuracy | Decision accuracy | Noul | Choice | Shared | Multi | Shared decision accuracy | Multi label accuracy |
| --- | :---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-nano 2.2B | GGUF Q8_0 | 80.0% | 90.5% | 96.4% | 90.9% | 81.8% | 50.9% | 92.7% | 88.6% |
| Decider 2B | GGUF Q8_0 | 79.5% | 90.5% | 90.9% | 92.7% | 83.6% | 50.9% | 93.9% | 88.6% |
| Basal 2.0 Mini — 1.5B | MLX 8-bit | 73.6% | 87.9% | 78.2% | 92.7% | 69.1% | 54.5% | 85.4% | 89.7% |

#### Performance

Median and P95 are per-request latencies; throughput is scored decisions per second over the whole run; cost is the estimated USD per 1 000 scored decisions from the API's reported input tokens and its published input price (remote profiles only; an estimate, not a bill).

| Model | Backend | Median | P95 | Throughput | Cost / 1k decisions |
| --- | :---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-nano 2.2B | GGUF Q8_0 | 333.5 ms | 1,394.1 ms | 5.58/s | — |
| Decider 2B | GGUF Q8_0 | 213.8 ms | 796.7 ms | 9.79/s | — |
| Basal 2.0 Mini — 1.5B | MLX 8-bit | 314.2 ms | 629.8 ms | 8.41/s | — |

#### Identity

| Model | Backend | Engine | Runtime | Precision | Source | Model / endpoint | Revision |
| --- | :---: | --- | --- | --- | --- | --- | --- |
| Decision 3.0 d3-nano 2.2B | GGUF Q8_0 | decision3 | llama.cpp-systemone | q8_0 | official | `ggml-org/vllm-d3-nano-GGUF` | 67baf946 |
| Decider 2B | GGUF Q8_0 | decider | decider-gguf-native | q8_0 | official | `Mapika/decider-2b-GGUF` | ff2e5e68 |
| Basal 2.0 Mini — 1.5B | MLX 8-bit | basal | basal-2.0-mlx 2.0.0 | mlx-8bit | official | `Remek/basal-2.0-mini-MLX-8bit` | f26d1e2d |

### Models above 3B — PL Bench (0.6.4 snapshot)

#### Run

- `20261018T093333Z` · suite `deqio-pl-120` — 120 cases / 344 scored decisions · Deqio 0.6.4 · 2026-10-18
- runtime errors across the listed profiles: 0

#### Quality

`Accuracy` is whole-case accuracy; `Decision accuracy` scores every decision/assertion (a Shared case holds several; a Multi case scores every label as present or absent).

| Model | Backend | Accuracy | Decision accuracy | Noul | Choice | Shared | Multi | Shared decision accuracy | Multi label accuracy |
| --- | :---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Decider 4B | GGUF Q8_0 | 80.8% | 92.4% | 93.3% | 93.3% | 96.7% | 40.0% | 98.8% | 89.6% |
| Decision 3.0 d3-mini 4.5B | GGUF Q8_0 | 82.5% | 92.2% | 96.7% | 96.7% | 93.3% | 43.3% | 97.6% | 88.6% |
| Basal 2.0 — 4.5B | MLX 8-bit | 80.0% | 91.6% | 90.0% | 93.3% | 83.3% | 53.3% | 92.8% | 91.0% |
| JevK5 4B | GGUF Q8_0 | 82.5% | 90.4% | 96.7% | 96.7% | 96.7% | 40.0% | 98.8% | 85.1% |
| SemIf / Qwen3.5 4B | MLX | 80.8% | 89.8% | 96.7% | 96.7% | 90.0% | 40.0% | 96.4% | 85.1% |
| Kev 4B | GGUF Q8_0 | 75.0% | 82.8% | 93.3% | 96.7% | 93.3% | 16.7% | 97.6% | 73.1% |

#### Performance

Median and P95 are per-request latencies; throughput is scored decisions per second over the whole run; cost is the estimated USD per 1 000 scored decisions from the API's reported input tokens and its published input price (remote profiles only; an estimate, not a bill).

| Model | Backend | Median | P95 | Throughput | Cost / 1k decisions |
| --- | :---: | ---: | ---: | ---: | ---: |
| Decider 4B | GGUF Q8_0 | 560.5 ms | 2,299.8 ms | 3.12/s | — |
| Decision 3.0 d3-mini 4.5B | GGUF Q8_0 | 1,031.1 ms | 4,694.0 ms | 1.56/s | — |
| Basal 2.0 — 4.5B | MLX 8-bit | 953.5 ms | 1,964.7 ms | 2.62/s | — |
| JevK5 4B | GGUF Q8_0 | 1,188.2 ms | 5,235.8 ms | 1.39/s | — |
| SemIf / Qwen3.5 4B | MLX | 968.1 ms | 3,005.5 ms | 2.03/s | — |
| Kev 4B | GGUF Q8_0 | 713.4 ms | 1,672.9 ms | 3.54/s | — |

#### Identity

| Model | Backend | Engine | Runtime | Precision | Source | Model / endpoint | Revision |
| --- | :---: | --- | --- | --- | --- | --- | --- |
| Decider 4B | GGUF Q8_0 | decider | decider-gguf-native | q8_0 | official | `Mapika/decider-4b-GGUF` | b79f09d9 |
| Decision 3.0 d3-mini 4.5B | GGUF Q8_0 | decision3 | llama.cpp-systemone | q8_0 | official | `ggml-org/vllm-d3-mini-GGUF` | c03ac9c3 |
| Basal 2.0 — 4.5B | MLX 8-bit | basal | basal-2.0-mlx 2.0.0 | mlx-8bit | official | `Remek/basal-2.0-4.5B-MLX-8bit` | 77d3b723 |
| JevK5 4B | GGUF Q8_0 | jevk5 | jevk5-gguf-native | q8_0 | official | `alibiserikbay/JevK5-GGUF` | ec67b0bf |
| SemIf / Qwen3.5 4B | MLX | semif | — | — | official | `Qwen/Qwen3.5-4B` | 851bf6e8 |
| Kev 4B | GGUF Q8_0 | kev | llama.cpp-systemone | q8_0 | official | `ggml-org/Kev-4B-GGUF` | d924f2e2 |

### Models above 3B — ENG Bench (0.6.4 snapshot)

#### Run

- `20261018T085501Z` · suite `deqio-basic-220` — 220 cases / 652 scored decisions · Deqio 0.6.4 · 2026-10-18
- runtime errors across the listed profiles: 2

#### Quality

`Accuracy` is whole-case accuracy; `Decision accuracy` scores every decision/assertion (a Shared case holds several; a Multi case scores every label as present or absent).

| Model | Backend | Accuracy | Decision accuracy | Noul | Choice | Shared | Multi | Shared decision accuracy | Multi label accuracy |
| --- | :---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-mini 4.5B | GGUF Q8_0 | 89.1% | 95.7% | 96.4% | 100.0% | 96.4% | 63.6% | 98.8% | 93.7% |
| Decider 4B | GGUF Q8_0 | 86.8% | 94.6% | 98.2% | 98.2% | 90.9% | 60.0% | 97.0% | 92.6% |
| Basal 2.0 — 4.5B | GGUF Q8_0 | 85.5% | 94.2% | 89.1% | 96.4% | 89.1% | 67.3% | 96.3% | 93.7% |
| Basal 2.0 — 4.5B | MLX 8-bit | 85.5% | 94.2% | 89.1% | 96.4% | 90.9% | 65.5% | 97.0% | 93.4% |
| JevK5 4B | GGUF Q8_0 | 85.5% | 92.9% | 96.4% | 98.2% | 96.4% | 50.9% | 98.8% | 89.2% |
| Kev 4B | GGUF Q8_0 | 80.5% | 91.7% | 96.4% | 100.0% | 78.2% | 47.3% | 92.1% | 89.7% |
| SemIf / Qwen3.5 4B | MLX | 80.5% | 88.2% | 98.2% | 96.4% | 87.3% | 40.0% | 95.1% | 82.5% |

#### Performance

Median and P95 are per-request latencies; throughput is scored decisions per second over the whole run; cost is the estimated USD per 1 000 scored decisions from the API's reported input tokens and its published input price (remote profiles only; an estimate, not a bill).

| Model | Backend | Median | P95 | Throughput | Cost / 1k decisions |
| --- | :---: | ---: | ---: | ---: | ---: |
| Decision 3.0 d3-mini 4.5B | GGUF Q8_0 | 1,096.4 ms | 5,012.8 ms | 1.61/s | — |
| Decider 4B | GGUF Q8_0 | 525.7 ms | 2,068.8 ms | 3.91/s | — |
| Basal 2.0 — 4.5B | GGUF Q8_0 | 1,243.1 ms | 2,059.4 ms | 2.43/s | — |
| Basal 2.0 — 4.5B | MLX 8-bit | 1,248.9 ms | 2,481.6 ms | 2.13/s | — |
| JevK5 4B | GGUF Q8_0 | 1,491.3 ms | 5,341.8 ms | 1.40/s | — |
| Kev 4B | GGUF Q8_0 | 663.7 ms | 1,576.8 ms | 3.90/s | — |
| SemIf / Qwen3.5 4B | MLX | 1,092.7 ms | 3,165.8 ms | 2.07/s | — |

#### Identity

| Model | Backend | Engine | Runtime | Precision | Source | Model / endpoint | Revision |
| --- | :---: | --- | --- | --- | --- | --- | --- |
| Decision 3.0 d3-mini 4.5B | GGUF Q8_0 | decision3 | llama.cpp-systemone | q8_0 | official | `ggml-org/vllm-d3-mini-GGUF` | c03ac9c3 |
| Decider 4B | GGUF Q8_0 | decider | decider-gguf-native | q8_0 | official | `Mapika/decider-4b-GGUF` | b79f09d9 |
| Basal 2.0 — 4.5B | GGUF Q8_0 | basal | basal-2.0-gguf 2.0.0 | q8_0 | official | `Remek/basal-2.0-4.5B` | e5ba9149, 237294ca |
| Basal 2.0 — 4.5B | MLX 8-bit | basal | basal-2.0-mlx 2.0.0 | mlx-8bit | official | `Remek/basal-2.0-4.5B-MLX-8bit` | 77d3b723 |
| JevK5 4B | GGUF Q8_0 | jevk5 | jevk5-gguf-native | q8_0 | official | `alibiserikbay/JevK5-GGUF` | ec67b0bf |
| Kev 4B | GGUF Q8_0 | kev | llama.cpp-systemone | q8_0 | official | `ggml-org/Kev-4B-GGUF` | d924f2e2 |
| SemIf / Qwen3.5 4B | MLX | semif | — | — | official | `Qwen/Qwen3.5-4B` | 851bf6e8 |

## Install from source

```bash
git clone https://github.com/ILuce/deqio.git && cd deqio
uv sync --frozen --extra dev        # Deqio core + the deqio-client SDK (workspace member)
uv run ruff check src tests clients
uv run pytest
uv run deqio models setup
uv run deqio serve
```

CI (`.github/workflows/ci.yml`) runs the same lint, byte-compile and test steps with Python 3.10 and 3.12. Release
notes: [CHANGELOG.md](https://github.com/ILuce/deqio/blob/v0.6.5/CHANGELOG.md); release procedure: [docs/RELEASE_CHECKLIST.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/RELEASE_CHECKLIST.md);
security reporting: [SECURITY.md](https://github.com/ILuce/deqio/blob/v0.6.5/SECURITY.md); the engineering rules of the project: [AGENTS.md](https://github.com/ILuce/deqio/blob/v0.6.5/AGENTS.md).

## Use the API

The server listens on `http://127.0.0.1:8787`. Every route takes and returns JSON; the full reference with an
example per route is [docs/API.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/API.md), and `/docs` renders the OpenAPI document of the running server.

| Route | Answers | Use it for |
| --- | --- | --- |
| `POST /v1/noul` | yes / no with probabilities | true/false checks, guardrails |
| `POST /v1/choice` | exactly one option | routing, classification, triage (`/v1/decision` is an alias) |
| `POST /v1/shared` | several decisions over one state, one engine call | checklists, multi-view validation |
| `POST /v1/score` | one level of an ordered scale | severity, risk, urgency |
| `POST /v1/multi` | a set of labels, one probability each | tagging, multi-label classification |
| `POST /v1/act` | the action with the lowest expected cost | decisions where mistakes have different prices |
| `POST /v1/soam` | one state, many typed questions, one native request | agent checkpoints, release validation |
| `POST /v1/systemone` | the same, in the raw System One contract | low-level integrations, adapters |

`noul`, `choice` and `shared` are the portable routes: the same request and response on every model. `score`,
`multi`, `act`, `soam` and `systemone` are the native typed routes: they forward the model's own System One
contract and are available when the active profile supports them — `deqio models installed` shows the capability
matrix, and a feature the model lacks is refused with `422`, never silently ignored.

A yes/no question:

```bash
curl -s http://127.0.0.1:8787/v1/noul -H 'content-type: application/json' -d '{
  "state": "The patch changed authentication code. Unit tests passed but integration tests have not run.",
  "question": "Should integration tests run before release?"
}'
```

Exactly one option:

```bash
curl -s http://127.0.0.1:8787/v1/choice -H 'content-type: application/json' -d '{
  "state": "A customer was charged twice for one subscription renewal.",
  "question": "Which team should handle this request?",
  "options": [
    {"id": "billing", "description": "Payments, refunds and invoices."},
    {"id": "support", "description": "Product and account support."},
    {"id": "sales", "description": "New purchases and upgrades."}
  ]
}'
```

One state, several typed questions in one native request:

```bash
curl -s http://127.0.0.1:8787/v1/soam -H 'content-type: application/json' -d '{
  "state": "The laptop arrived with a cracked screen and the customer requests a refund.",
  "questions": {
    "route":    {"type": "choice", "instructions": "Which workflow handles this?",
                 "criteria": {"refund": "Refund workflow.", "repair": "Repair workflow.", "review": "Human review."}},
    "damaged":  {"type": "noul", "instructions": "Is the item damaged?"},
    "severity": {"type": "score", "instructions": "Rate severity.", "criteria": ["low", "medium", "high"]}
  }
}'
```

Every answer carries `probabilities` and a `provenance` block (engine, model, backend, pinned revisions, and the
provider, request id and cost estimate for a remote profile), so a decision can always be tied to the exact model
that made it. Operational routes: `/health`, `/v1/models`, `/v1/models/installed`, `/v1/models/activate`,
`/v1/recent`, `/v1/watch`.

## Use the Python client

`deqio-client` is a dependency-free Python SDK, installed together with Deqio (or alone with
`pip install deqio-client`). One method per route; every method returns the server's JSON unchanged and raises
`DeqioError` with the status and `detail` when the server rejects the request:

```python
from deqio_client import DeqioClient, DeqioError

client = DeqioClient("http://127.0.0.1:8787")

degraded = client.noul("Checkout works after retry, but payment latency is elevated.", "Is checkout degraded?")
report = client.choice("The API returns HTTP 500 after the latest deployment.", "Classify the report.",
                       {"bug": "An existing function no longer works", "feature": "A request for new functionality"})
severity = client.score("Payment latency is elevated for some customers.",
                        {"instructions": "Rate the operational severity.", "criteria": ["low", "medium", "high", "critical"]})
views = client.soam("One confirmed state.", {"ok": {"type": "noul", "instructions": "Is it fine?"},
                                            "kind": {"type": "choice", "criteria": {"a": "A", "b": "B"}}})

print(report["decision"], report["probabilities"])
print(severity["answers"]["decision"], degraded["provenance"]["engine"])
```

`client.shared`, `client.multi`, `client.act` and `client.systemone` cover the remaining routes; `contract=True`
negotiates the fail-closed input-completeness contract on the portable routes. The same calls are available from the
shell through the SDK:

```bash
deqio decide noul   --state "Payment latency is elevated." --question "Is checkout degraded?"
deqio decide choice --state "API returns 500 after deploy." --question "Classify." --option bug="Broken function" --option feature="New request"
deqio decide multi  --state "Cracked screen, damaged box, refund asked." --criterion damage="Product damage" --criterion refund="Refund requested"
deqio decide soam   --state-file state.txt --questions-json @questions.json --json
```

## Summary

Deqio turns a decision model into a local service: one API for yes/no, single-choice, scored, multi-label and
cost-aware decisions, 31 models from 322M to 27B parameters on MLX, GGUF and CUDA plus two remote APIs, every answer
with probabilities and provenance, a browser dashboard and live request view, and benchmark suites in English and
Polish whose current results you have just read. Install it with `uv tool install deqio`, pick a model with
`deqio models setup`, and start with `/v1/choice` or the Python client. Operational details live in
[docs/API.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/API.md), [docs/MODELS.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/MODELS.md) and [docs/BENCHMARKS.md](https://github.com/ILuce/deqio/blob/v0.6.5/docs/BENCHMARKS.md).

## License

MIT. See [LICENSE](https://github.com/ILuce/deqio/blob/v0.6.5/LICENSE).
