onpar proves which open-weight model can replace your closed-model API, on your own traffic and hardware, then moves you across with a quality gate and a rollback.
uvx --from onpar onpar fit
$ onpar fit --context 32k --concurrency 8 M4 Max · 128 GB · 546 GB/s usable: 96 GB model quant total ~tok/s licence ───────────────────────────────────────────────── Qwen3 30B-A3B (MoE) q8 56.2G 60 Apache-2.0 Phi-4 14B q8 66.3G 18 MIT NOT FEASIBLE Kimi K3 (2.8T MoE) weights alone need 1,467 GB at q4 — sparsity cuts compute, not memory $ onpar receipt receipt.json Migration receipt · gpt-4o-mini → qwen3-30b-a3b NOT PROVEN either way — not enough evidence: · tool-calling (2 tools) 100% [76%–100%] over 12 items — needs 35 Proven at or above the 90% bar: · structured JSON · <=1k context 100% [91%–100%] over 40 items Movable: 20% of captured traffic
The problem
A leaderboard says nothing about whether a model can extract line items from your invoices. onpar turns your own traffic into a proof, a serving config and a cutover with a rollback. Seven stages, and each one can say no.
One real run · recorded 2026-11-03 on an M4 Max
gpt-4o-mini was the incumbent. Qwen3 30B-A3B, served locally through llama.cpp at q8, was the candidate. Every figure below is quoted from the recordings the docs embed.
| Stage | What the run printed |
|---|---|
| Distill | 197 captures → 7 task shapes → 197 eval items |
| Agent traffic | 12 agent sessions · 36 steps (18% of captures) |
| Agent moves | decision agreement 100% [90–100] over 36 tool-context steps · κ=1.00 |
| Whole sessions | 12/12 sessions followed the incumbent's path to the end |
| Structured JSON | 100% [91–100] over 40 items: every reply the JSON object asked for, because the server enforces response_format |
| Weighted verdict | 96% [90–97] |
| Decision | Move 20% of traffic, the one proven shape; six are not proven either way |
The agent work is flawless and still not proven, because 12 flawless items give a lower bound of 76%, not 90%. Only the shape with 40 items moves, and the receipt prints what is not proven before what is. Walk the whole run.
It says no
Each card quotes a recording from the run above. A refusal names the number that caused it and what would change the answer.
The Wilson lower bound at a perfect score is n / (n + 3.84). It takes 35 flawless items to clear 90%, and the report says how many more each task needs:
needs 35 graded items to clear the 90% bar at its observed 100% (has 12, 23 to go)
Asked for a 2 ms inter-token budget, the planner did not tune around it. It printed the gap under CANNOT MEET, with the flags it would have used:
inter-token budget is 2ms, but decode is a roofline estimate of 60 tok/s — about 17ms per token
Qwen3 thinks before it answers and spends a small token budget on reasoning. So
onpar run serves it with thinking off, here
--reasoning-budget 0 on llama.cpp, and the probe got its answer:
answered 'billing' in 2 tokens
What it sets for you
The planner decides what should be true, then each engine adapter spells it in
that engine's own flags. The output is a real vllm serve, a real
InferencePool, a real mlx_lm.server line. It runs with onpar
uninstalled.
Context length, max concurrent sequences, prefill chunk, prefix reuse, speculative decoding, memory fraction, tensor parallelism, structured output, quantisation, KV-cache dtype, LoRA fleet, the tool-calling parser and thinking. Each carries its reason.
vLLM, vLLM-TPU, SGLang, llm-d, llama.cpp, MLX and Ollama. On Apple silicon the CUDA engines cannot run at all, so the choice there is MLX, llama.cpp or Ollama, and the plan says why.
For the run above, MLX had no flag for four settings, and the plan said so:
context_length · speculative · memory_fraction · quantization
Each line under NOT EXPRESSED carries the reason, never a guessed flag.
The tool-calling parser is a knob because a wrong one fails
silently. With the wrong parser, tool calls come back as plain text with
finish_reason: stop. Nothing errors; the agent stops acting.
Every catalogue model on every machine
Pick a machine and a context. Every number is the solver's output for that
machine, the same arithmetic onpar fit runs. Use
onpar fit --explain <model> to see the full breakdown.
Fit data could not be loaded. Run
python3 tools/site_data.py to regenerate
site/assets/fit-explorer.json.
| Sizing trap | What people do | What it costs |
|---|---|---|
| MoE | Size on active parameters, because that's the headline number. | Weights need total params resident. Sparsity cuts compute, not memory: a 2.8T MoE needs 1,467 GB at q4 whatever its active count. |
| GQA | Use attention heads in the KV formula instead of KV heads. | Up to an 8× overestimate. The model fits; the tool says it doesn't. |
| MLA | Apply the GQA formula to a DeepSeek-family model. | 64× overestimate for DeepSeek V3. MLA stores a compressed latent, not per-head K/V. |
Compare · as of 31 Oct 2026
The five products closest to the thesis, against the six stages a migration needs. Each is more mature than onpar at what it does. None joins the stages up.
| Product | Capture proxy | Session replay, recorded tool results | Statistical verdict | Hardware sizing | Native serving config | Cutover gate + rollback |
|---|---|---|---|---|---|---|
| Kitaru 316 stars · v0.7.0, 30 Oct replay with a model swap | — | ✓ | — | — | — | — |
| Langfuse 36.8k stars · v4.72.0, 29 Oct experiments on traces | — | — | — | — | — | — |
| Braintrust closed · Pro $249/mo, third-party reported trace to eval, hosted | — | — | — | — | — | — |
| promptfoo 26.7k stars · 0.125.1, 25 Oct output-level evals, CLI | — | — | — | — | — | — |
| LiteLLM 62.4k stars · v1.106.6, 28 Oct gateway, no quality judgement | ✓ | — | — | — | — | — |
| onpar this project the whole path | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Three doors in
Every door leads to the same solver and the same loop. Pick the one that matches what you know today.
Type onpar and describe what you are building. It asks one question
at a time, only when the answer changes the plan, and you leave with a model, a
quantisation and the memory it needs.
onpar learn basics is the first of thirty-six lessons in five tracks:
basics, intermediate, hardware, serving and hosting. Every figure is computed by onpar
for the machine you are on.
onpar doctor checks the gateway, the extension and an engine in the order
the loop needs them, and prints the next command. Then observe, distill, fit, prove,
migrate. Each stage can say no.
Agents come through the same doors.
onpar-mcp exposes eleven read-only tools: fit, explain, where, catalog, advise,
build, distill, prove, learn, receipt and guard. There is no cutover, deploy or route tool to
call. Exit code 2 means the call was wrong, so fix it. Exit code 1 means the world did not
cooperate or the answer is no, so read why.
No login, no tokens, no account
Run onpar with no arguments and it asks what you are
building — one question at a time, only when the answer would change the plan, and you
can stop at any point and still walk away with the best answer so far. In a script or a
CI step it stays a usage error rather than waiting on input.
The sizing solver has zero runtime dependencies and runs offline. Nothing leaves your machine — captured traffic is the most sensitive data you have, and export is an explicit command, never a background sync.
The package is onpar; the command is
onpar. PyPI accepted the bare name, so pip install onpar and
uvx onpar both work. npm refused it as too close to an existing package,
so there the distribution is onpar-cli while the command it installs is
still onpar, and npx onpar-cli works. On macOS, brew install
dshakes/tap/onpar installs from the shared tap; the formula is pinned to the release
tarball by checksum and a clean arm64 runner installs and runs it on every release and
weekly (the brew-install workflow).
Then ask the inverse.
onpar where deepseek-v3 --context 128k answers what would I need to
run this? across 20 hardware classes — including TPU v5e, v6e and v5p, sized per
host because a TPU is not rented by the chip.