local-first · zero telemetry · Apache-2.0

Is the open model
on par?

onpar proves which open-weight model can replace your closed-model API, on your own traffic and hardware, then moves you across with a quality gate and a rollback.

uvx --from onpar onpar fit

90%the interval, not the point97%
no config file to write your traffic, not a benchmark a rollback button, not a leap
onpar · M4 Max · recorded 2026-11-03
$ onpar fit --context 32k --concurrency 8   M4 Max · 128 GB · 546 GB/s       usable: 96 GB   model                 quant  total   ~tok/s  licence  ─────────────────────────────────────────────────  Qwen3 30B-A3B (MoE)   q8     56.2G   60     Apache-2.0  Phi-4 14B             q8     66.3G   18      MIT   NOT FEASIBLE  Kimi K3 (2.8T MoE)    weights alone need 1,467 GB at q4 —                        sparsity cuts compute, not memory $ onpar receipt receipt.json  Migration receipt · gpt-4o-mini → qwen3-30b-a3b  NOT PROVEN either way — not enough evidence:    · tool-calling (2 tools) 100% [76%–100%] over 12 items — needs 35  Proven at or above the 90% bar:    · structured JSON · <=1k context 100% [91%–100%] over 40 items  Movable: 20% of captured traffic 
2781
tests · 2484 Python, 297 Rust
0
bytes of telemetry, egress or accounts
21
diagrams, generated and palette-validated
7
engines, native config for each

The problem

Benchmarks are someone else's exam.
Your traffic is yours.

A leaderboard says nothing about whether a model can extract line items from your invoices. onpar turns your own traffic into a proof, a serving config and a cutover with a rollback. Seven stages, and each one can say no.

Observe your traffic, distill it into an eval set, fit the candidates to your hardware, serve one, prove it per task, migrate in shadow first, and guard the receipt once traffic moves.

One real run · recorded 2026-11-03 on an M4 Max

A perfect-looking run that moves one shape.
That is the point.

gpt-4o-mini was the incumbent. Qwen3 30B-A3B, served locally through llama.cpp at q8, was the candidate. Every figure below is quoted from the recordings the docs embed.

StageWhat the run printed
Distill197 captures → 7 task shapes → 197 eval items
Agent traffic12 agent sessions · 36 steps (18% of captures)
Agent movesdecision agreement 100% [90–100] over 36 tool-context steps · κ=1.00
Whole sessions12/12 sessions followed the incumbent's path to the end
Structured JSON100% [91–100] over 40 items: every reply the JSON object asked for, because the server enforces response_format
Weighted verdict96% [90–97]
DecisionMove 20% of traffic, the one proven shape; six are not proven either way

The agent work is flawless and still not proven, because 12 flawless items give a lower bound of 76%, not 90%. Only the shape with 40 items moves, and the receipt prints what is not proven before what is. Walk the whole run.

It says no

Every stage can refuse.
Here are three it did.

Each card quotes a recording from the run above. A refusal names the number that caused it and what would change the answer.

🎲thin sample

100% over 12 items is not a pass

The Wilson lower bound at a perfect score is n / (n + 3.84). It takes 35 flawless items to clear 90%, and the report says how many more each task needs:

needs 35 graded items to clear the 90% bar at its observed 100% (has 12, 23 to go)

⏱cannot meet

A budget no flag can close

Asked for a 2 ms inter-token budget, the planner did not tune around it. It printed the gap under CANNOT MEET, with the flags it would have used:

inter-token budget is 2ms, but decode is a roofline estimate of 60 tok/s — about 17ms per token

💭thinking off

A thinking model would answer nothing

Qwen3 thinks before it answers and spends a small token budget on reasoning. So onpar run serves it with thinking off, here --reasoning-budget 0 on llama.cpp, and the probe got its answer:

answered 'billing' in 2 tokens

What it sets for you

Thirteen knobs. Seven engines.
No config file to write.

The planner decides what should be true, then each engine adapter spells it in that engine's own flags. The output is a real vllm serve, a real InferencePool, a real mlx_lm.server line. It runs with onpar uninstalled.

Three forks decide most of it: prefix sharing at or above 0.55 picks the engine, concurrency above 16 turns drafting off, and the latency budget sets the prefill chunk. Projections are roofline estimates, not measured.
🎛️thirteen knobs

Set from your machine and load

Context length, max concurrent sequences, prefill chunk, prefix reuse, speculative decoding, memory fraction, tensor parallelism, structured output, quantisation, KV-cache dtype, LoRA fleet, the tool-calling parser and thinking. Each carries its reason.

🔀seven engines

One plan, native everywhere

vLLM, vLLM-TPU, SGLang, llm-d, llama.cpp, MLX and Ollama. On Apple silicon the CUDA engines cannot run at all, so the choice there is MLX, llama.cpp or Ollama, and the plan says why.

📋the honest part

What it cannot set, it lists

For the run above, MLX had no flag for four settings, and the plan said so:

context_length · speculative · memory_fraction · quantization

Each line under NOT EXPRESSED carries the reason, never a guessed flag.

The tool-calling parser is a knob because a wrong one fails silently. With the wrong parser, tool calls come back as plain text with finish_reason: stop. Nothing errors; the agent stops acting.

Every catalogue model on every machine

Will it fit?

Pick a machine and a context. Every number is the solver's output for that machine, the same arithmetic onpar fit runs. Use onpar fit --explain <model> to see the full breakdown.

Concurrency
Sizing trapWhat people doWhat it costs
MoE Size on active parameters, because that's the headline number. Weights need total params resident. Sparsity cuts compute, not memory: a 2.8T MoE needs 1,467 GB at q4 whatever its active count.
GQA Use attention heads in the KV formula instead of KV heads. Up to an 8× overestimate. The model fits; the tool says it doesn't.
MLA Apply the GQA formula to a DeepSeek-family model. 64× overestimate for DeepSeek V3. MLA stores a compressed latent, not per-head K/V.

Compare · as of 31 Oct 2026

Gateways assume the model is chosen.
Eval tools assume it's served.

The five products closest to the thesis, against the six stages a migration needs. Each is more mature than onpar at what it does. None joins the stages up.

ProductCapture proxySession replay, recorded tool results Statistical verdictHardware sizingNative serving config Cutover gate + rollback
Kitaru
316 stars · v0.7.0, 30 Oct
replay with a model swap
—✓————
Langfuse
36.8k stars · v4.72.0, 29 Oct
experiments on traces
——————
Braintrust
closed · Pro $249/mo, third-party reported
trace to eval, hosted
——————
promptfoo
26.7k stars · 0.125.1, 25 Oct
output-level evals, CLI
——————
LiteLLM
62.4k stars · v1.106.6, 28 Oct
gateway, no quality judgement
✓—————
onpar
this project
the whole path
✓✓✓✓✓✓

Stars, releases and pricing as recorded in a market scan dated 31 Oct 2026. A dash means the stage is absent or out of scope in the vendor's own material. Langfuse runs experiments on production traces but has no capture proxy or statistical verdict; Braintrust and Langfuse go trace to dataset, not tool-history replay; promptfoo is output-level only.

Three doors in

Ask in a sentence, learn on your numbers,
or run the whole loop.

Every door leads to the same solver and the same loop. Pick the one that matches what you know today.

💬no jargon

New to all of this

Type onpar and describe what you are building. It asks one question at a time, only when the answer changes the plan, and you leave with a model, a quantisation and the memory it needs.

📐your numbers

Curious how it works

onpar learn basics is the first of thirty-six lessons in five tracks: basics, intermediate, hardware, serving and hosting. Every figure is computed by onpar for the machine you are on.

All thirty-six lessons · the memory lab

🩺the loop

An engineer with traffic

onpar doctor checks the gateway, the extension and an engine in the order the loop needs them, and prints the next command. Then observe, distill, fit, prove, migrate. Each stage can say no.

Agents come through the same doors. onpar-mcp exposes eleven read-only tools: fit, explain, where, catalog, advise, build, distill, prove, learn, receipt and guard. There is no cutover, deploy or route tool to call. Exit code 2 means the call was wrong, so fix it. Exit code 1 means the world did not cooperate or the answer is no, so read why.

The MCP surface: eleven read-only tools, two resources confined to the eval root, three prompts, and a wall where cutover, deploy, route and promote would be
Not absent by policy, absent by construction. Captured traffic stays data, never instructions, for every agent that reads it.

No login, no tokens, no account

Try it in ten seconds.

A real recording of a bare onpar session: the user describes a support chatbot for 20 agents, the tool answers with the model, quantisation and memory it would use, shows which inference came from which words, states its assumptions, and asks one follow-up question. The user answers with a prompt length and it is understood with its provenance.
One question at a time, asked only when the answer would change the plan.

Run onpar with no arguments and it asks what you are building — one question at a time, only when the answer would change the plan, and you can stop at any point and still walk away with the best answer so far. In a script or a CI step it stays a usage error rather than waiting on input.

The sizing solver has zero runtime dependencies and runs offline. Nothing leaves your machine — captured traffic is the most sensitive data you have, and export is an explicit command, never a background sync.

The package is onpar; the command is onpar. PyPI accepted the bare name, so pip install onpar and uvx onpar both work. npm refused it as too close to an existing package, so there the distribution is onpar-cli while the command it installs is still onpar, and npx onpar-cli works. On macOS, brew install dshakes/tap/onpar installs from the shared tap; the formula is pinned to the release tarball by checksum and a clean arm64 runner installs and runs it on every release and weekly (the brew-install workflow).

Then ask the inverse. onpar where deepseek-v3 --context 128k answers what would I need to run this? across 20 hardware classes — including TPU v5e, v6e and v5p, sized per host because a TPU is not rented by the chip.