kvprof

what your clients send to llama-server, and why each request cost what it did connecting…
How to read this page

Every time a coding agent (Claude Code, OpenCode, …) needs the model it sends the whole conversation as one prompt: the system prompt, every tool definition, and all messages so far.

The server keeps the last prompt it processed. If the new prompt starts with exactly the same tokens, it reuses that work and only computes ("prefills") the new tail. Time to first token is almost entirely prefill time, so the question for every request is: how much was new, and why?

The request list answers it at a glance (grey = reused, orange = computed fresh), grouped by conversation when several agents or sessions share the server. Click a request: the panel on the right explains it in words, What's in the prompt shows what the tokens are, Why it cost this shows where the prompt first differs from the previous one, and Tokens shows the exact tokens the model read.

One rule to remember: reuse works best when the new prompt extends the previous one. Normal turns and tool results do that. Some architectures (hybrid/recurrent models: DeltaNet or Mamba hybrids) reuse only when the new prompt extends the previous one; plain-attention models reuse everything up to the first change. Anything that changes an earlier message costs a re-prefill from there (or from the last checkpoint before it).

Glossary
token
the unit the model reads; ~4 characters of English, fewer for code and JSON.
prefill
processing prompt tokens before the first output token.
reused / cached
prompt tokens the server already had processed for an earlier request and did not compute again (Anthropic usage: cache_read_input_tokens; OpenAI usage: prompt_tokens_details.cached_tokens).
TTFT
time to first token: from sending the request until the first piece of the answer arrived. Includes prefill and any waiting for the GPU.
decode
generating the answer, token by token.
slot
one conversation's cached state on the server. With more than one, a small side request need not evict the main conversation.
LCP / LRU
how the server picked a slot: LCP = it found a slot whose cached prompt shares a prefix with the new one (the fraction kept is f_keep); LRU = nothing matched, it took the least recently used slot and started from scratch.
end-extension
a prompt that is the previous prompt plus new tokens at the end. The cheap case.
tool round-trip
the client sending a tool's result back to the model; an end-extension whose cost is the size of the result.
template overhead
tokens added by the chat template (markup around tool schemas, turn markers) on top of the raw text the client sent.

Overview

Timeline

When each request ran, one lane per server slot. Bar length is wall time; the darker start of a bar is the wait for the first token (mostly prefill). Bars that overlap were sharing the GPU, which slows both. Click a bar to select it.

Requests

One row per request that reached the tap. When several conversations (sessions of Claude Code, OpenCode, or other clients) share the server, rows are grouped by conversation; click a group row to fold it, or untick "group by conversation" for a single timeline. Anything sent by a script rather than an agent (curl, a benchmark script) is marked "via …". The bar shows the prompt: grey is what the server reused, orange is what it computed fresh (this is what you wait for). Hover a column title or a kind badge for its meaning; j/k move the selection.

# time kind promptreused | fresh prompt reused fresh TTFT prefill decode cache first change check