Northwind Agent Ops

Tracing · audit · online & offline evals

Response latency

Per-turn latency, p50 and p95 per time bucket (seconds)

Online eval quality

Mean LLM-judge scores on live turns (0–1; policy = pass rate)

Turns

Customer turns per time bucket (offline eval traffic excluded)

Tool calls

How often the agent used each tool in the selected range

Offline eval runs

Golden-set pass rate per run (oldest → newest). Run python ops/offline_evals.py to add one.