Northwind Agent Ops
Tracing · audit · online & offline evalsResponse latency
Per-turn latency, p50 and p95 per time bucket (seconds)
Online eval quality
Mean LLM-judge scores on live turns (0–1; policy = pass rate)
Turns
Customer turns per time bucket (offline eval traffic excluded)
Tool calls
How often the agent used each tool in the selected range
Offline eval runs
Golden-set pass rate per run (oldest → newest). Run python ops/offline_evals.py to add one.