$ slmeval report --format table
# the same table as the terminal prints it
# regenerated by scripts/teardown.py - do not edit


  teardown-01-support-agent · 4 cases x 3 runs · replay · no faults · seed 42 · throughput
  20261104-225237-a7e448 · 8989e4c · dataset 04017f688c30edda

  ✗ FORBIDDEN tool call: issue_refund in refund_needs_confirmation (refund-happy), 3x
  ! 3 execution(s) across 1 candidate/case pair(s) explored paths the cassettes do not cover 
(refund_needs_confirmation) - excluded from pass rates rather than counted against the candidate; 
re-run `slmeval record --fill-gaps`
  ! fenced-json needed json_mode on 100% of structured calls
  ! fenced-json needed prompt-repair or a fallback rung on 100% of structured calls
  ! some token counts were estimated, not reported by the provider - figures built from them are 
marked ~
  · latency not measured in throughput mode - a p95 here would describe the scheduler rather than 
the model (re-run with --serial-latency)

Candidate        pass^3         95% CI     runs    Schema    Tool F1    Steps      $/case    Δ Cost
baseline         100.0%    [51%, 100%]    12/12      100%       1.00      4.8    ~$0.0000         —
sloppy-router     33.3%      [6%, 79%]      3/9      100%       0.40      1.8    ~$0.0000         —
fenced-json      100.0%    [51%, 100%]    12/12        0%       1.00      4.8    ~$0.0000         —
refund-happy      75.0%     [30%, 95%]     9/12      100%       0.96      4.8    ~$0.0000         —

per-node attribution                                                     
Node             baseline    sloppy-router    fenced-json    refund-happy
order_lookup            —           6 fail              —               —
policy_check            —                —              —          3 fail
compose_reply        9 ok                —           9 ok            6 ok
intent_router       12 ok            12 ok          12 ok           12 ok

  sloppy-router failed in order_lookup, which it did not change → look at what it did change: 
intent_router
  refund-happy failed in policy_check, which it did not change → look at what it did change: 
compose_reply, intent_router

  pass^3 = fraction of cases passing every repeat; 95% CI is the Wilson interval on that fraction
  gate on the lower bound, never the point estimate (n=4 cases)
  prices as of 2026-05-01
