bench dense arm · model answerdotai/answerai-colbert-small-v1 · split selection · code fa0d8cafb35c
corpus: 182 live records, 182 vectors (title+symptom, aliases, triggers on 173 lessons) · queries sha 97ecbcb5d681 · nulls 550: 282 calibrate, 268 test
cost: cold load in a fresh process 0.99 s · load here 0.54 s · index 19.6 s · encode per prompt median 29.6 ms
decision score = evidence + W * maxsim; line at alpha 0.05 by split conformal

    W    line cap |           recall [95%] briefed/q | null test briefed anything [95%] | paraphrase
  0.0   8.250   3 |   83.9% [79.1%, 88.4%]      2.57 | 6 of 268 (2.2%, [1.0%, 4.8%]) | 60/83
  0.0   8.250   5 |   84.3% [79.5%, 88.8%]      3.59 | 6 of 268 (2.2%, [1.0%, 4.8%]) | 60/83
  5.0  12.909   3 |   83.9% [79.1%, 88.4%]      2.55 | 10 of 268 (3.7%, [2.0%, 6.7%]) | 60/83
  5.0  12.909   5 |   84.3% [79.5%, 88.8%]      3.54 | 10 of 268 (3.7%, [2.0%, 6.7%]) | 60/83
 10.0  17.569   3 |   83.9% [79.1%, 88.4%]      2.52 | 10 of 268 (3.7%, [2.0%, 6.7%]) | 60/83
 10.0  17.569   5 |   84.3% [79.5%, 88.8%]      3.48 | 10 of 268 (3.7%, [2.0%, 6.7%]) | 60/83
 20.0  26.890   3 |   83.9% [79.1%, 88.4%]      2.50 | 10 of 268 (3.7%, [2.0%, 6.7%]) | 60/83
 20.0  26.890   5 |   84.3% [79.5%, 88.8%]      3.41 | 10 of 268 (3.7%, [2.0%, 6.7%]) | 60/83
 40.0  45.524   3 |   83.9% [79.1%, 88.8%]      2.40 | 11 of 268 (4.1%, [2.3%, 7.2%]) | 59/83
 40.0  45.524   5 |   84.3% [79.5%, 88.8%]      3.18 | 11 of 268 (4.1%, [2.3%, 7.2%]) | 59/83
