bench v2 prototype · label v2-shared_words · split heldout · code cd04216662a0
corpus: 182 live records · triggers on 173 lessons (triggers.jsonl, sha 814c7bb5aa68) · queries sha 97ecbcb5d681 · nulls 550 (sha 6e100bad522c): 282 calibrate the line, 268 test it
score: shared_words (How many of the prompt's subject words the about-text holds, ties broken by their)

alpha   line cap |           recall [95%] briefed/q | null test: briefed anything [95%]
 0.01  4.002   3 |   80.7% [76.3%, 85.2%]      2.10 | 4 of 268 (1.5%, [0.6%, 3.8%])
 0.01  4.002   5 |   81.1% [76.7%, 85.6%]      2.67 | 4 of 268 (1.5%, [0.6%, 3.8%])
 0.01  4.002   8 |   81.9% [77.8%, 86.3%]      3.03 | 4 of 268 (1.5%, [0.6%, 3.8%])
 0.05  2.002   3 |   86.3% [81.9%, 90.4%]      2.92 | 9 of 268 (3.4%, [1.8%, 6.3%])
 0.05  2.002   5 |   90.0% [86.7%, 93.3%]      4.69 | 9 of 268 (3.4%, [1.8%, 6.3%])
 0.05  2.002   8 |   93.7% [90.7%, 96.3%]      7.05 | 9 of 268 (3.4%, [1.8%, 6.3%])
  0.1  2.002   3 |   86.3% [81.9%, 90.4%]      2.92 | 9 of 268 (3.4%, [1.8%, 6.3%])
  0.1  2.002   5 |   90.0% [86.7%, 93.3%]      4.69 | 9 of 268 (3.4%, [1.8%, 6.3%])
  0.1  2.002   8 |   93.7% [90.7%, 96.3%]      7.05 | 9 of 268 (3.4%, [1.8%, 6.3%])

at alpha 0.05, cap 5, by register: bug-report 86/90 · paraphrase 73/90 · ticket 84/90
latency per decision: median 1.1 ms · p95 1.7 ms
