real prompts · split tuning (seed 11) · store ee5406f19cf9 · code 7c1aed203706
1688 pooled pairs, labels missing 0

task prompts: 120 person-typed · 47 with an applying lesson · 104 applying lessons in the pool
  arm                precision [95%]  applying found  prompts hit  briefed any  lessons/prompt  off-topic/prompt  repeat
  floor           9.8% [8.0%, 12.0%]      85 of 104      40 of 47     110 of 120             7.23              6.49      1%
  calibrated    16.5% [11.1%, 24.0%]      21 of 104      17 of 47      42 of 120             1.07              0.88      2%
  prev           10.5% [7.3%, 14.9%]      26 of 104      21 of 47      69 of 120             2.08              1.85     20%
  prev-own       10.9% [7.6%, 15.5%]      26 of 104      21 of 47      68 of 120             1.99              1.77     11%
  reply-own       8.6% [6.4%, 11.4%]      42 of 104      31 of 47     114 of 120             4.11              3.73      5%

null prompts: 60 person-typed · 9 with an applying lesson · 11 applying lessons in the pool
  arm                precision [95%]  applying found  prompts hit  briefed any  lessons/prompt  off-topic/prompt  repeat
  floor            2.0% [0.8%, 4.5%]       5 of 11        5 of 9       44 of 60              4.57              4.17      0%
  calibrated      0.0% [0.0%, 17.6%]       0 of 11        0 of 9        6 of 60              0.30              0.30      0%
  prev            7.3% [2.9%, 17.3%]       4 of 11        2 of 9       17 of 60              0.92              0.85     45%
  prev-own         0.0% [0.0%, 9.6%]       0 of 11        0 of 9       11 of 60              0.60              0.60     11%
  reply-own        3.2% [1.3%, 8.0%]       4 of 11        4 of 9       39 of 60              2.15              2.00      2%

selection rule: no context arm (none is at least as precise as calibrated on task prompts)
