effect: off (6679e8dcde7c) -> on (c952d8176362)  errors=[]
| prefill | conc | TTFT ms A | TTFT ms B | dTTFT | verdict | prefill tok/s A | prefill tok/s B | dprefill | verdict | groupings A / B |
|---:|---:|---:|---:|---:|---|---:|---:|---:|---|---|
| 2048 | 4 | 347.7±1.2 | 341.0±2.7 | -0.22% | NOISE | 5699 [5201-5869] | 5573 [5114-5811] | -0.44% | NOISE | 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 / 1x3,1x3,1x3,1x1 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 |
| 16384 | 4 | 1965.7±1.4 | 1989.9±1.0 | +1.23% | NOISE | 5565 [5455-5671] | 5505 [5420-5573] | -1.05% | NOISE | 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 / 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 |
| 65536 | 4 | 7547.4±0.7 | 7622.6±0.4 | +1.00% | NOISE | 5425 [5378-5482] | 5373 [5350-5399] | -0.96% | NOISE | 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 / 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 |

control: on (c952d8176362) -> null (b3c81cc456c2)  errors=[]
| prefill | conc | TTFT ms A | TTFT ms B | dTTFT | verdict | prefill tok/s A | prefill tok/s B | dprefill | verdict | groupings A / B |
|---:|---:|---:|---:|---:|---|---:|---:|---:|---|---|
| 2048 | 4 | 348.8±0.5 | 353.6±1.2 | -0.23% | NOISE | 5396 [4991-5741] | 5603 [5028-5784] | +0.23% | NOISE | 1x2,1x2,1x2,1x2 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 / 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 |
| 16384 | 4 | 2003.1±0.6 | 1984.1±0.6 | -0.91% | NOISE | 5472 [5437-5516] | 5515 [5468-5563] | +0.76% | NOISE | 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 / 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 |
| 65536 | 4 | 7647.7±0.2 | 7648.6±0.4 | +0.01% | NOISE | 5356 [5344-5375] | 5352 [5309-5376] | -0.09% | NOISE | 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 / 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 |

Effect beside the null control; an effect inside the control is a null,
whatever the per-run verdict says (that verdict's spread is one boot's):
  prefill   2048 conc 4:  ttft_ms effect -0.22% control -0.23%  tpot_ms effect +7.71% control +2.56%  tps_prefill effect -0.44% control +0.23%
  prefill  16384 conc 4:  ttft_ms effect +1.23% control -0.91%  tpot_ms effect -1.65% control -5.46%  tps_prefill effect -1.05% control +0.76%
  prefill  65536 conc 4:  ttft_ms effect +1.00% control +0.01%  tpot_ms effect -1.55% control -2.93%  tps_prefill effect -0.96% control -0.09%
streams: /home/arbi-admin/roh/runs/c4_base_r6/effect.sse /home/arbi-admin/roh/runs/c4_base_r6/control.sse  (stack left up; 'scripts/bench/console_build_ab.sh down' tears it down)
