effect: off (67b5048d8b51) -> on (97fa8425f080)  errors=[]
| prefill | conc | TTFT ms A | TTFT ms B | dTTFT | verdict | prefill tok/s A | prefill tok/s B | dprefill | verdict | groupings A / B |
|---:|---:|---:|---:|---:|---|---:|---:|---:|---|---|
| 2048 | 4 | 331.1±0.8 | 332.6±1.1 | +1.10% | NOISE | 5986 [5330-6142] | 5777 [5066-6115] | -1.11% | NOISE | 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 / 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 |
| 16384 | 4 | 1848.4±0.3 | 1852.1±0.4 | +0.23% | NOISE | 5905 [5863-5929] | 5897 [5862-5929] | -0.14% | NOISE | 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 / 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 |
| 65536 | 4 | 7113.4±0.1 | 7121.3±0.2 | +0.11% | NOISE | 5754 [5745-5766] | 5747 [5734-5760] | -0.11% | NOISE | 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 / 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 |

control: on (97fa8425f080) -> null (d23b3c07579c)  errors=[]
| prefill | conc | TTFT ms A | TTFT ms B | dTTFT | verdict | prefill tok/s A | prefill tok/s B | dprefill | verdict | groupings A / B |
|---:|---:|---:|---:|---:|---|---:|---:|---:|---|---|
| 2048 | 4 | 331.9±0.7 | 325.9±0.6 | -0.77% | NOISE | 5784 [5218-6099] | 5702 [5331-6116] | +0.78% | NOISE | 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 / 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 |
| 16384 | 4 | 1858.5±0.2 | 1849.8±0.1 | -0.31% | NOISE | 5878 [5848-5896] | 5904 [5869-5919] | +0.19% | NOISE | 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 / 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 |
| 65536 | 4 | 7131.3±0.1 | 7114.5±0.1 | -0.24% | WIN | 5741 [5729-5754] | 5752 [5741-5761] | +0.18% | NOISE | 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 / 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 |

Effect beside the null control; an effect inside the control is a null,
whatever the per-run verdict says (that verdict's spread is one boot's):
  prefill   2048 conc 4:  ttft_ms effect +1.10% control -0.77%  tpot_ms effect +1.70% control +5.57%  tps_prefill effect -1.11% control +0.78%
  prefill  16384 conc 4:  ttft_ms effect +0.23% control -0.31%  tpot_ms effect +4.31% control +3.45%  tps_prefill effect -0.14% control +0.19%
  prefill  65536 conc 4:  ttft_ms effect +0.11% control -0.24%  tpot_ms effect -3.03% control -0.14%  tps_prefill effect -0.11% control +0.18%
streams: /home/arbi-admin/roh/runs/c4_oh_r6/effect.sse /home/arbi-admin/roh/runs/c4_oh_r6/control.sse  (stack left up; 'scripts/bench/console_build_ab.sh down' tears it down)
