effect: off (67b5048d8b51) -> on (97fa8425f080)  errors=[]
| prefill | conc | TTFT ms A | TTFT ms B | dTTFT | verdict | prefill tok/s A | prefill tok/s B | dprefill | verdict | groupings A / B |
|---:|---:|---:|---:|---:|---|---:|---:|---:|---|---|
| 2048 | 4 | 324.4±2.4 | 320.8±1.1 | +2.06% | NOISE | 5774 [5358-5964] | 5522 [5242-5961] | -0.46% | NOISE | 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x2,1x2,1x2,1x2 1x4,1x4,1x4,1x4 / 1x3,1x3,1x3,1x1 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x3,1x3,1x3,1x1 1x2,1x2,1x2,1x2 1x4,1x4,1x4,1x4 |
| 16384 | 4 | 1894.0±0.1 | 1895.4±0.1 | +0.07% | NOISE | 5768 [5745-5783] | 5763 [5754-5774] | -0.07% | NOISE | 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 / 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 3x2,3x2,3x2,2x1 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 |
| 65536 | 4 | 7311.7±0.1 | 7309.4±0.0 | -0.03% | NOISE | 5598 [5585-5608] | 5600 [5596-5604] | +0.04% | NOISE | 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 / 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 |

control: on (97fa8425f080) -> null (d23b3c07579c)  errors=[]
| prefill | conc | TTFT ms A | TTFT ms B | dTTFT | verdict | prefill tok/s A | prefill tok/s B | dprefill | verdict | groupings A / B |
|---:|---:|---:|---:|---:|---|---:|---:|---:|---|---|
| 2048 | 4 | 343.7±0.3 | 343.4±0.1 | -0.10% | NOISE | 5951 [5924-5966] | 5839 [5260-5965] | +0.06% | NOISE | 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 / 1x3,1x3,1x3,1x1 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 1x4,1x4,1x4,1x4 |
| 16384 | 4 | 1898.0±0.1 | 1900.0±0.0 | +0.11% | NOISE | 5758 [5751-5763] | 5754 [5748-5769] | -0.12% | LOSS | 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 / 3x2,3x2,3x2,2x1 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 4x2,3x2,3x2,2x2 |
| 65536 | 4 | 7304.4±0.1 | 7314.3±0.1 | +0.14% | NOISE | 5606 [5598-5611] | 5594 [5590-5599] | -0.20% | LOSS | 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 / 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 10x2,9x2,9x2,8x2 |

Effect beside the null control; an effect inside the control is a null,
whatever the per-run verdict says (that verdict's spread is one boot's):
  prefill   2048 conc 4:  ttft_ms effect +2.06% control -0.10%  tpot_ms effect -3.63% control +2.01%  tps_prefill effect -0.46% control +0.06%
  prefill  16384 conc 4:  ttft_ms effect +0.07% control +0.11%  tpot_ms effect +2.54% control -0.88%  tps_prefill effect -0.07% control -0.12%
  prefill  65536 conc 4:  ttft_ms effect -0.03% control +0.14%  tpot_ms effect +0.41% control -0.05%  tps_prefill effect +0.04% control -0.20%
streams: /home/arbi-admin/roh/runs/c4_oh_r6/effect.sse /home/arbi-admin/roh/runs/c4_oh_r6/control.sse  (stack left up; 'scripts/bench/console_build_ab.sh down' tears it down)
