    max_context            = auto
15:25:50 INFO    max_context=auto → resolved to model max_position_embeddings=262144 (the model's trained context ceiling).
15:27:32 INFO    TKV recent ring: sinks=4 w=32 slack=2 for 4 sequences (288 rows); 16 attention layer(s) read it through 16 ring layer(s); 9.16 MiB in scratch.attn_codec, 2.290 MiB per sequence.
15:27:57 INFO    TKV recent ring: sinks=4 w=32 slack=2 for 4 sequences (288 rows); 16 attention layer(s) read it through 16 ring layer(s); 9.16 MiB in scratch.attn_codec, 2.290 MiB per sequence.
15:27:58 INFO    dflash: loading draft model from /models/Qwen3.8-27B-DFlash2-exl3-6.0bpw (dtype=torch.bfloat16, rank=0)
15:31:50 INFO    drafter flat-cache dump written — 1.60 GB in 2.33s at /cache/arbi-serve/flat-weights/Qwen3.8-27B-DFlash2-exl3-6.0bpw.3c3fe724b73751b8.v3; later boots load the drafter from one sequential local read.
15:31:50 INFO    dflash: draft forward_block_fixed routed through torch.compile
15:31:50 INFO    dflash: draft-forward cudagraph capture armed (max_graphs=32 for max_batch=4)
15:31:50 INFO    dflash: block-realization capture armed (realizer=selector, greedy_walk=True, max_batch=4)
15:31:50 INFO    dflash: DFlash2 candidate selector declared — the block is realized by a left-to-right candidate re-rank
15:31:50 INFO    dflash drafter attached (rank=0, owner): block_size=8 (from --mtp-n-draft) max_k=7 rope=plain tap_layers=[5, 19, 33, 47, 61] draft_kv_effective_context=100480 (GDN rollback mode=replay, T=8 × max_batch=4; replay keeps only the per-token inputs and recurses from the state slab, recompute keeps a full base frame per layer and replays on the host — same tokens, materially slower, and it is what ARBI_ACCEPT_INVARIANT requires)
15:31:50 INFO    EXL3 kernel-shape pin ACTIVE at post-drafter: 450 of 450 bound linears pinned over 18 geometries (47 newly pinned here; 3 timed this boot, 15 restored from the pin cache); drafter DFlashDrafter.model: 47 linears, all pinned, raced over rows [1, 16, 32] (shipped default). 5120x10240xK4->K32N128[2, 5]@sms0[shipped]; 5120x6144xK4->K32N128[2, 5]@sms0[shipped]; 6144x5120xK4->K32N128[2, 5]@sms0[shipped]; 5120x17408xK4->K32N256[3]@sms0[shipped]; 17408x5120xK4->K32N128[2, 5]@sms0[shipped]; 5120x12288xK4->K32N256[3]@sms0[shipped]; 5120x1024xK4->K32N128[2, 5]@sms64[shipped]; 5120x248320xK6->K32N128[2, 5]@sms0[shipped]; 5120x131072xK6->K32N128[2, 5]@sms0[measured]; 5120x384xK6->K16N128[1, 7, 8, 9, 10, 11]@sms0[measured]; 5120x4096xK6->K32N128[2, 5]@sms0[shipped]; 5120x1024xK6->K32N128[2, 5]@sms64[shipped]; 4096x5120xK6->K32N128[2, 5]@sms0[shipped]; 5120x17408xK6->K32N128[2, 5]@sms0[shipped]; 17408x5120xK6->K32N128[2, 5]@sms0[shipped]; 5120x1280xK6->K32N128[2, 5]@sms80[shipped]; 25600x5120xK6->K32N128[2, 5]@sms0[shipped]; 5120x256xK6->K16N128[1, 7, 8, 9, 10, 11]@sms0[measured]. One reduction geometry per linear at every row count — decode, MTP verify and a short prefill tail agree bit-for-bit. The rows are what the race minimised over and are part of the key the answer is recorded under. A geometry printed with @sms<N> runs a RECORDED grid width; @sms0 is the kernel's own grid choice, which is what an unrecorded geometry keeps. A width is recorded only where it beat that choice on the verify slates AND did not cost the plain decode rows more than shipped_pins.json's declared decode_band_pct, so @sms<N> is not a slate win charged to decode: the cell's decode_cost_pct records which, and runs the opposite way to the verify_margin_pct beside it. Selection timings: arbi_serve.weight_quant.exl3.custom_op.kernel_shape_pins(); which of them this boot took: kernel_shape_pin_origins().
15:32:18 INFO    dflash: draft forward warmed at 4 capture shape(s) in 20.3s — {'async_compile.wait': 0.3, 'backend_compile': 3.0, 'entire_frame_compile': 15.9, 'CachingAutotuner.benchmark_all_configs': 3.5, 'total_wall_time': 15.9}
15:32:24 INFO    tkv verify-split table [codec d=256 q=24 kv=4 K4/V4]: {1: 32, 2: 32, 3: 32, 4: 32} in 0.0s (block_m=8, batches 1..4, max_context=262144)
15:32:27 INFO    compile-warmup ledger: 32.3s total, 26 dynamo frame compile(s) — decode-compile 2.7s/4f · dflash-draft-compile 20.3s/8f · megacache-save 1.0s/0f · prefill-multirow 2.4s/5f · prefill-single-row 2.3s/2f · tkv-cute-prewarm 0.2s/0f · decode-recurrent 0.9s/3f · verify-hidden 0.9s/2f · verify-mixed-k 1.7s/2f (step seconds/frames; a frame is a Dynamo trace, which the Inductor caches do NOT skip — they cover the backend only)
15:34:46 INFO    deferred KV: released the warmup KV slab — 68 MiB reserved in state.attn_kv (16 warmup pages, live 0). Reclaimed BEFORE free VRAM is measured, so it sizes into servable KV instead of staying a resident free list (empty_cache cannot return it: pytorch#145168).
15:34:46 INFO    pre-decode arena: reserved 520.0 MiB in scratch.attn_codec at deferred KV, before the KV pool is sized.
15:34:46 INFO    split-KV arena: reserved 48.2 MiB in scratch.attn_codec at deferred KV, before the KV pool is sized.
15:34:56 INFO    TKV recent ring: sinks=4 w=32 slack=2 for 4 sequences (288 rows); 16 attention layer(s) read it through 16 ring layer(s); 9.16 MiB in scratch.attn_codec, 2.290 MiB per sequence.
15:34:56 INFO    Boot: deferred KV pool resize — 11.8s
15:35:05 INFO    drafter penalty alignment ON (dflash, APPROXIMATE): the committed-prefix presence/frequency bias rides the drafter's own block proposal (4 rows x 248320 vocab, accumulator-backed). A block drafter realizes every position from one forward, so the <= K columns this step itself drafts stay un-penalised at positions > 0; that costs acceptance on those columns and nothing else — q is reported exactly as drawn.
15:35:05 INFO    drafter-chain capture skipped: DFlash drafter has no head chain
15:35:05 INFO    dflash pre-sweep: captured 4/4 draft graphs (pool sealed; graph pool now 72 MiB reserved)
15:35:05 INFO    dflash realize pre-sweep: captured 12/12 realization graphs (pool sealed; graph pool now 72 MiB reserved)
15:35:09 INFO    dflash observe: context-assemble span high-water 2048 rows (the serving floor's draft row is sized at the widest span the step limits allow; this is the widest one served so far).
15:35:09 INFO    dflash: a request has outgrown the draft-KV slab — its drafter now attends only the trailing 2056 positions its widest layer holds, not the whole context. Output stays correct (the target verifies every drafted token); accept rate falls off. Counted per row per step in path_counter 'dflash_draft_ctx_wrap'.
15:35:10 INFO    MTP verify-pass active: 3 opted-in row(s) this step; drafter=DFlashDrafter (accept rate depends on drafter quality)
15:35:14 INFO    dflash draft-step reserve: per-step context destination DROPPED — every draft step assembles into a captured graph (graph-assemble route proved for every batch width 1..4)
15:35:14 INFO    dflash tile sweep (2048 rows, 102400 B/row): 256:12.354ms+-0.236; 512:8.206ms+-0.015; 1024:6.658ms+-0.018; 2048:6.009ms+-0.007 -> tile 2048 (fastest 6.009 ms; the runner-up is 0.649 ms behind against a worst-rung resolution of 0.236 ms, and ties within a rung's own resolution take the smallest tile; 200 MiB of reserve, which is what the KV pool pays for this choice).
15:35:14 INFO    serving floor: DFlash draft row = 141 MiB — context-assemble 141 MiB [MEASURED] (rows=2048 (the row bound — the widest span a served observe can be handed, so the reserve covers every width the sweep could install and no timing reaches this number; serving runs at 2048 — MEASURED fastest of 4 tile widths on this card; the ring retains 2048 of the step's rows), L=5, nkv=8 (TP-local), hd=128, H=5120, taps=5x5120, dtype=torch.bfloat16, per-layer K/V projection; the loaded drafter's own peak at that shape (200 MiB closed-form)) vs draft step 15 MiB [MEASURED] (B=4 (max_batch), caps=(2056, 2056, 2056, 2056, 2056) (per-layer draft slab, the capture pre-sweep's top bucket), max_full_len=2056, dtype=torch.bfloat16, tp=1, route=graph-assemble (no per-step destination — proved for every batch width); the loaded drafter's own step peak at that shape (40 MiB closed-form)), larger wins.
15:35:14 INFO    serving grow floor: 316 MiB (base 4 [MEASURED] + arena-regrow 18 [MEASURED: admission-bound 424 MiB against the 582 MiB the pool already holds (serving mark UNMEASURED; the residency is the boot probes')] + max[spec-reserve 220 (=add(device_slots) of verify-tail 47 + verify-logits 32 [32 rows, measured 32, bound 31], dflash-draft 141), tkv-bypass-scratch 0, tkv-prefill-staging 9] + logprobs-tile 0 [SEEDED] + media-encode 0 [stochastic_on=True rejection_flag=True measured=47 analytic=227] + token2wav-vocoder 0 + driver-growth 0 [UNBUDGETED] + autotune-workspace 74 [MEASURED]) vs gmu_floor 0 MiB vs residency-gate unpaid-graph-exec 0 MiB (predicted 96 MiB for this configuration's capture ladder, 130 MiB already metered onto the card as driver.cudagraph_exec -> 0 MiB still unpaid; 0 member record(s)) vs serving-consumption 0 MiB [MEASURED] -> 316 MiB [tts-codec-peak 0 + tts-kv-growth 0 (5000 ticks)]
15:35:14 INFO    serving grow floor: 370 MiB (base 4 [MEASURED] + arena-regrow 72 [MEASURED: admission-bound 424 MiB against the 582 MiB the pool already holds (serving mark UNMEASURED; the residency is the boot probes')] + max[spec-reserve 220 (=add(device_slots) of verify-tail 47 + verify-logits 32 [32 rows, measured 32, bound 31], dflash-draft 141), tkv-bypass-scratch 0, tkv-prefill-staging 9] + logprobs-tile 0 [SEEDED] + media-encode 0 [stochastic_on=True rejection_flag=True measured=47 analytic=227] + token2wav-vocoder 0 + driver-growth 0 [UNBUDGETED] + autotune-workspace 74 [MEASURED]) vs gmu_floor 0 MiB vs residency-gate unpaid-graph-exec 0 MiB (predicted 96 MiB for this configuration's capture ladder, 130 MiB already metered onto the card as driver.cudagraph_exec -> 0 MiB still unpaid; 0 member record(s)) vs serving-consumption 76 MiB [MEASURED] -> 370 MiB [tts-codec-peak 0 + tts-kv-growth 0 (5000 ticks)]
15:35:19 INFO    serving grow floor: 334 MiB (base 4 [MEASURED] + arena-regrow 36 [MEASURED: admission-bound 424 MiB against the 582 MiB the pool already holds (serving mark UNMEASURED; the residency is the boot probes')] + max[spec-reserve 220 (=add(device_slots) of verify-tail 47 + verify-logits 32 [32 rows, measured 32, bound 31], dflash-draft 141), tkv-bypass-scratch 0, tkv-prefill-staging 9] + logprobs-tile 0 [SEEDED] + media-encode 0 [stochastic_on=True rejection_flag=True measured=47 analytic=227] + token2wav-vocoder 0 + driver-growth 0 [UNBUDGETED] + autotune-workspace 74 [MEASURED]) vs gmu_floor 0 MiB vs residency-gate unpaid-graph-exec 0 MiB (predicted 96 MiB for this configuration's capture ladder, 130 MiB already metered onto the card as driver.cudagraph_exec -> 0 MiB still unpaid; 0 member record(s)) vs serving-consumption 40 MiB [MEASURED] -> 334 MiB [tts-codec-peak 0 + tts-kv-growth 0 (5000 ticks)]
15:35:23 INFO    serving grow floor: 334 MiB (base 4 [MEASURED] + arena-regrow 36 [MEASURED: admission-bound 424 MiB against the 428 MiB the pool already holds (serving mark UNMEASURED; the residency is the boot probes')] + max[spec-reserve 220 (=add(device_slots) of verify-tail 47 + verify-logits 32 [32 rows, measured 32, bound 31], dflash-draft 141), tkv-bypass-scratch 0, tkv-prefill-staging 9] + logprobs-tile 0 [SEEDED] + media-encode 0 [stochastic_on=True rejection_flag=True measured=47 analytic=227] + token2wav-vocoder 0 + driver-growth 0 [UNBUDGETED] + autotune-workspace 74 [MEASURED]) vs gmu_floor 0 MiB vs residency-gate unpaid-graph-exec 0 MiB (predicted 96 MiB for this configuration's capture ladder, 130 MiB already metered onto the card as driver.cudagraph_exec -> 0 MiB still unpaid; 0 member record(s)) vs serving-consumption 40 MiB [MEASURED] -> 334 MiB [tts-codec-peak 0 + tts-kv-growth 0 (5000 ticks)]
15:35:23 INFO    served context: 262144 tokens — the realized KV pool (1099 servable pages, prefix 7, 0-page margin) confirms the sizing bound.
15:35:23 INFO    cudaGraphInstantiate MEASURED: decode=13.25 MiB/graph x8, dflash=1.50 MiB/graph x16 (24 exec(s), 130.0 MiB driver total) — persisting; retires the modelled per-graph constant.
15:35:23 INFO    graph_pool budget cache: persisted 1056964608 B (1008.0 MiB) + measured cudaGraphInstantiate decode=13.25 MiB/graph, dflash=1.50 MiB/graph → /cache/arbi-serve/budget-cache/13f61cee3c249c99.json
15:35:23 INFO    serving grow floor: 334 MiB (base 4 [MEASURED] + arena-regrow 36 [MEASURED: admission-bound 424 MiB against the 428 MiB the pool already holds (serving mark UNMEASURED; the residency is the boot probes')] + max[spec-reserve 220 (=add(device_slots) of verify-tail 47 + verify-logits 32 [32 rows, measured 32, bound 31], dflash-draft 141), tkv-bypass-scratch 0, tkv-prefill-staging 9] + logprobs-tile 0 [SEEDED] + media-encode 0 [stochastic_on=True rejection_flag=True measured=47 analytic=227] + token2wav-vocoder 0 + driver-growth 0 [UNBUDGETED] + autotune-workspace 74 [MEASURED]) vs gmu_floor 0 MiB vs residency-gate unpaid-graph-exec 0 MiB (predicted 130 MiB for this configuration's capture ladder, 130 MiB already metered onto the card as driver.cudagraph_exec -> 0 MiB still unpaid; 0 member record(s)) vs serving-consumption 40 MiB [MEASURED] -> 334 MiB [tts-codec-peak 0 + tts-kv-growth 0 (5000 ticks)]
15:35:23 INFO    Boot: KV cache ready at 4.56 GiB (19% of 23.52 GiB) — 1099 pages × 256 tok/page = 281344 servable tokens, free 0.36 GiB (2% of 23.52 GiB)
15:35:23 INFO    VRAM LEDGER — every row SUMS TO THE CARD (no subtraction needed) and declares HOW its number was obtained; '!' marks a row that is not a measurement of this boot's own bytes.  card 23.988 GiB = driver-reserved 481 MiB + CUDA-visible 23.518 GiB
  ── state.*          5.266 GiB  22.4%  PER-SEQUENCE — SCALES WITH CONCURRENCY × CONTEXT (the knobs: --max-batch, --max-context)
  driver.modules_loaded is measured per boot phase: model weights loaded +82 MiB, KV profile + active state built (minimal warmup pool) +24 MiB, decode-kernel autotune/compile +2 MiB, torch.compile + autotune warmup +10 MiB, serving-kernel prewarm +4 MiB, capture: decode graphs +220 MiB, capture: dflash draft graphs +26 MiB, KV grow (post-capture) +38 MiB
  NOTE: unpooled.torch_default_pool holds 7M of live tensors (FoldEmitStaging._group_buf 3M, LinearEXL3.trellis 1M, DraftReach._tables['cuda:0'] 1M, DFlashVarlenPlan._t, DFlashVarlenPlan.src_row, DFlashVarlenPlan.src_col, DFlashVarlenPlan._d, DFlashVarlenPlan._cu_row, DFlashVarlenPlan._src_flat 0M) and 29M of relocatable slack — only the slack can be repacked; the tensors are freed only by removing or shrinking them. Naming every storage took 1408 ms of this boot.
15:35:24 INFO    Boot: KV cache ready at 4.56 GiB (19% of 23.52 GiB) — 1099 pages × 256 tok/page = 281344 servable tokens, free 0.36 GiB (2% of 23.52 GiB)
15:35:28 INFO    compiled-entry census at ready: rank 0 holds DFlashDraftModel.forward_block_fixed=8, _Qwen3_5AttentionDecoderLayer._post_attn=4, _Qwen3_5AttentionDecoderLayer._pre_attn=4, _Qwen3_5GdnDecoderLayer.forward=19. A piece at 1 entry is warm for ONE shape class only; the shape a request brings that misses is refused by name in jit_recompile_refused_serving and runs eager from there on.
15:35:33 WARNING driver serving-growth OVER RESERVE: the driver holds 6 MiB more than it did when the boot brackets closed, against the 0 MiB the serving floor's transient.serving_step.driver_growth term held for it. Those bytes are in no pool and no allocator counter -- the live card reaches them as driver.modules_serving and the boot ledger only as driver.residual -- and the KV layout is frozen, so the overage comes out of the free VRAM the floor is holding for one in-flight step (the verify tail, the DFlash context-assemble) and those reserves are the ones that fail first. The reading is on record for this configuration, so the next boot holds it; this process serves the rest of its life short by the overage.
