# t12 — hand on Thor: STILL BLOCKED, #181 re-attributed (boot failure gone, inference broken)
#
# Plan: docs/plans/2026-08-20-nemotron-lightning-worker.md, task t12 (covers c20/h9, Thor half).
# Box: Jetson AGX Thor (sm_110). Date: 2026-08-20. Engine-direct probes (Orin template),
# LiquidAI/LFM2.5-1.2B-Instruct, util 0.06 @ 32768, --enable-lora --max-loras 4
# --max-lora-rank 32 --enable-auto-tool-choice --tool-call-parser lfm2.
# Faithful negative result — reported blocked-with-attribution, not skipped.

## Run A — NEW fleet nightly (8bd082, vLLM 0.26.1rc1.dev942)

BOOT: HEALTHY — including LoRA init. #181's original signature (LoRA
embedding-slot allocation failure at boot) does NOT reproduce on this engine.
  GPU KV cache size: 292,963 tokens, max concurrency for 32,768/request: 8.94x

INFERENCE: BROKEN.
  known-answer ("capital of France?"): replied unrelated French chat filler —
  wrong answer, no crash.
  tool-call probe: 500 → engine death:
    torch.AcceleratorError: CUDA error: unspecified launch failure
  Container exited (restart=no on this probe run).

## Run B — counterfactual on the OLD production engine (7c5a10e9, vLLM 0.23.1rc1.dev672)

BOOT: HEALTHY (LoRA init fine here too).
INFERENCE: BROKEN differently.
  known-answer at temperature=0: replied unrelated text ("The 2017–18 Cypriot
  First Division …") — deterministic sampling, garbage topic.
  tool-call probe: structured tool_calls DID parse correctly once
  ({"name": "get_weather", "arguments": {"city": "Paris"}}, finish=tool_calls).
  third request (deterministic "2+2"): 500 EngineCore death.

## Control

The IDENTICAL config (same image 8bd082, same flags, same probes) passed
cleanly on the DGX Spark GB10 the same day: known-answer "Paris", structured
tool_calls, no crash (docs/evidence/2026-08-20-accept-hand-spark.txt).

## Re-attribution

#181 as filed ("LoRA embedding-slot allocation failure at boot, cause
unattributed") is RETIRED: boot + LoRA allocation now succeed on both engines.
The real blocker is sm_110-specific inference corruption/instability for the
LFM2 conv-hybrid architecture: wrong deterministic generations on BOTH vLLM
builds, escalating to CUDA unspecified-launch-failure / EngineCore death within
1-3 requests. Same-day pattern evidence: the Qwen3.6-35B GDN MTP decode kernel
on the SAME 8bd082 digest fails with "no kernel image is available" on sm_110
(docs/evidence/2026-08-20-baseline-worker-qwen35b-thor.txt) — the digest's csrc
coverage for non-standard (conv/Mamba/GDN) decode paths on sm_110 is deficient,
while dense-transformer paths serve fine.

CONSEQUENCE: hand stays UNSERVED on Thor (feasible=false remains truthful);
#181 should be re-titled/commented to the inference-level attribution above and
tracked against upstream vLLM sm_110 kernel coverage. The #183 matrix after
today: Orin VALIDATED, Spark VALIDATED (same day), Thor BLOCKED (re-attributed),
base untestable; adapter serving still blocked on unsloth-cli#16.
