Metadata-Version: 2.4
Name: narwhal-inference
Version: 0.8.0
Summary: Adaptive hot-swap disaggregation for LLM inference
Author: Athrael Soju
License-Expression: Apache-2.0
Project-URL: Documentation, https://athrael-soju.github.io/Narwhal/
Project-URL: Repository, https://github.com/athrael-soju/Narwhal
Project-URL: Issues, https://github.com/athrael-soju/Narwhal/issues
Project-URL: Changelog, https://github.com/athrael-soju/Narwhal/blob/main/CHANGELOG.md
Project-URL: Release notes, https://github.com/athrael-soju/Narwhal/releases
Keywords: llm,inference,llm-inference,llm-serving,disaggregation,disaggregated-inference,disaggregated-serving,prefill,decode,scheduling,vllm,kv-cache,nixl,gpu,cuda,nvidia,rocm,amd
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: GPU :: NVIDIA CUDA
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: fastapi>=0.136.3
Requires-Dist: httptools>=0.8.0
Requires-Dist: httpx>=0.28.1
Requires-Dist: msgpack>=1.2.2
Requires-Dist: pydantic>=2.13.5
Requires-Dist: pyzmq>=27.2.0
Requires-Dist: uvicorn>=0.52.4
Requires-Dist: uvloop>=0.22.1; sys_platform != "win32"
Provides-Extra: dev
Requires-Dist: coverage==7.16.1; extra == "dev"
Requires-Dist: ruff==0.16.7; extra == "dev"
Requires-Dist: mypy==2.3.1; extra == "dev"
Requires-Dist: pre-commit>=4; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs-material==9.7.7; extra == "docs"
Requires-Dist: mkdocs-glightbox==0.5.2; extra == "docs"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/athrael-soju/Narwhal/main/docs/assets/social-preview.png" alt="The Narwhal logo, a black narwhal with a teal spiral tusk above the wordmark" width="100%">
</p>

<p align="center">
  <img src="https://img.shields.io/badge/license-Apache--2.0-blue" alt="Apache-2.0 license">
  <img src="https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue" alt="Python 3.11 through 3.13">
  <img src="https://img.shields.io/badge/style-ruff-261230" alt="Lint and format by ruff">
  <img src="https://img.shields.io/badge/types-mypy-blue" alt="Types checked with mypy">
  <a href="https://pypi.org/project/narwhal-inference/"><img src="https://img.shields.io/pypi/v/narwhal-inference" alt="Latest PyPI version"></a>
</p>

<p align="center">
  <a href="https://athrael-soju.github.io/Narwhal/"><img src="https://img.shields.io/badge/Documentation-0f766e" alt="Documentation"></a>
  <a href="https://athrael-soju.github.io/Narwhal/Deploy/"><img src="https://img.shields.io/badge/Deployment-0f766e" alt="Deployment"></a>
  <a href="https://athrael-soju.github.io/Narwhal/HTTP-API/"><img src="https://img.shields.io/badge/API%20reference-0f766e" alt="API reference"></a>
  <a href="https://github.com/athrael-soju/Narwhal/issues"><img src="https://img.shields.io/badge/Issues-0f766e" alt="Issues"></a>
  <a href="https://github.com/athrael-soju/Narwhal/blob/main/CONTRIBUTING.md"><img src="https://img.shields.io/badge/Contributing-0f766e" alt="Contributing"></a>
</p>

## What is Narwhal?

Narwhal is a disaggregated inference framework that automatically hot-swaps prefill and decode roles as demand changes, without reloading model weights. It runs on a [single GPU](https://athrael-soju.github.io/Narwhal/Dev-Runtime/) or across [multi-node deployments](https://athrael-soju.github.io/Narwhal/Deploy/).

<table width="100%" align="center">
  <thead>
    <tr>
      <th align="left" width="18%">Capability</th>
      <th align="left" width="52%">Behavior</th>
      <th align="left" width="30%">Guide</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Role hot-swap</td>
      <td>Reassigns prefill and decode roles across a fixed GPU fleet, with NIXL key-value (KV) transfer between them</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/Core-Concepts/"><img src="https://img.shields.io/badge/docs-Core%20concepts-0f766e" alt="Core concepts documentation"></a></td>
    </tr>
    <tr>
      <td>Serving</td>
      <td>Serves completion and chat requests, streamed or buffered, with latency-aware admission</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/HTTP-API/"><img src="https://img.shields.io/badge/docs-HTTP%20API%20reference-0f766e" alt="HTTP API reference documentation"></a></td>
    </tr>
    <tr>
      <td>Fault tolerance</td>
      <td>Fails over to a warm-standby router and readmits engines against their live process generation</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/Operate/"><img src="https://img.shields.io/badge/docs-Operating%20Narwhal-0f766e" alt="Operating Narwhal documentation"></a></td>
    </tr>
    <tr>
      <td>Measurement</td>
      <td>Profiles engines, runs ordered benchmark points, and keeps the evidence from each point</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/Measure/"><img src="https://img.shields.io/badge/docs-Measuring%20a%20fleet-0f766e" alt="Measuring a fleet documentation"></a></td>
    </tr>
    <tr>
      <td>Observability</td>
      <td>Exports router and engine metrics to Prometheus and a provisioned Grafana dashboard</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/Observability/"><img src="https://img.shields.io/badge/docs-Setting%20up%20observability-0f766e" alt="Setting up observability documentation"></a></td>
    </tr>
    <tr>
      <td>Operator tooling</td>
      <td>Validates fleet files offline and collects private diagnostic bundles</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/CLI-Reference/"><img src="https://img.shields.io/badge/docs-CLI%20reference-0f766e" alt="CLI reference documentation"></a></td>
    </tr>
    <tr>
      <td>Development mode</td>
      <td>Runs two to eight engines on one NVIDIA CUDA GPU under Ubuntu or WSL2</td>
      <td><a href="https://athrael-soju.github.io/Narwhal/Dev-Runtime/"><img src="https://img.shields.io/badge/docs-Narwhal%20dev-0f766e" alt="Narwhal dev documentation"></a></td>
    </tr>
  </tbody>
</table>

## How engines change roles

The role controller scores the current role split and each adjacent split, one engine move away. It uses measured engine profiles, offered demand, and resident work, and moves one engine only when the worst projected SLO ratio improves and every role, lifecycle, and resident-work guard passes. New requests follow the revised split, and resident requests finish on their assigned engines.

<p align="center">
  <a href="https://athrael-soju.github.io/Narwhal/concepts/02-Role-Control/"><img src="https://img.shields.io/badge/docs-Role%20control%20and%20capacity%20floors-0f766e" alt="Role control and capacity floors documentation"></a>
</p>

![Role controller changing engine roles with weights resident.](https://raw.githubusercontent.com/athrael-soju/Narwhal/main/docs/assets/architectures/hotswap.svg)

## Evaluation

<h3 align="center"><a href="https://athrael.net/posts/evaluating-narwhal/">Read the full evaluation: Evaluating Narwhal</a></h3>

## Getting the commands

Install on Linux with Python 3.11 or newer:

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install narwhal-inference
narwhal-serve --version
narwhal --help
```

The wheel installs `narwhal`, `narwhal-engine`, `narwhal-serve`, `narwhal-attest`, `narwhal-profile`, and `narwhal-check`. To verify the installation, follow [Installing from PyPI](https://athrael-soju.github.io/Narwhal/Install-from-PyPI/). For each command's options, use the [CLI reference](https://athrael-soju.github.io/Narwhal/CLI-Reference/).

## Trying it on one GPU

Narwhal dev runs a local NVIDIA CUDA fleet on Ubuntu, either directly or under WSL2.

```bash
narwhal dev init
narwhal dev up
narwhal dev verify
narwhal dev status
narwhal dev down
```

The default template starts two vLLM engines and the SGLang template starts two SGLang engines, each on an NVIDIA GPU whose free VRAM is at least 80% of its total plus 512 MiB. The RTX 5090 reference template starts four vLLM engines on an RTX 5090.

<p align="center">
  <a href="https://athrael-soju.github.io/Narwhal/Dev-Runtime/"><img src="https://img.shields.io/badge/docs-Installed%20template-0f766e" alt="Installed template documentation"></a>
  <a href="https://athrael-soju.github.io/Narwhal/dev/RTX-5090-Reference/"><img src="https://img.shields.io/badge/docs-RTX%205090%20reference-0f766e" alt="RTX 5090 reference documentation"></a>
</p>

## Bringing up a fleet

Run the deployment gates in [Deploying a fleet](https://athrael-soju.github.io/Narwhal/Deploy/) from a management workstation.

## Documentation

The [documentation site](https://athrael-soju.github.io/Narwhal/) covers installation, deployment, operation, measurement, and reference tasks.

## Contributing

<p align="center">
  <a href="https://github.com/athrael-soju/Narwhal/blob/main/CONTRIBUTING.md"><img src="https://img.shields.io/badge/community-Contributing-0f766e" alt="Contributing"></a>
  <a href="https://github.com/athrael-soju/Narwhal/blob/main/CODE_OF_CONDUCT.md"><img src="https://img.shields.io/badge/community-Code%20of%20conduct-0f766e" alt="Code of conduct"></a>
  <a href="https://github.com/athrael-soju/Narwhal/blob/main/SECURITY.md"><img src="https://img.shields.io/badge/community-Security%20policy-0f766e" alt="Security policy"></a>
</p>

## Built on Arrow

Narwhal's scheduling algorithms are based on Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture by Wu et al. (2025).

<p align="center">
  <a href="https://arxiv.org/abs/2505.11916"><img src="https://img.shields.io/badge/paper-Arrow%20%C2%B7%20arXiv%202505.11916-0f766e" alt="Arrow paper on arXiv"></a>
  <a href="https://github.com/athrael-soju/Narwhal/blob/main/CITATION.cff"><img src="https://img.shields.io/badge/cite-CITATION.cff-0f766e" alt="Citation metadata for Arrow and Narwhal"></a>
  <a href="https://github.com/athrael-soju/Narwhal/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-0f766e" alt="Apache-2.0 license"></a>
</p>

# Changelog

## [0.8.0](https://github.com/athrael-soju/Narwhal/compare/v0.7.0...v0.8.0) (2026-10-20)


### Features

* operator visibility and setup ([#384](https://github.com/athrael-soju/Narwhal/issues/384)) ([3261c0d](https://github.com/athrael-soju/Narwhal/commit/3261c0db09242499ab57ec75108a4ddf8f00737e))
* reuse still-valid calibrations and profiles ([#373](https://github.com/athrael-soju/Narwhal/issues/373)) ([0181298](https://github.com/athrael-soju/Narwhal/commit/0181298dae294b8eb9ffddd500a4e516564bb023))
* take prefill capacity back on load surges and fix decode pressure and seat placement ([#364](https://github.com/athrael-soju/Narwhal/issues/364)) ([d4ee7b0](https://github.com/athrael-soju/Narwhal/commit/d4ee7b0d0443fb7ecca4f2ff4348983056535aaa))


### Fixes

* ship characterization golden records in the sdist ([#371](https://github.com/athrael-soju/Narwhal/issues/371)) ([5a0ae2e](https://github.com/athrael-soju/Narwhal/commit/5a0ae2e95f4dbb0479f22b8b301c2c68741c2a9a))
* ship docs walkthroughs in the sdist and widen stage test budgets ([#369](https://github.com/athrael-soju/Narwhal/issues/369)) ([4c08716](https://github.com/athrael-soju/Narwhal/commit/4c087164d10f8e39add958c9ac79e48d7ed3aad9))

## [0.7.0](https://github.com/athrael-soju/Narwhal/compare/v0.6.0...v0.7.0) (2026-10-12)


### Features

* SGLang engine backend and engine-neutral backend interfaces ([#336](https://github.com/athrael-soju/Narwhal/issues/336)) ([c7ccf28](https://github.com/athrael-soju/Narwhal/commit/c7ccf2852f207e00b94c1acea2a30ff42307e7e9))


### Documentation

* fit fleet walkthrough to viewport ([#330](https://github.com/athrael-soju/Narwhal/issues/330)) ([0457952](https://github.com/athrael-soju/Narwhal/commit/0457952c34c956489b475b89394417d6bdcae130))
* replace dashboard screenshots with a walkthrough ([#333](https://github.com/athrael-soju/Narwhal/issues/333)) ([055f547](https://github.com/athrael-soju/Narwhal/commit/055f547fc565c7c807fac9669985cb5ff27e24b5))
* trim duplicated documentation ([#325](https://github.com/athrael-soju/Narwhal/issues/325)) ([cdeadb8](https://github.com/athrael-soju/Narwhal/commit/cdeadb8ebc9ca38126e5d3e88426680b02b7e438))

## [0.6.0](https://github.com/athrael-soju/Narwhal/compare/v0.5.0...v0.6.0) (2026-10-10)


### Features

* fleet control service ([#301](https://github.com/athrael-soju/Narwhal/issues/301)) ([f1f1164](https://github.com/athrael-soju/Narwhal/commit/f1f116473b9ff4c9e92a6a3bfbd23c699dfaf6a6))
* qualified admission, queueing and retries ([#312](https://github.com/athrael-soju/Narwhal/issues/312)) ([05df12f](https://github.com/athrael-soju/Narwhal/commit/05df12f02ed113a2d661a2d8571fdb56557aa347))
* redesign the fleet control console as a standalone page linked from Grafana ([#313](https://github.com/athrael-soju/Narwhal/issues/313)) ([4d7be2e](https://github.com/athrael-soju/Narwhal/commit/4d7be2e32c6523c4a9b6ab10470fcd017b5053d7))


### Documentation

* align the docs index with the README ([#284](https://github.com/athrael-soju/Narwhal/issues/284)) ([00a288e](https://github.com/athrael-soju/Narwhal/commit/00a288e901062182c7aa1cd6e14dc34111b29233))
* remove the Recognition page ([#311](https://github.com/athrael-soju/Narwhal/issues/311)) ([690ad5e](https://github.com/athrael-soju/Narwhal/commit/690ad5ee9adf0ff3e642726770793b58566b7cd8))

## [0.5.0](https://github.com/athrael-soju/Narwhal/compare/v0.4.1...v0.5.0) (2026-10-05)


### ⚠ BREAKING CHANGES

* engine-role environments drop NARWHAL_ENGINE_LAUNCHER and NARWHAL_ENGINE_LAUNCHER_SHA256; run engine actions with narwhal-engine. The attestation contract tool runs as python -m narwhal.deployment.attestation_contract.cli.
* concurrent first-token calibration with measured and reused reporting ([#242](https://github.com/athrael-soju/Narwhal/issues/242))

* split oversized modules by concern ([#252](https://github.com/athrael-soju/Narwhal/issues/252)) ([e29cadc](https://github.com/athrael-soju/Narwhal/commit/e29cadc484cdc09a81eb4522361771acb8f735e3)), closes [#239](https://github.com/athrael-soju/Narwhal/issues/239)


### Features

* colour dashboard series by meaning ([#270](https://github.com/athrael-soju/Narwhal/issues/270)) ([9212382](https://github.com/athrael-soju/Narwhal/commit/9212382d703c3d6590f8bc5a1ad06cfe9c671ba2))
* concurrent first-token calibration with measured and reused reporting ([#242](https://github.com/athrael-soju/Narwhal/issues/242)) ([683b83b](https://github.com/athrael-soju/Narwhal/commit/683b83baaf29e0cb41207f415b57fc92ae488964))
* enhance reactive policy to manage unsettled evaluations and departure leads ([#251](https://github.com/athrael-soju/Narwhal/issues/251)) ([4712899](https://github.com/athrael-soju/Narwhal/commit/47128997188aada12829831c81e360af2f605eb2))
* faster role changes and slot-aware decode admission ([#234](https://github.com/athrael-soju/Narwhal/issues/234)) ([55f7854](https://github.com/athrael-soju/Narwhal/commit/55f78541ca2766e7969641be5ddd8051ad2eedea))
* make AIPerf 0.13.0 the documented benchmark client ([#280](https://github.com/athrael-soju/Narwhal/issues/280)) ([cadcc87](https://github.com/athrael-soju/Narwhal/commit/cadcc8730cc341069d54b556f1432d7951ddc161)), closes [#271](https://github.com/athrael-soju/Narwhal/issues/271)
* router data-plane throughput ([#267](https://github.com/athrael-soju/Narwhal/issues/267)) ([1be680b](https://github.com/athrael-soju/Narwhal/commit/1be680bc4bf357afcbb5f40a77f3c9b6d2ad521d))


### Fixes

* check idle engine connections with poll() ([#281](https://github.com/athrael-soju/Narwhal/issues/281)) ([714adb7](https://github.com/athrael-soju/Narwhal/commit/714adb74a4c9c689ab2bc1ec4e375cdc09777f6a))
* keep a departure leading through one evaluation below the margin ([#250](https://github.com/athrael-soju/Narwhal/issues/250)) ([551fa2f](https://github.com/athrael-soju/Narwhal/commit/551fa2f0070f80a8ca6087dc491d2b08b1692e33))
* pin Narwhal dev to vLLM 0.30.0 ([#269](https://github.com/athrael-soju/Narwhal/issues/269)) ([25f5383](https://github.com/athrael-soju/Narwhal/commit/25f5383cb6f57059e96b0ea63a1c83ae6db99771))
* use one red across the dashboard ([#279](https://github.com/athrael-soju/Narwhal/issues/279)) ([37b5f5b](https://github.com/athrael-soju/Narwhal/commit/37b5f5b6f9f1cc53669fbbbeaa693d1cae84eba8))


### Documentation

* document resetting engine prefix caches ([#283](https://github.com/athrael-soju/Narwhal/issues/283)) ([01cc58d](https://github.com/athrael-soju/Narwhal/commit/01cc58d9f64817d069e63ccfa52bde2ec933489d))

## [0.4.1](https://github.com/athrael-soju/Narwhal/compare/v0.4.0...v0.4.1) (2026-10-02)


* release 0.4.1 ([#237](https://github.com/athrael-soju/Narwhal/issues/237)) ([977a251](https://github.com/athrael-soju/Narwhal/commit/977a2519c801f39526266855d6d7be9ac24c8ef7))


### Features

* give dashboard series one colour palette ([#235](https://github.com/athrael-soju/Narwhal/issues/235)) ([c160ad5](https://github.com/athrael-soju/Narwhal/commit/c160ad56f4ba9df3fbba6084855b468450e930f5))

## [0.4.0](https://github.com/athrael-soju/Narwhal/compare/v0.3.2...v0.4.0) (2026-10-01)


### Features

* cache-aware placement ([#227](https://github.com/athrael-soju/Narwhal/issues/227)) ([d256647](https://github.com/athrael-soju/Narwhal/commit/d256647a116ade0a1a26f3e0980df184a42a4313))

## [0.3.2](https://github.com/athrael-soju/Narwhal/compare/v0.3.1...v0.3.2) (2026-09-30)


### Documentation

* add Google Search Console verification file ([#225](https://github.com/athrael-soju/Narwhal/issues/225)) ([ded08a8](https://github.com/athrael-soju/Narwhal/commit/ded08a869919c198614f45a0fd8e556ad0ee6455))
* define milestone delivery and deployment privacy rules ([#202](https://github.com/athrael-soju/Narwhal/issues/202)) ([6255337](https://github.com/athrael-soju/Narwhal/commit/6255337074aaab68fe9fa2c34da12810e3696c4f))
* Refresh ([#223](https://github.com/athrael-soju/Narwhal/issues/223)) ([f71a178](https://github.com/athrael-soju/Narwhal/commit/f71a178c2d3f303de198e33bd0ec4de2384881b5))
* refresh README introduction and link benchmark write-up ([#217](https://github.com/athrael-soju/Narwhal/issues/217)) ([c10c3ae](https://github.com/athrael-soju/Narwhal/commit/c10c3ae1cb9e781dcbafd11cbb3af9210d843e9b))
* split Narwhal dev guide into numbered pages ([#222](https://github.com/athrael-soju/Narwhal/issues/222)) ([5f2d3fa](https://github.com/athrael-soju/Narwhal/commit/5f2d3fa4a1d98db1bd541e12e756981032a51e02))
* tighten contributor and issue-form prose ([#210](https://github.com/athrael-soju/Narwhal/issues/210)) ([8d85ed8](https://github.com/athrael-soju/Narwhal/commit/8d85ed83f6c8559d8e5f46e25111bcfafdbd7f73))

## [0.3.1](https://github.com/athrael-soju/Narwhal/compare/v0.3.0...v0.3.1) (2026-09-27)


### Fixes

* calibrate and enforce fleet timeout budgets ([#158](https://github.com/athrael-soju/Narwhal/issues/158)) ([0629f01](https://github.com/athrael-soju/Narwhal/commit/0629f0150e022c4c8d83a6c9c06e057c491f4e71))
* enforce generation-bound engine readmission and document lifecycle drills ([#161](https://github.com/athrael-soju/Narwhal/issues/161)) ([03d3ac1](https://github.com/athrael-soju/Narwhal/commit/03d3ac1c062f7d2cd5269861c77b32dd04e31460))


### Documentation

* **agent:** define operator contract and plugin boundaries ([#140](https://github.com/athrael-soju/Narwhal/issues/140)) ([b142e6a](https://github.com/athrael-soju/Narwhal/commit/b142e6ab663382a6f8106c7cc05b1b125952c200))
* clarify cancellation in deployment attainment ([#139](https://github.com/athrael-soju/Narwhal/issues/139)) ([2215401](https://github.com/athrael-soju/Narwhal/commit/2215401bb045f23f93c4ea3a3b44008f0d5d0e5a))
* remove repeated guidance and clarify benchmark scope ([27b573c](https://github.com/athrael-soju/Narwhal/commit/27b573cb034d35eae31e546c8e59135c1752ac74))

## [0.3.0](https://github.com/athrael-soju/Narwhal/compare/v0.2.1...v0.3.0) (2026-09-25)


### Features

* add four-engine WSL2 development mode ([8c5b980](https://github.com/athrael-soju/Narwhal/commit/8c5b98051ce83cfebd80149b26694842a0cbfea9))
* **cli:** add offline inspection, diagnostics and bounded lifecycle ([9df3904](https://github.com/athrael-soju/Narwhal/commit/9df390467a7494f104f2b423a75d5a95c41c0938)), closes [#122](https://github.com/athrael-soju/Narwhal/issues/122) [#123](https://github.com/athrael-soju/Narwhal/issues/123) [#124](https://github.com/athrael-soju/Narwhal/issues/124) [#125](https://github.com/athrael-soju/Narwhal/issues/125) [#126](https://github.com/athrael-soju/Narwhal/issues/126)
* **docs:** add selectable dark mode ([1c87cae](https://github.com/athrael-soju/Narwhal/commit/1c87cae8d62eb8925ff6ca1d2cd7910c234d6b86))
* link Grafana icon to Narwhal repository ([ea1d6df](https://github.com/athrael-soju/Narwhal/commit/ea1d6dfc4c5f2057310273920015d90e70bfd045))
* reproducible benchmark runs ([#55](https://github.com/athrael-soju/Narwhal/issues/55)) ([671bf35](https://github.com/athrael-soju/Narwhal/commit/671bf35bcdc2f549deff60fa29305fa11da4f179))


### Fixes

* bind profiles to live engine generations ([4ea5be0](https://github.com/athrael-soju/Narwhal/commit/4ea5be0d9ba0162052785118ce4ea9aed0a23ea7))
* **ci:** remove release build pip cache ([#77](https://github.com/athrael-soju/Narwhal/issues/77)) ([28fe2e0](https://github.com/athrael-soju/Narwhal/commit/28fe2e072b5eb1639336572d23deeada7d4ab09f))
* **ci:** validate release SHA before candidate checkout ([1976ef8](https://github.com/athrael-soju/Narwhal/commit/1976ef89089580fc40e1db4fe31ad1670bf0a7b7))
* **cli:** enforce launch checks and consistent command output ([49133c9](https://github.com/athrael-soju/Narwhal/commit/49133c9eed696de2f8d5cfc1a4b7e80ba9aa493d))
* include documentation commits in releases ([d6e84bc](https://github.com/athrael-soju/Narwhal/commit/d6e84bc32745793cd9e42f7a3e0dae48eabcc3e3))
* **serving:** hide upstream exception details from clients ([#75](https://github.com/athrael-soju/Narwhal/issues/75)) ([15482c8](https://github.com/athrael-soju/Narwhal/commit/15482c8c4450dcec28f3cf7977e317e5606783d3))


### Documentation

* clarify runbooks and align validation guidance ([#104](https://github.com/athrael-soju/Narwhal/issues/104)) ([9cfe74b](https://github.com/athrael-soju/Narwhal/commit/9cfe74bf4ca8126e7c93599a47e5ccbc596e2052))
* document issue labels and fix bug template ([d8c37a5](https://github.com/athrael-soju/Narwhal/commit/d8c37a59e8eaad9d75335fe07af3a70170803673))
* hide page edit action ([04ed085](https://github.com/athrael-soju/Narwhal/commit/04ed085dc5cb9b368c35e306b7a51104d512d340))
* refresh README installation and references ([1612192](https://github.com/athrael-soju/Narwhal/commit/1612192043f239b92ca5a28417b014e30ca2bc6c))
* remove obsolete CPU tooling references ([#52](https://github.com/athrael-soju/Narwhal/issues/52)) ([b836a91](https://github.com/athrael-soju/Narwhal/commit/b836a910f6d51d7d0064a74f23987f75e9a313b1))
* simplify fleet deployment runbook ([#92](https://github.com/athrael-soju/Narwhal/issues/92)) ([bd93aaf](https://github.com/athrael-soju/Narwhal/commit/bd93aaf43d4f0bf9f6cc1abc04dfb397701d53c8))

## [0.2.1](https://github.com/athrael-soju/Narwhal/compare/v0.2.0...v0.2.1) (2026-09-23)


### Features

* publish Narwhal distributions to PyPI ([a328d00](https://github.com/athrael-soju/Narwhal/commit/a328d00af8909220ee8c1319a9d4bf873b93da9d))

## [0.2.0](https://github.com/athrael-soju/Narwhal/compare/v0.1.0...v0.2.0) (2026-09-23)


### Features

* make first fleet deployment reproducible ([14f27f0](https://github.com/athrael-soju/Narwhal/commit/14f27f03ef9d7909afad31da18ce1fb2c4da835e))


### Fixes

* **ci:** give deployment bundle tests Git history ([33c2f9f](https://github.com/athrael-soju/Narwhal/commit/33c2f9fe6e8a220d0e04bfe4dbeb7d54c3bc875f))

## 0.1.0 (2026-09-20)

Narwhal's initial public release routes LLM inference across prefill and decode engines and changes their roles while model weights remain loaded.

### Serving and admission

- OpenAI-compatible completion and chat endpoints with streaming, input validation, and vLLM engine support.
- Profile-based request placement with predictive admission and bounded queueing.
- Optional transient retries restart the complete prefill/decode attempt before visible output. One deadline and a shared retry budget bound every original request.
- Deployment-owned engine credentials authenticate serving, profiling, preflight and lifecycle probes.
- Client cancellation releases capacity and bypasses the engine-failure counters. First-token deadlines remain active through metadata-only stream frames.
- Engine health monitoring, quarantine, ejection, and recovery of the configured prefill and decode floors.

### Fleet control

- One reactive controller moves engines between prefill and decode roles. Each decision scores the current split and its adjacent alternatives. Role pinning and minimum pool sizes constrain those changes.
- Decode capacity estimates account for active sequences and resident KV tokens. Profile coverage, demand evidence, resident work, and consolidation safety constrain role changes.
- Warm-standby control-plane handoff preserves router state across takeover.

### Deployment and validation

- Generic vLLM/NIXL `kv_both` fleet support binds one model, hardware shape and KV layout through fleet configuration, engine attestations and measured profiles. Operators own image selection and process launch.
- Kubernetes, Ansible, Terraform or site automation provisions hosts, distributes artifacts, configures routes and launches engines; Narwhal validates the running engine and KV-transfer contracts.
- Preflight checks cover engine health, process-bound compatibility, generation, profiles, SLOs, and the transfer fabric.
- Fleet schema 1 groups controller, serving, engine I/O, recovery and profile policy under
  their runtime owners. Ingress owns client identity and content capture; Narwhal owns one
  global admission budget and retains timings, identifiers, placements and outcomes.
- Profile and configuration validation reject invalid types, non-finite values, incomplete measured domains, and mismatched engine sets.
- Deployments record source identity and file digests.

### Measurement and operations

- Profiling measures per-engine prefill and decode curves; correctness canaries track exact output across live controller events.
- Per-request journals, controller state, Prometheus metrics, a Grafana dashboard, and
  engine-down alerts. One startup command derives router and engine scrape targets from the
  deployed fleet, then verifies target health and populated dashboard queries.

### Deployment qualification

- Collect per-engine profiles, live runtime evidence, exact-output canaries and deployment load results before serving client traffic.
- Validate prefix caching and speculative decoding against the selected model, engine build and KV layout before enabling them. Prefix caching requires exact-replay and continuation checks. Use whole-wave restart policy when the engine build shares peer registrations across the fleet.

See the [configuration reference](docs/Configuration.md) for defaults and the [measurement guide](docs/Measure.md) for deployment requirements.
