arbi-serve
Copyright 2026 Arbi City

Licensed under the Apache License, Version 2.0 (see LICENSE).

This product reads and serves the EXL3 quantized-weight format, and contains
code derived from third-party work credited below.

-------------------------------------------------------------------------------
exllamav3 (EXL3 format, trellis quantization, kernels)
https://github.com/turboderp-org/exllamav3
Copyright (c) 2025 Turboderp
MIT License (full text: arbi_serve/weight_quant/gemv/csrc/exl3_moe/LICENSE.upstream)

The EXL3 checkpoint layout, its trellis coding, the 3INST / MCG / MUL1
codebooks and their constants, the half-integer bitrate scheme, the 128-wide
Hadamard incoherence processing with suh/svh scales and the tensor-core tile
permutation are turboderp's design. arbi-serve's EXL3 reader and kernels
implement that format.

Files copied verbatim from exllamav3 v1.6.6 (sha256 in MANIFEST there):
  arbi_serve/weight_quant/gemv/csrc/exl3_moe/{util.h, util.cuh, compat.cuh,
    ptx.cuh, arch.cuh, ptx_portable.cuh, rocm/ptx_rocm.cuh,
    quant/codebook.cuh, quant/exl3_dq.cuh, quant/hadamard_inner.cuh}

Files that restate or adapt exllamav3 code (codebook decode, 128-wide
Hadamard), MIT-licensed as above:
  arbi_serve/weight_quant/exl3/csrc/exl3_had128.cuh
  arbi_serve/weight_quant/exl3/csrc/exl3_a_prep.cu
  arbi_serve/weight_quant/exl3/csrc/exl3_i8_decode.cuh
  arbi_serve/weight_quant/exl3/csrc/exl3_i8_constants.cuh
  arbi_serve/weight_quant/gemv/csrc/_gemv_loaders.cuh
  arbi_serve/weight_quant/gemv/csrc/_gemv_container.cuh
  arbi_serve/weight_quant/gemv/csrc/_gemv_exl3_moe.cu

-------------------------------------------------------------------------------
QTIP: Quantization with Trellises and Incoherence Processing
Albert Tseng, Qingyao Sun, David Hou, Christopher De Sa. NeurIPS 2024.
arXiv:2406.11235. https://github.com/Cornell-RelaxML/qtip

EXL3's trellis coding and computed codebooks build on QTIP. Hadamard
incoherence processing follows QuIP# (Tseng et al., 2024, arXiv:2402.04396)
and QuIP (Chee et al., 2023, arXiv:2307.13304).
