ComposeLM
Copyright 2026 ComposeLM Contributors

This product includes software developed by the ComposeLM Contributors,
licensed under the Apache License, Version 2.0 (see the LICENSE file).


===============================================================================
THIRD-PARTY ATTRIBUTIONS
===============================================================================

ComposeLM is an independent implementation of transformer architectures and
techniques that are described in the publications listed at the end of this
file. Architectures, algorithms and mathematical formulations are not
themselves subject to copyright; the implementations below were consulted as
references, and several helper routines in ComposeLM follow their formulations,
parameter conventions and naming (for example `repeat_kv`, `_rotate_half`,
`apply_rotary_pos_emb`, the `_yarn_*` correction helpers, and the ALiBi slope
computation). Attribution is provided here accordingly.

Weight-key naming used by `composelm/convert/` (`q_proj`, `gate_proj`,
`input_layernorm`, `c_attn`, ...) mirrors the Hugging Face and GPT-2 layouts
purely for checkpoint interoperability.


-------------------------------------------------------------------------------
Hugging Face Transformers
-------------------------------------------------------------------------------
https://github.com/huggingface/transformers
Licensed under the Apache License, Version 2.0.

    Copyright 2022 EleutherAI and the HuggingFace Inc. team. All rights reserved.

    This code is based on EleutherAI's GPT-NeoX library and the GPT-NeoX
    and OPT implementations in this library. It has been modified from its
    original forms to accommodate minor architectural differences compared
    to GPT-NeoX and OPT used by the Meta AI team that trained the model.

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.

Referenced for: RoPE application (`_rotate_half` / `apply_rotary_pos_emb`),
grouped-query KV expansion (`repeat_kv`), RoPE scaling variants (linear, NTK,
dynamic NTK, Llama 3 frequency smoothing), and HF checkpoint key layouts.


-------------------------------------------------------------------------------
YaRN
-------------------------------------------------------------------------------
https://github.com/jquesnelle/yarn
Licensed under the MIT License.

    Copyright (c) 2023 Jeffrey Quesnelle and Bowen Peng

Referenced for: the NTK-by-parts ramp and correction-range helpers
(`_yarn_find_correction_dim`, `_yarn_find_correction_range`,
`_yarn_linear_ramp_mask`) and the attention temperature factor in
`composelm/layers/pos_emb.py`.


-------------------------------------------------------------------------------
ALiBi (Attention with Linear Biases)
-------------------------------------------------------------------------------
https://github.com/ofirpress/attention_with_linear_biases
Licensed under the MIT License.

    Copyright (c) Facebook, Inc. and its affiliates.

Referenced for: the per-head slope geometric series, including the
non-power-of-two head-count fallback (`ALiBiSlopes._get_slopes` in
`composelm/layers/pos_emb.py`).


-------------------------------------------------------------------------------
MIT License notice
-------------------------------------------------------------------------------
The following permission notice applies to the MIT-licensed works listed
above:

    Permission is hereby granted, free of charge, to any person obtaining a
    copy of this software and associated documentation files (the "Software"),
    to deal in the Software without restriction, including without limitation
    the rights to use, copy, modify, merge, publish, distribute, sublicense,
    and/or sell copies of the Software, and to permit persons to whom the
    Software is furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included
    in all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
    DEALINGS IN THE SOFTWARE.


===============================================================================
OPTIONAL RUNTIME DEPENDENCIES
===============================================================================

ComposeLM does not bundle, vendor or redistribute any of the following. They are
optional imports resolved at runtime, and each remains under its own license.

    PyTorch                 https://github.com/pytorch/pytorch
    FlashAttention          https://github.com/Dao-AILab/flash-attention
    Liger-Kernel            https://github.com/linkedin/Liger-Kernel
    DeepSpeed               https://github.com/deepspeedai/DeepSpeed
    Hugging Face Transformers, safetensors, datasets, NumPy, PyYAML


===============================================================================
REFERENCED PUBLICATIONS
===============================================================================

The architectures and techniques ComposeLM implements are described in:

    Attention Is All You Need
      Vaswani et al., 2017 — arXiv:1706.03762
      (multi-head attention, sinusoidal positional encoding)

    Layer Normalization
      Ba, Kiros, Hinton, 2016 — arXiv:1607.06450

    Root Mean Square Layer Normalization
      Zhang, Sennrich, 2019 — arXiv:1910.07467

    RoFormer: Enhanced Transformer with Rotary Position Embedding
      Su et al., 2021 — arXiv:2104.09864  (RoPE)

    GLU Variants Improve Transformer
      Shazeer, 2020 — arXiv:2002.05202  (SwiGLU, GEGLU, ReGLU)

    Fast Transformer Decoding: One Write-Head is All You Need
      Shazeer, 2019 — arXiv:1911.02150  (MQA)

    GQA: Training Generalized Multi-Query Transformer Models from
    Multi-Head Checkpoints
      Ainslie et al., 2023 — arXiv:2305.13245

    Train Short, Test Long: Attention with Linear Biases Enables Input
    Length Extrapolation
      Press, Smith, Lewis, 2022 — arXiv:2108.12409  (ALiBi)

    YaRN: Efficient Context Window Extension of Large Language Models
      Peng, Quesnelle, Fan, Shippole, 2023 — arXiv:2309.00071

    NTK-Aware Scaled RoPE
      Community contribution originally published by "bloc97" (2023);
      the base-scaling closed form in `RotaryEmbedding._compute_inv_freq`
      follows the formulation that became standard practice thereafter.

    Switch Transformers: Scaling to Trillion Parameter Models with Simple
    and Efficient Sparsity
      Fedus, Zoph, Shazeer, 2021 — arXiv:2101.03961
      (top-k routing, load-balancing auxiliary loss)

    Mistral 7B
      Jiang et al., 2023 — arXiv:2310.06825  (sliding window attention)

    DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts
    Language Model
      DeepSeek-AI, 2024 — arXiv:2405.04434  (Multi-head Latent Attention)

    FlashAttention: Fast and Memory-Efficient Exact Attention with
    IO-Awareness
      Dao et al., 2022 — arXiv:2205.14135

    Fast Inference from Transformers via Speculative Decoding
      Leviathan, Kalman, Matias, 2023 — arXiv:2211.17192

    Accelerating Large Language Model Decoding with Speculative Sampling
      Chen et al., 2023 — arXiv:2302.01318


===============================================================================
TRADEMARKS
===============================================================================

Architecture preset names such as "gpt", "llama", "mistral", "gemma", "qwen",
"deepseek" and "phi" identify the structural configuration a preset reproduces.
They are used descriptively, for interoperability and identification only.

Llama is a trademark of Meta Platforms, Inc. Mistral is a trademark of Mistral
AI. Gemma and Gemini are trademarks of Google LLC. Qwen is a trademark of
Alibaba Group. Phi is a trademark of Microsoft Corporation. DeepSeek is a
trademark of DeepSeek. All other marks are the property of their respective
owners.

ComposeLM is not affiliated with, endorsed by, or sponsored by any of these
organizations. ComposeLM distributes no model weights; pretrained weights are
governed by their own separate licenses and use policies.
