Metadata-Version: 2.4
Name: mcr-attention
Version: 2.0.5
Summary: MCR-Attention V2.0: Multi-Scale Compressed Recurrent Attention Model for efficient language modeling with adaptive attention routing.
Author-email: "Gerson Fabian Buenahora Ormaza (BUEORM)" <dalusx64@gmail.com>
License: BUEORM MCR-Attention Community & Research License v1.0
        
        Copyright © 2026 Gerson Fabian Buenahora Ormaza (BUEORM). All rights reserved.
        
        Author / Rights Holder:  Gerson Fabian Buenahora Ormaza
        Public Name:             BUEORM
        GitHub:                  bueormnew
        Contact:                 dalusx64@gmail.com
        Project:                 MCR-Attention — Multi-Scale Compressed Recurrent Attention Model
        
        ─────────────────────────────────────────────────────────────────────────────
        1. PREAMBLE
        ─────────────────────────────────────────────────────────────────────────────
        
        MCR-Attention is an original software project developed and maintained by
        Gerson Fabian Buenahora Ormaza, publicly known as BUEORM.
        
        MCR-Attention is a PyTorch-based artificial intelligence library implementing
        a multi-scale recurrent attention architecture, including, among other
        components: multi-scale recurrent memory, learned decay mechanisms,
        associative scan processing, adaptive attention routing, rotary position
        encoding, streaming inference, and associated model creation, training,
        fine-tuning, saving, loading, and generation functionality.
        
        This License is intended to make the MCR-Attention source code publicly
        accessible for research, education, personal experimentation, and other
        non-commercial purposes, while expressly reserving commercial exploitation
        and other restricted rights to the Author.
        
        By accessing, downloading, copying, modifying, executing, studying, or
        otherwise using the Software, you agree to be bound by the terms of this
        License. If you do not agree to these terms, you may not use the Software.
        
        ─────────────────────────────────────────────────────────────────────────────
        2. DEFINITIONS
        ─────────────────────────────────────────────────────────────────────────────
        
        2.1 "Author"
            Gerson Fabian Buenahora Ormaza, publicly known as BUEORM, GitHub account
            bueormnew, residing in Colombia.
        
        2.2 "Software"
            MCR-Attention and all versions, releases, source code, object code,
            documentation, examples, configuration files, scripts, tests, utilities,
            implementation details, and other materials distributed by the Author
            under this License.
        
        2.3 "Architecture"
            The implementation embodied in the Software, including its documented and
            implemented structural organization, mechanisms, combinations, components,
            computational arrangements, and implementation-specific expression.
        
            For MCR-Attention, this includes without limitation: the multi-scale
            recurrent memory system, learned decay mechanisms, adaptive attention
            routing, compressed recurrent representations, associative scan
            implementation, streaming recurrent inference mechanisms, and the
            combination and implementation of these elements as expressed or embodied
            in the Software.
        
            This definition does not purport to claim copyright ownership over
            abstract mathematical ideas, general concepts, algorithms, principles, or
            methods to the extent that applicable law does not permit such protection.
        
        2.4 "Derivative Work"
            Any work that is based upon, incorporates, adapts, modifies, translates,
            ports, refactors, restructures, extends, or otherwise derives from the
            Software or a substantial portion of the Software.
        
            A work may constitute a Derivative Work regardless of whether its source
            code is substantially rewritten, translated into another programming
            language, reorganized, optimized, renamed, refactored, or otherwise
            modified.
        
        2.5 "MCR-Derived Implementation"
            Any implementation, reimplementation, rewrite, port, adaptation, or
            substantially similar implementation that:
            (a) incorporates or reproduces a substantial portion of the Software;
            (b) is based materially upon the source code, documentation,
                implementation details, or technical design of the Software;
            (c) reproduces a substantial portion of the implementation-specific
                structure or expression of the Software; or
            (d) is demonstrated, through technical evidence or other reasonable
                evidence, to be materially derived from the Software.
        
            Changing programming languages, variable names, class names, function
            names, file organization, APIs, or implementation syntax does not by
            itself prevent an implementation from being considered a Derivative Work
            or MCR-Derived Implementation.
        
        2.6 "Model"
            Any machine-learning, artificial-intelligence, neural-network,
            language-model, or other computational model created, trained, fine-tuned,
            modified, initialized, or otherwise produced using the Software or a
            Derivative Work.
        
        2.7 "Commercial Use"
            Any use intended, directly or indirectly, to obtain monetary compensation,
            financial benefit, economic advantage, business value, revenue, profit,
            investment value, commercial advantage, or other consideration.
        
            Commercial Use includes without limitation: selling Software or Derivative
            Works; selling Models; licensing Models; offering Models through paid
            access; offering APIs; offering inference services; offering hosted or
            cloud services; SaaS products; subscriptions; memberships; paid
            applications; paid platforms; paid plugins or integrations;
            advertising-supported services; sponsorship-supported services where the
            Software or a Model materially contributes to the commercial offering;
            usage-based billing; token-based billing; charging for inference; charging
            for access to a Model; commercial consulting or services materially based
            upon the Software or Architecture; embedding the Software or a Model into a
            commercial product; using the Software or a Model to provide a commercial
            service to third parties; or any other direct or indirect monetization of
            the Software, Architecture, Derivative Work, MCR-Derived Implementation,
            or Model.
        
            Commercial Use does not require that the Software itself be sold. Indirect
            commercial exploitation is also Commercial Use.
        
        2.8 "Non-Commercial Use"
            Use that is not intended to generate direct or indirect monetary
            compensation, commercial advantage, business revenue, or other economic
            benefit. Examples include personal experimentation, education, academic
            research, scientific research, benchmarking, testing, and development
            conducted without Commercial Use.
        
        ─────────────────────────────────────────────────────────────────────────────
        3. COPYRIGHT AND OWNERSHIP
        ─────────────────────────────────────────────────────────────────────────────
        
        The Software is protected by applicable copyright and intellectual-property
        laws. All rights, title, and interest in and to the Software remain with the
        Author except for the limited rights expressly granted by this License.
        Nothing in this License constitutes a transfer of ownership of the Software.
        The Author reserves all rights not expressly granted herein.
        
        ─────────────────────────────────────────────────────────────────────────────
        4. LIMITED NON-COMMERCIAL LICENSE GRANT
        ─────────────────────────────────────────────────────────────────────────────
        
        Subject to full compliance with this License, the Author grants you a
        limited, worldwide, non-exclusive, non-transferable, non-sublicensable
        license to:
        
          (a) access and download the Software;
          (b) inspect and study the source code;
          (c) execute the Software;
          (d) modify the Software;
          (e) create Derivative Works;
          (f) create and train Models using the Software;
          (g) fine-tune Models using the Software;
          (h) benchmark and evaluate the Software and Models; and
          (i) conduct academic, scientific, educational, personal, and other
              Non-Commercial experimentation.
        
        All of the foregoing permissions are granted solely for Non-Commercial Use
        and subject to every other condition of this License.
        
        ─────────────────────────────────────────────────────────────────────────────
        5. MODEL CREATION AND TRAINING
        ─────────────────────────────────────────────────────────────────────────────
        
        The Author expressly permits users to create, train, fine-tune, evaluate, and
        experiment with Models using the Software for Non-Commercial purposes.
        
        A Model created or trained using the Software may be used for: research,
        education, personal experimentation, academic experimentation, scientific
        experimentation, benchmarking, testing, and other lawful Non-Commercial
        purposes.
        
        No Commercial Use of such Models is permitted under this License.
        
        ─────────────────────────────────────────────────────────────────────────────
        6. MODEL DISTRIBUTION
        ─────────────────────────────────────────────────────────────────────────────
        
        Models created or trained using the Software may be distributed for
        Non-Commercial purposes only, provided that:
        
          (a) the Model is distributed together with a copy of this License or a
              clearly accessible reference to this License;
          (b) the Model documentation identifies the use of MCR-Attention;
          (c) the Model does not falsely claim that the underlying MCR-Attention
              technology was independently created by the distributor;
          (d) no Commercial Use is authorized or implied; and
          (e) the distributor does not grant downstream rights inconsistent with
              this License.
        
        The distribution of a Model does not grant any commercial license to the
        Model or to the Software.
        
        ─────────────────────────────────────────────────────────────────────────────
        7. MODIFICATIONS AND DERIVATIVE WORKS
        ─────────────────────────────────────────────────────────────────────────────
        
        You may modify the Software for Non-Commercial purposes. However, every
        Derivative Work, modification, adaptation, port, fork, or MCR-Derived
        Implementation must:
        
          (a) retain this License and include a copy of it;
          (b) preserve the copyright notice;
          (c) identify significant modifications;
          (d) identify the original MCR-Attention project and Author;
          (e) clearly indicate that the work is derived from MCR-Attention where
              applicable; and
          (f) remain subject to the same restrictions contained in this License.
        
        You may not relicense a Derivative Work under a license that permits rights
        broader than those granted by this License. You may not remove or circumvent
        the restrictions of this License by changing the programming language,
        rewriting the implementation, restructuring the source tree, changing names,
        changing APIs, or substantially refactoring the Software.
        
        ─────────────────────────────────────────────────────────────────────────────
        8. REIMPLEMENTATIONS AND SUBSTANTIAL REWRITES
        ─────────────────────────────────────────────────────────────────────────────
        
        A complete or substantial rewrite of the Software does not automatically
        eliminate the obligations of this License. Where a reimplementation is
        materially derived from the Software, its source code, implementation-
        specific design, documentation, technical implementation, or other protected
        expression, the resulting work shall be treated as a Derivative Work or
        MCR-Derived Implementation to the extent permitted by applicable law.
        
        This clause does not prohibit genuinely independent development that was
        created without copying or materially deriving from the Software, nor does it
        claim ownership of ideas, mathematical principles, algorithms, methods, or
        systems that are not protectable under applicable law.
        
        ─────────────────────────────────────────────────────────────────────────────
        9. MANDATORY SAME-LICENSE REQUIREMENT
        ─────────────────────────────────────────────────────────────────────────────
        
        Any Derivative Work, MCR-Derived Implementation, modification, fork, port, or
        other work covered by Sections 7 or 8 must be distributed under this License
        unless the Author provides separate written authorization.
        
        You may not replace this License with MIT, Apache-2.0, BSD, GPL, LGPL, AGPL,
        another open-source license, a proprietary license with broader rights, or any
        other license that conflicts with this License.
        
        ─────────────────────────────────────────────────────────────────────────────
        10. COMMERCIAL USE IS NOT GRANTED
        ─────────────────────────────────────────────────────────────────────────────
        
        NO COMMERCIAL LICENSE IS GRANTED BY THIS LICENSE.
        
        You may not use the Software, Architecture, Derivative Works, MCR-Derived
        Implementations, or Models for Commercial Use unless you have obtained a
        separate written commercial license from the Author.
        
        This prohibition applies regardless of whether the Commercial Use is direct
        or indirect.
        
        ─────────────────────────────────────────────────────────────────────────────
        11. PROHIBITED COMMERCIAL ACTIVITIES
        ─────────────────────────────────────────────────────────────────────────────
        
        Without a separate written commercial license from the Author, you may not:
        
          - sell the Software, a Derivative Work, or a Model;
          - monetize, license, or offer paid access to a Model;
          - offer a paid API, inference service, SaaS, or cloud service based on
            the Software, Architecture, or a Model;
          - embed the Software or Architecture into a commercial product;
          - charge users based on tokens, requests, inference, usage, or access
            where the Software or Architecture materially contributes;
          - use advertising, sponsorship, or indirect monetization to commercially
            exploit the Software, Architecture, or Models; or
          - create a commercial business whose material technical offering depends
            upon the Software or Architecture.
        
        ─────────────────────────────────────────────────────────────────────────────
        12. COMMERCIAL LICENSING
        ─────────────────────────────────────────────────────────────────────────────
        
        Commercial Use may be authorized exclusively through a separate written
        agreement with the Author. Commercial licenses are individually negotiated.
        The Author may determine, at his sole discretion, all applicable terms
        including license fees, royalty rates, revenue-sharing percentages,
        restrictions, and other commercial conditions.
        
        No person or organization is entitled to a commercial license merely because
        the Software is publicly available.
        
        ─────────────────────────────────────────────────────────────────────────────
        13. REVENUE SHARING AND ROYALTIES
        ─────────────────────────────────────────────────────────────────────────────
        
        Where a commercial license is negotiated, the Author may require a percentage
        of revenue, profit, gross receipts, net receipts, or other agreed economic
        measure generated directly or indirectly through the licensed use. All such
        terms shall be established exclusively in the separate commercial agreement.
        
        ─────────────────────────────────────────────────────────────────────────────
        14. NO IMPLIED COMMERCIAL PERMISSION
        ─────────────────────────────────────────────────────────────────────────────
        
        The public availability of the Software on GitHub, PyPI, package repositories,
        or other public locations does not constitute permission for Commercial Use.
        The ability to download, install, execute, modify, or train Models using the
        Software does not constitute a commercial license. The absence of a technical
        restriction preventing Commercial Use does not constitute authorization.
        
        ─────────────────────────────────────────────────────────────────────────────
        15. ATTRIBUTION
        ─────────────────────────────────────────────────────────────────────────────
        
        Any permitted distribution, publication, Model, Derivative Work, benchmark,
        research result, or other public material that materially uses the Software
        must provide reasonable attribution to:
        
            MCR-Attention — Multi-Scale Compressed Recurrent Attention Model
            Created by Gerson Fabian Buenahora Ormaza (BUEORM)
            GitHub: bueormnew
        
        Where technically reasonable, the distributor should provide a reference to
        the original project repository. Attribution does not grant any additional
        rights.
        
        ─────────────────────────────────────────────────────────────────────────────
        16. NO REMOVAL OF NOTICES
        ─────────────────────────────────────────────────────────────────────────────
        
        You may not remove, obscure, alter, or falsify copyright notices, attribution
        notices, license notices, authorship information, project identification,
        references to BUEORM, references to MCR-Attention, or notices identifying the
        origin of the Software.
        
        ─────────────────────────────────────────────────────────────────────────────
        17. LAWFUL USE ONLY
        ─────────────────────────────────────────────────────────────────────────────
        
        Nothing in this License authorizes unlawful use. The Software may not be used
        for activities prohibited by applicable law, including activities that violate
        applicable criminal, civil, regulatory, privacy, intellectual-property,
        security, or other laws.
        
        ─────────────────────────────────────────────────────────────────────────────
        18. NO WARRANTY
        ─────────────────────────────────────────────────────────────────────────────
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW. THE AUTHOR
        DISCLAIMS ALL WARRANTIES INCLUDING, WITHOUT LIMITATION, WARRANTIES OF
        MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, NON-INFRINGEMENT,
        ACCURACY, RELIABILITY, PERFORMANCE, AND AVAILABILITY.
        
        ─────────────────────────────────────────────────────────────────────────────
        19. LIMITATION OF LIABILITY
        ─────────────────────────────────────────────────────────────────────────────
        
        TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, THE AUTHOR SHALL NOT BE
        LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, EXEMPLARY, OR
        PUNITIVE DAMAGES ARISING FROM OR RELATING TO THE SOFTWARE OR THIS LICENSE.
        
        ─────────────────────────────────────────────────────────────────────────────
        20. TERMINATION
        ─────────────────────────────────────────────────────────────────────────────
        
        This License automatically terminates upon any material violation of its
        terms. Upon termination, you must immediately cease all uses of the Software
        authorized solely by this License. Termination does not eliminate any rights
        or remedies the Author may have under applicable law.
        
        ─────────────────────────────────────────────────────────────────────────────
        21. UNAUTHORIZED COMMERCIAL USE
        ─────────────────────────────────────────────────────────────────────────────
        
        Any Commercial Use without a valid written commercial license from the Author
        constitutes an unauthorized use of the Software and may constitute
        infringement of applicable intellectual-property, contractual, or other
        rights. The Author reserves all rights and remedies available under
        applicable law.
        
        ─────────────────────────────────────────────────────────────────────────────
        22. NO RETROACTIVE COMMERCIAL AUTHORIZATION
        ─────────────────────────────────────────────────────────────────────────────
        
        A user who wishes to engage in Commercial Use must obtain written
        authorization before beginning such Commercial Use. The Author may, at his
        discretion, negotiate a commercial license after an inquiry, but has no
        obligation to grant one.
        
        ─────────────────────────────────────────────────────────────────────────────
        23. SEPARATE COMMERCIAL AGREEMENTS
        ─────────────────────────────────────────────────────────────────────────────
        
        A commercial agreement must be written and accepted by the Author. Informal
        statements, comments, emails, public posts, discussions, or assumptions shall
        not constitute a commercial license unless they clearly and expressly
        constitute written authorization by the Author.
        
        ─────────────────────────────────────────────────────────────────────────────
        24. NO SUBLICENSING
        ─────────────────────────────────────────────────────────────────────────────
        
        This License does not grant you the right to sublicense the Software. Any
        distribution of a Derivative Work must remain subject to this License.
        
        ─────────────────────────────────────────────────────────────────────────────
        25. THIRD-PARTY COMPONENTS
        ─────────────────────────────────────────────────────────────────────────────
        
        The Software may contain or interact with third-party software, libraries,
        datasets, models, or other materials subject to separate licenses. Nothing in
        this License overrides the independent rights or licenses applicable to
        third-party components. Users are responsible for complying with all
        applicable third-party licenses.
        
        ─────────────────────────────────────────────────────────────────────────────
        26. INDEPENDENCE OF PROVISIONS
        ─────────────────────────────────────────────────────────────────────────────
        
        If any provision of this License is found to be invalid, unlawful, or
        unenforceable under applicable law, that provision shall be interpreted or
        limited to the minimum extent necessary, and the remaining provisions shall
        remain in effect to the maximum extent permitted by law.
        
        ─────────────────────────────────────────────────────────────────────────────
        27. NO WAIVER
        ─────────────────────────────────────────────────────────────────────────────
        
        Failure by the Author to enforce any provision of this License on one
        occasion shall not constitute a waiver of the Author's right to enforce that
        provision in the future. A waiver must be expressly granted in writing.
        
        ─────────────────────────────────────────────────────────────────────────────
        28. RESERVATION OF RIGHTS
        ─────────────────────────────────────────────────────────────────────────────
        
        All rights not expressly granted by this License are reserved by the Author.
        In particular, this License does not grant any commercial rights, patent
        rights, trademark rights, rights to sublicense, or rights to commercially
        exploit the Software, Architecture, Derivative Works, MCR-Derived
        Implementations, or Models except where expressly authorized by a separate
        written agreement.
        
        ─────────────────────────────────────────────────────────────────────────────
        29. GOVERNING LAW AND JURISDICTION
        ─────────────────────────────────────────────────────────────────────────────
        
        This License shall be interpreted in accordance with the laws applicable to
        the Author and the applicable jurisdiction of Colombia, without prejudice to
        any mandatory laws or rights that may apply to a particular user or
        jurisdiction. The Author reserves the right to seek appropriate relief in any
        jurisdiction where unauthorized use occurs or where otherwise permitted by
        law.
        
        ─────────────────────────────────────────────────────────────────────────────
        30. COMMERCIAL LICENSING CONTACT
        ─────────────────────────────────────────────────────────────────────────────
        
        For commercial licensing, partnerships, API deployment, commercial model
        deployment, SaaS use, subscriptions, enterprise use, commercialization, or
        other commercial arrangements, contact:
        
            Gerson Fabian Buenahora Ormaza (BUEORM)
            Email:  dalusx64@gmail.com
            GitHub: bueormnew
        
        Commercial terms, pricing, royalties, revenue sharing, usage restrictions,
        and other conditions shall be negotiated individually.
        
        ─────────────────────────────────────────────────────────────────────────────
        31. LICENSE VERSION
        ─────────────────────────────────────────────────────────────────────────────
        
        BUEORM MCR-Attention Community & Research License v1.0
        Copyright © 2026 Gerson Fabian Buenahora Ormaza (BUEORM).
        All rights reserved except for the limited rights expressly granted herein.
        
        ─────────────────────────────────────────────────────────────────────────────
        IMPORTANT NOTICE
        ─────────────────────────────────────────────────────────────────────────────
        
        This License is a custom licensing document describing the Author's intended
        permissions and restrictions. It is NOT an Open Source license as defined by
        the Open Source Initiative.
        
        The Software is publicly available for inspection, research, education,
        experimentation, and other permitted Non-Commercial purposes, but public
        availability does not grant Commercial Use rights.
        
        For commercial use, please contact the Author to negotiate a separate
        commercial license.
        
            Contact:     dalusx64@gmail.com
            Author:      Gerson Fabian Buenahora Ormaza
            Public Name: BUEORM
            GitHub:      bueormnew
        
Project-URL: Homepage, https://github.com/bueormnew/MCRA
Project-URL: Repository, https://github.com/bueormnew/MCRA
Project-URL: Documentation, https://github.com/bueormnew/MCRA#readme
Project-URL: Issues, https://github.com/bueormnew/MCRA/issues
Keywords: attention,recurrent,language-model,transformer,pytorch,deep-learning,mcr-attention,bueorm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: hypothesis>=6.0; extra == "dev"
Provides-Extra: benchmark
Requires-Dist: matplotlib>=3.5; extra == "benchmark"
Provides-Extra: hf
Requires-Dist: transformers>=4.35.0; extra == "hf"
Requires-Dist: huggingface-hub>=0.20.0; extra == "hf"
Provides-Extra: all
Requires-Dist: mcr-attention[benchmark,dev,hf]; extra == "all"
Dynamic: license-file

# MCR-Attention V2.0

**Multi-Scale Compressed Recurrent Attention Model**

A PyTorch library for building, training, and deploying language models using multi-scale recurrent memories with adaptive attention routing. MCR-Attention replaces traditional self-attention with O(N) linear recurrences at multiple time scales, enabling efficient processing of long sequences without the quadratic cost of transformers.

## Features

- **Multi-scale recurrent memory** — K independent decay scales capture patterns from short-range to long-range context
- **Parallel associative scan** — O(N) time, O(log N) parallel depth for training
- **Streaming inference** — Single-token updates with O(1) per-step cost
- **Attention routing** — Learned soft-attention over compressed memory vectors
- **Rotary position encoding (RoPE)** — Position-aware representations
- **Simple high-level API** — Create, train, save, load, and generate in a few lines

## Installation

```bash
pip install mcr-attention
```

For development (includes pytest, hypothesis):
```bash
pip install mcr-attention[dev]
```

For benchmarking (adds matplotlib):
```bash
pip install mcr-attention[all]
```

## Quick Start

```python
from mcr_attention import MCRConfig, MCRAttention, create_model, save_model, load_model

# Create a model
config = MCRConfig(vocab_size=32000, d_model=512)
model = create_model(config)

# Or use kwargs directly
model = create_model(vocab_size=32000, d_model=512, num_scales=8)
```

## API Overview

### Core Classes

| Class | Description |
|-------|-------------|
| `MCRConfig` | Model hyperparameters (vocab, dimensions, scales, etc.) |
| `MCRAttention` | The full model (embedding → memory → router → head) |
| `Trainer` | High-level training loop with warmup and checkpointing |
| `GenerationConfig` | Settings for token generation (temp, top-k, top-p) |

### Core Functions

| Function | Description |
|----------|-------------|
| `create_model(config, **kwargs)` | Create a new model instance |
| `save_model(model, path)` | Save model weights + config to directory |
| `load_model(path, device)` | Load model from directory |
| `generate(model, prompt, config)` | Generate tokens from a prompt |

## Configuration Reference

```python
from mcr_attention import MCRConfig

config = MCRConfig(
    vocab_size=32000,    # Token vocabulary size
    d_model=512,         # Embedding dimension
    d_state=128,         # Recurrent hidden state dimension per scale
    num_scales=8,        # Number of memory scales (K)
    scale_min=256,       # Shortest window length
    scale_factor=2,      # Geometric factor between scales
    rope_base=10000.0,   # RoPE base frequency
    dropout=0.1,         # Dropout probability
    use_input_gate=True, # Learned input gate per scale
    tie_embeddings=False, # Tie embedding weights with language head
    mlp_hidden=2048,     # Language head MLP hidden dim
    max_seq_len=32768,   # Maximum sequence length
)
```

## Training

```python
from mcr_attention import MCRConfig, create_model, Trainer
from mcr_attention.api import TrainingConfig
import torch

# Create model
model = create_model(vocab_size=32000, d_model=512)

# Define data source
def data_fn():
    """Return (input_ids [B, N], targets [B]) each call."""
    tokens = torch.randint(0, 32000, (16, 256))
    return tokens[:, :-1], tokens[:, -1]

# Train
config = TrainingConfig(
    learning_rate=5e-4,
    num_steps=1000,
    warmup_steps=100,
    save_dir="checkpoints/",
    save_interval=500,
    device="auto",
)
trainer = Trainer(model, config)
stats = trainer.fit(data_fn)

print(f"Final loss: {stats['final_loss']:.4f}")
```

### Training with Callbacks

```python
losses = []

def track_loss(step, loss, model):
    losses.append(loss)

stats = trainer.fit(data_fn, callbacks=[track_loss])
```

### Training with Evaluation

```python
def eval_fn(model):
    model.eval()
    # Run your evaluation logic
    return {"accuracy": 0.85, "perplexity": 12.3}

stats = trainer.fit(data_fn, eval_fn=eval_fn)
```

## Fine-tuning

```python
from mcr_attention import load_model
from mcr_attention.finetune import finetune, FinetuneConfig, freeze_all_except

# Load pretrained model
model = load_model("pretrained_model/")

# Fine-tune with frozen embeddings and memory
config = FinetuneConfig(
    learning_rate=1e-4,
    num_steps=500,
    freeze_embeddings=True,
    freeze_memory=True,
)
stats = finetune(model, data_fn, config, save_path="finetuned_model/")

# Or freeze everything except the head
freeze_all_except(model, ["head"])
```

## Saving and Loading

```python
from mcr_attention import save_model, load_model, create_model

model = create_model(vocab_size=32000, d_model=512)

# Save (creates directory with config.json, model.pt, metadata.json)
save_model(model, "my_model/", metadata={"trained_on": "my_dataset"})

# Load
model = load_model("my_model/", device="auto")  # auto-detects cuda/mps/cpu
```

## Generation

```python
from mcr_attention import load_model, GenerationConfig
from mcr_attention.api import generate

model = load_model("my_model/")

# Basic generation
tokens = generate(model, prompt=[1, 2, 3, 4, 5], max_tokens=100)

# With sampling controls
config = GenerationConfig(
    max_tokens=200,
    temperature=0.8,
    top_k=50,
    top_p=0.9,
    repetition_penalty=1.2,
    seed=42,
)
tokens = generate(model, prompt=[1, 2, 3], config=config)

# Greedy decoding
tokens = generate(model, prompt=[1, 2, 3], temperature=0.0)
```

## CLI Usage

```bash
# Show model info
mcr-attention info path/to/model/

# Run speed benchmark
mcr-attention benchmark --lengths 32 128 512 2048 --d-model 256

# Generate tokens
mcr-attention generate path/to/model/ --prompt 1 2 3 4 5 --max-tokens 50 --temperature 0.8
```

## Architecture

MCR-Attention processes sequences through four stages:

```
Token IDs → [Embedding + RoPE] → [Multi-Scale Memory] → [Attention Router] → [Language Head] → Logits
```

1. **Embedding Layer**: Token embeddings with Rotary Position Encoding for position-awareness.

2. **Multi-Scale Memory**: K independent recurrent scales, each with a geometric window length (L_k = scale_min × scale_factor^(k-1)). Each scale runs a linear recurrence with learned decay:
   ```
   h_t = a ⊙ h_(t-1) + B @ e_t
   ```
   Parallelized via associative scan during training.

3. **Attention Router**: 1×K soft attention where the query is formed from the last token embedding and the longest-scale memory. Attends over all K compressed memory vectors.

4. **Language Head**: Two-layer MLP (GELU activation) projecting to vocabulary logits. Supports optional embedding weight tying.

### Complexity

| Mode | Time | Space |
|------|------|-------|
| Training (parallel scan) | O(N) | O(N × K × d_state) |
| Inference (streaming) | O(1) per token | O(K × d_state) |

## HuggingFace Hub Integration

Load, push, and use MCR-Attention models directly from the HuggingFace ecosystem.

```bash
pip install mcr-attention[hf]  # includes transformers + huggingface-hub
```

```python
from mcr_attention.hf import from_pretrained, push_to_hub

# Load from Hub
model = from_pretrained("bueorm/my-model")
text = model.generate_text("Once upon a time", max_new_tokens=200)

# Push to Hub
push_to_hub(model, "username/my-mcr-model", tokenizer=tokenizer)
```

### HuggingFace Spaces (Gradio)

```python
from mcr_attention.hf import from_pretrained
import gradio as gr

model = from_pretrained("bueorm/tinystories-mcr")

gr.Interface(
    fn=lambda prompt: model.generate_text(prompt, max_new_tokens=200),
    inputs=gr.Textbox(label="Prompt"),
    outputs=gr.Textbox(label="Generated text"),
    title="MCR-Attention Story Generator",
).launch()
```

## Development

```bash
# Clone and install in dev mode
git clone https://github.com/bueormnew/MCRA.git
cd MCRA
pip install -e ".[dev]"

# Run tests
pytest tests/ -v

# Run specific test
pytest tests/test_model.py -v
```

## License

**BUEORM MCR-Attention Community & Research License v1.0**
Copyright © 2026 Gerson Fabian Buenahora Ormaza (BUEORM). All rights reserved.

This software is publicly available for **non-commercial use only** (research, education, personal experimentation). Commercial use requires a separate written license.

- **Author**: Gerson Fabian Buenahora Ormaza (BUEORM)
- **Commercial licensing**: dalusx64@gmail.com
- **GitHub**: [bueormnew](https://github.com/bueormnew)

See [LICENSE](LICENSE) for full terms.
