Metadata-Version: 2.4
Name: spiral-lm
Version: 0.1.0
Summary: Spiral — a reversible looped transformer (~1M core) with per-iteration subspace reprojection, online-distilled from a ~135M teacher.
Author-email: Marius-Constantin Dinu <dinu.marius.constantin@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://dentate.cortex.a2olabs.com
Project-URL: Source, https://github.com/Xpitfire/spiral
Project-URL: Dentate, https://github.com/Xpitfire/dentate
Keywords: looped-transformer,reversible,distillation,small-language-models,dentate
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.6
Provides-Extra: train
Requires-Dist: transformers; extra == "train"
Requires-Dist: datasets; extra == "train"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# Spiral — a reversible looped transformer (~1M core)

> **One small block, iterated. Each pass folded into a new subspace. Reversible in time.**
> Spiral tests a single hypothesis: **a ~1M-parameter block iterated N times can recover the *reasoning depth*
> of a ~125M model** — online-distilled from a teacher (`SmolLM-135M`) that is also its baseline. Developed
> inside [Cortex](https://github.com/Xpitfire)/Dentate; usable standalone.

## The idea in one paragraph
A regular transformer stacks *D distinct* layers. Spiral has **one** shared block (`f_θ`, ~1M params) and runs
it for **N iterations**. To stop every iteration doing the *same* thing (provably weaker than untied depth —
Xu & Sato 2024), each pass is **folded into a different latent subspace** by an orthogonal rotation `Q_k` and
conditioned on the iteration index by **AdaLN-Zero**. The embedded input is **re-injected every step**
(DEQ/Huginn). The update is a **momentum** step, making each iteration *exactly invertible* — the network can
reconstruct earlier states **backward in time** with O(1) activation memory. Stability ("must not diverge")
comes from three independent guarantees: the rotation preserves norm, the block is spectrally contractive
(Lip < 1, Banach), and the AdaLN gate is **zero at init** (the loop starts as the identity). Training is
**online knowledge distillation**: the teacher completes random subsequences from a data pool, and the student
learns those distributions on the fly (sequence-level warm-start → on-policy reverse-KL GKD).

## Install
```bash
pip install -e .            # core (torch)
pip install -e '.[train]'   # + transformers/datasets for standalone distillation
pip install -e '.[dev]'     # + pytest/ruff
```

## Use
```python
from spiral import SpiralConfig, build_model
cfg = SpiralConfig(d_model=256, mean_iters=32, reproject="orthogonal", reversible="momentum",
                   teacher_model="smollm-135m")
model = build_model(cfg)            # lazy torch import; moves to cfg.device (mps)
logits = model(input_ids, n_iters=32)
model.num_params(core_only=True)    # 1,023,553
```
A run is reproduced 1:1 from its `SpiralConfig` (`cfg.to_dict()` / `SpiralConfig.from_dict(...)`).

## Validate first, trust numbers later
`pytest` runs the **architecture-intrinsic invariants** that must hold before any metric is believed
(see [docs/VALIDATION.md](docs/VALIDATION.md)): exact reversibility, identity-at-init, no output explosion over
many iterations, and finite/non-vanishing/non-exploding gradients.
```bash
pytest -q
```

## Measured (this commit)
| Quantity | Value |
|---|---|
| Recurrent **core** params | **1,023,553 (1.024 M)** |
| Total (vocab 49,152, factorized E=128) | 7.315 M |
| Exact reversibility (fp64) | `max|Δh| = 1.6e-14`, `max|Δv| = 0` |
| AdaLN-Zero force at init | `0.0` |
| Devices | CPU, MPS |

## Docs
- [docs/REQUIREMENTS.md](docs/REQUIREMENTS.md) — F/N spec: architecture, training loop, target, decisions.
- [docs/MATH.md](docs/MATH.md) — iteration map, exact reversible inverse, non-divergence, expressivity limits, and
  *why momentum beyond memory-saving* (representational class, γ knob, gradient-stability lever).
- [docs/STABILITY.md](docs/STABILITY.md) — **deep-loop stability**: the non-normal N-step Jacobian `∏J`, why deeper
  breaks, the gate-driven root cause (measured + ablated), and the non-contractive fixes (gate cap · JSRR · random depth).
- [docs/SPIRAL_LATENT.md](docs/SPIRAL_LATENT.md) / [docs/SPIRAL_MEM.md](docs/SPIRAL_MEM.md) — the memory variants.
- [docs/SOURCES.md](docs/SOURCES.md) — verified references (arXiv IDs), grouped by design question (incl. §C.2
  gradient-dynamics and §E.2 memory-variant internals).
- [docs/VALIDATION.md](docs/VALIDATION.md) — the test/monitor-before-you-trust protocol (reconstruction,
  gradient health, theory-matches-practice).
- [docs/PLAN.md](docs/PLAN.md) — phased build (P0 scaffold ✅ → P6), risk register.

## Honesty
~1M coherence needs a **narrow distribution** (TinyStories 2023); broad-pool 100× knowledge compression is
unsupported (capacity gap). Spiral targets **narrow per-domain competence first** and treats broad-pool 100× as
the research hypothesis under test — not a promise. The bet is on *reasoning depth*, where looping, not
parameters, is the bottleneck. License: Apache-2.0.
