Metadata-Version: 2.4
Name: mda-memory
Version: 0.1.2
Summary: Online associative memory for LLMs
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: torch
Requires-Dist: rich
Requires-Dist: prompt_toolkit
Requires-Dist: python-dotenv
Requires-Dist: requests
Requires-Dist: anthropic
Requires-Dist: ollama
Requires-Dist: chromadb
Requires-Dist: sentence-transformers
Requires-Dist: tqdm
Requires-Dist: matplotlib
Requires-Dist: ijson
Requires-Dist: deep-translator
Requires-Dist: langdetect
Dynamic: license-file

# MDA — Modular Dynamic Architecture

**Online associative memory for LLMs. Learns during inference. No backpropagation. No GPU required.**

[![PyPI](https://img.shields.io/pypi/v/mda-memory.svg)](https://pypi.org/project/mda-memory/)
[![License: SSPL](https://img.shields.io/badge/License-SSPL-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/)

---

## What is MDA?

Large language models can reason but cannot remember. RAG partially addresses this — but it cannot update during a conversation, and it cannot learn from the conversation itself.

MDA fills precisely these gaps.

It encodes knowledge as **256-dimensional Holographic Distributed Representations (HDRs)**, connects concepts through a **sparse synapse graph**, and retrieves context by activating entity networks — not by text-chunk similarity search. New knowledge is integrated immediately, without rebuilding any index.

**MDA is not a RAG replacement. It is the persistent learning layer that RAG and LLMs are missing.**

---

## Key Properties

- **Token-free** — no tokenizer, no vocabulary
- **Attention-free** — no transformer encoder required
- **Online learning** — learns during inference via the Oja rule
- **No catastrophic forgetting** — entities are independent; new knowledge never overwrites old
- **CPU-only** — runs entirely on numpy, no GPU required
- **Model-agnostic** — works with Ollama, OpenAI, Anthropic, or any LLM

---

## Benchmark Results

Evaluated against a strong RAG baseline (bge-large-en-v1.5 + ChromaDB, top-6 retrieval) across 80 questions spanning 8 cognitive categories:

| Category | RAG | MDA | Δ |
|---|---|---|---|
| ATOMIC_RECALL | 100% | 85% | −15% |
| MULTI_HOP | 90% | 90% | 0% |
| CROSS_DOCUMENT | 80% | 70% | −10% |
| REASONING | 70% | 90% | **+20%** |
| INCREMENTAL_LEARNING | 0% | 60% | **+60%** |
| NOISE_RESISTANCE | 100% | 100% | 0% |
| MEMORY_COMPRESSION | 90% | 70% | −20% |
| BOUNDARY | 100% | 100% | 0% |
| **OVERALL** | **78.8%** | **83.1%** | **+4.3%** |

MDA uses **3.1× less context per query** than RAG while achieving higher overall accuracy.

**Long-context retention (200 turns):** RAG 0% — MDA 92%.

---

## Quick Start

```bash
pip install mda-memory
```

Or clone for development:

```bash
git clone https://github.com/Rangle2/mda
cd mda
pip install -e .
```

### Basic Usage

```python
from mda import MDA

memory = MDA()

# Teach facts
memory.learn("The capital of Veloria is Aranthos.")
memory.learn("Aranthos was founded by Queen Seraphel in 412 AE.")

# Retrieve context — returns memory string ready for LLM injection
context = memory.context_for("Who founded the capital?")
print(context)
# → [MEMORY] Aranthos was founded by Queen Seraphel in 412 AE.
```

### With Ollama

```bash
# Start Ollama with any model
ollama pull qwen3:4b

# Run MDA CLI
mda --model qwen3:4b
```

### With Anthropic (Claude)

```bash
# Set your API key
export ANTHROPIC_API_KEY=your_key_here

# Run MDA CLI with Claude
mda --model claude-haiku-4-5-20251001 --provider anthropic
```

### Running Benchmarks

```bash
# Main benchmark — Ollama
python benchmark/benchmark.py --model qwen3:4b

# Main benchmark — Anthropic
python benchmark/benchmark.py --model claude-haiku-4-5-20251001 --provider anthropic

# Long-context retention benchmark — Ollama
python benchmark/long_context_benchmark.py --model qwen3:4b

# Long-context retention benchmark — Anthropic
python benchmark/long_context_benchmark.py --model claude-haiku-4-5-20251001 --provider anthropic
```

---

## Project Structure

```
mda/
├── mda/
│   ├── __init__.py        # Public API — exposes MDA class
│   ├── mda.py             # MDA class implementation
│   ├── core/
│   │   ├── bind.py        # HDR ops: bind, unbind, normalize, cosine
│   │   ├── encoder.py     # HolisticEncoder: text → 256-dim vector
│   │   ├── entity.py      # Entity: v, r, h, W, neurons, synapses, senses
│   │   ├── neuron.py      # Neuron (Oja rule), Synapse (Hebbian)
│   │   └── registry.py    # EntityRegistry: surface → Entity lookup
│   ├── inference/
│   │   ├── associative.py # AssociativeChain: multi-hop traversal
│   │   ├── broca.py       # BrocaModule: W-hybrid context scoring
│   │   ├── reasoning.py   # ReasoningEngine: cross-entity inference
│   │   └── memory.py      # ConversationMemory: multi-turn context
│   ├── training/
│   │   └── checkpoint.py  # save/load: float32, atomic write
│   └── integrations/
│       ├── engine.py      # MDAEngine: LLM bridge (Ollama / Anthropic)
│       └── cli.py         # Interactive CLI (Rich + prompt_toolkit)
├── benchmark/
│   ├── benchmark.py       # 80-question main benchmark
│   └── long_context_benchmark.py  # 200-turn retention benchmark
├── tests/                 # pytest test suite
└── pyproject.toml
```

---

## How It Works

### Entity & W Matrix
Every concept is an `Entity` with a 256-dim identity vector `v` and a lazy-initialized Hebbian weight matrix `W`. `W` is `None` until the entity is first activated — memory overhead is proportional to usage.

### Online Learning (Oja Rule)
```
ΔW = η(yxᵀ − y²W)
```
where `x` is the query vector and `y = Wx`. No backpropagation. No gradient descent. Runs in O(d²) per entity per turn.

### AssociativeChain
Query → origin entity → breadth-first synapse traversal (depth 3) → context assembly. Dynamic inhibition prevents context flooding.

### BrocaModule
Facts are scored with a W-hybrid formula:
```
score = 0.35·s_query + 0.45·s_W + 0.20·s_sense
```
Facts below 0.20 are suppressed.

---

## Running Tests

```bash
pytest tests/
```

---

## Roadmap

- [ ] **GPU acceleration** — PyTorch tensor ops, parallel entity traversal
- [ ] **Low-rank W** — W ≈ A×B factorization for higher-dimensional HDRs
- [ ] **`mda.cloud` API** — persistent memory as a service
- [ ] **MDA + RAG hybrid** — offline corpus retrieval + online conversational learning
- [ ] **Real-world corpus evaluation** — beyond fictional benchmark domains
- [ ] **Independent benchmark** — community-constructed evaluation set

---

## License

[SSPL 1.0](LICENSE) — free for research and personal use. Commercial use requires a separate agreement.

For commercial licensing: mert@kairfy.com
