Metadata-Version: 2.5
Name: nanoscope-lab
Version: 0.6.1
Summary: See what your language model learns
Project-URL: Homepage, https://github.com/almajd3713/nanoscope
Project-URL: Source, https://github.com/almajd3713/nanoscope
Project-URL: Issues, https://github.com/almajd3713/nanoscope/issues
Author: almajd3713
License-Expression: MIT
License-File: LICENSE
Keywords: education,interpretability,language-models,pytorch,research,transformers
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.13,>=3.10
Requires-Dist: datasets<5,>=3.0
Requires-Dist: huggingface-hub<2,>=0.27
Requires-Dist: jsonschema<5,>=4.20
Requires-Dist: matplotlib<4,>=3.8
Requires-Dist: numpy<3,>=1.26
Requires-Dist: scipy<2,>=1.11
Requires-Dist: tiktoken<1,>=0.8
Requires-Dist: tokenizers<1,>=0.20
Requires-Dist: tomli-w>=1.0
Requires-Dist: tomli>=2; python_full_version < '3.11'
Requires-Dist: torch<3,>=2.2
Provides-Extra: dev
Requires-Dist: httpx<1,>=0.27; extra == 'dev'
Requires-Dist: hypothesis<7,>=6.100; extra == 'dev'
Requires-Dist: jsonschema<5,>=4.20; extra == 'dev'
Requires-Dist: libcst<2,>=1.1; extra == 'dev'
Requires-Dist: nbformat<6,>=5.10; extra == 'dev'
Requires-Dist: pyright>=1.1.390; extra == 'dev'
Requires-Dist: pytest-cov<8,>=6; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.9; extra == 'dev'
Provides-Extra: graph
Requires-Dist: libcst<2,>=1.1; extra == 'graph'
Provides-Extra: notebook
Requires-Dist: marimo>=0.9; extra == 'notebook'
Provides-Extra: server
Requires-Dist: fastapi<1,>=0.115; extra == 'server'
Requires-Dist: libcst<2,>=1.1; extra == 'server'
Requires-Dist: ruff<1,>=0.9; extra == 'server'
Requires-Dist: uvicorn[standard]<1,>=0.30; extra == 'server'
Requires-Dist: watchfiles<2,>=0.24; extra == 'server'
Provides-Extra: wandb
Requires-Dist: wandb<1,>=0.19; extra == 'wandb'
Description-Content-Type: text/markdown

# nanoscope

A web app for learning how small language models work and for testing changes to them properly.
You build a model in lessons, edit it as code or as a graph, train it, and compare it with another
model with confidence intervals instead of a single loss number.

It has three parts that share one code path:

- **Lessons** that build a model from a bigram up to a modern transformer block. Each lesson has a
  check that says why it fails, and passing one unlocks the bigger blocks.
- **A model page** where one file is both code and a graph. Drag a block onto another to swap it and
  the file changes by one line.
- **Experiments** with seeds, paired comparisons, 95% intervals, preregistered studies and baselines
  for the shipped models.

## Installation

```bash
docker compose up
docker compose logs api | grep login
```

Open the link that grep prints. The app is at `http://127.0.0.1:8765` and trains on one CPU worker.
For an NVIDIA GPU use `docker compose --profile gpu up`.

Without Docker, on Python 3.10 to 3.12:

```bash
pip install "nanoscope-lab[server]"
nanoscope serve --worker cpu
```

That serves on `http://127.0.0.1:8000`. The login token lets whoever has it run code on your
machine, so keep the server on localhost or behind an SSH tunnel. Setup for GPUs, remote use and
backups is in [docs/deploy.md](docs/deploy.md).

## Usage

The Learn, Tinker, Research and Extend switch hides controls you don't need yet. It doesn't
change what runs. [docs/gui.md](docs/gui.md) describes each screen. The app calls the same code as
Python and the command line:

```python
from nanoscope import compare, run

result = run(MyModel, preset="tinystories-5min")
compare(result, "gpt2")
```

```bash
nanoscope run models/my_model.py:MyModel --seeds 3
nanoscope compare modern gpt2 --preset tinystories-5min
nanoscope --help
```

[docs/research.md](docs/research.md) covers studies and [docs/blocks.md](docs/blocks.md) covers
writing your own blocks.

## Develop

```bash
uv sync --all-extras
make check        # what CI runs
make web          # build the app into the package
pnpm -C web dev   # app with hot reload
```

## Data

Tokens come from [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories)
(CDLA-Sharing-1.0) and [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)
(ODC-By 1.0), cached under `~/.nanoscope/data`.

Architecture notes are in [docs/architecture.md](docs/architecture.md). MIT licensed.
