Metadata-Version: 2.5
Name: interp-engine
Version: 1.3.5
Summary: A fast, standardized, and easy to use interpretability engine.
Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
Project-URL: Repository, https://github.com/decoderesearch/interp-engine
Project-URL: Issues, https://github.com/decoderesearch/interp-engine/issues
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Python: <3.14,>=3.11
Requires-Dist: einops
Requires-Dist: numpy>=1.24
Requires-Dist: torch>=1.10
Requires-Dist: transformers>=4.57.1
Provides-Extra: awq
Requires-Dist: accelerate>=1.0; extra == 'awq'
Requires-Dist: gptqmodel>=5.0; extra == 'awq'
Provides-Extra: dev
Requires-Dist: pyright<1.2,>=1.1.411; extra == 'dev'
Requires-Dist: pytest<9,>=8.3.1; extra == 'dev'
Requires-Dist: pyyaml>=6; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16.2; extra == 'dev'
Provides-Extra: parity
Requires-Dist: transformer-lens>=3.0; extra == 'parity'
Provides-Extra: quant
Requires-Dist: accelerate>=1.0; extra == 'quant'
Requires-Dist: kernels<0.17.0,>=0.15.2; extra == 'quant'
Provides-Extra: vllm
Requires-Dist: vllm>=0.27.1; (sys_platform == 'linux') and extra == 'vllm'
Description-Content-Type: text/markdown

# interp-engine

<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ielogo.png" alt="interp-engine logo, a magnifying glass where the handle is a rocket" width="160">
</p>
<p align="center">
  🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
</p>
<p align="center">
  <a href="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml"><img src="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml/badge.svg?branch=main" alt="CI status"></a>
  <a href="https://pypi.org/project/interp-engine/"><img src="https://img.shields.io/pypi/v/interp-engine.svg" alt="PyPI version"></a>
  <a href="LICENSE"><img src="https://img.shields.io/pypi/l/interp-engine.svg" alt="Apache-2.0 license"></a>
  <a href="https://join.slack.com/t/opensourcemechanistic/shared_invite/zt-3z9o0hxjl-MDX9pbATO2qESOazNDLpdQ"><img src="https://img.shields.io/badge/Slack-Open%20Source%20Mechanistic%20Interpretability-4A154B?logo=slack&logoColor=white" alt="Join the Slack"></a>
</p>

`interp-engine` is an interpretability engine that is fast, standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.

<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-benchmark.gif" alt="benchmark between ie and transformers" width="100%">
</p>
<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/toks.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-static on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
</p>
<p align="center">
  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/interp-engine-demo.gif" alt="interp-engine demo gif" width="100%">
</p>

This repo contains:

1. [`validator/`](validator/), which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
2. [`visualizer-web/`](visualizer-web/), a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.

## Installation

```bash
pip install 'interp-engine[vllm]'  # preferred install: includes vLLM support (CUDA required)
pip install interp-engine          # eager backend only
```

## Simple Usage

```python
from interp_engine import Address, load_model, run_with_cache

# VLLM (default): low VRAM, medium speed, every point, chosen per request
model = load_model("Qwen/Qwen3-8B")

# VLLM-STATIC: high VRAM, high speed, only the points you declare (default resid_post)
# model = load_model("Qwen/Qwen3-8B", backend="vllm-static")

# VLLM-GENERATE: fastest, generation only -- no capture, no steering
# model = load_model("Qwen/Qwen3-8B", backend="vllm-generate")

# EAGER: low VRAM, low speed
# model = load_model("Qwen/Qwen3-8B", backend="eager")

point = Address("resid_post", 10)      # or string: "resid_post.10"
cache = run_with_cache(model, model.to_tokens("Hello, world"), [point])
cache[point]  # [batch, pos, ...]
```

### AI Agents

Add "use interp-engine" to your prompt and let your agent figure it out - everything is fully documented in this repo and open source.

## Supported Points ("Addresses")

`interp-engine` supports 34 standardized points ("Addresses") across architectures: every one of them on the eager backend, 28 of them on vLLM. Check [interp-engine.org](https://interp-engine.org) for the "cheat sheet", or [SUPPORTED_POINTS.md](docs/SUPPORTED_POINTS.md) for a markdown version with the per-backend detail.

## Performance / Speed

vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-static"` gives even higher throughput at the cost of higher VRAM usage. Every column below is capture-capable.

<!-- THROUGHPUT:START -->

<!-- Generated by `python -m benchmarks.report_bench`. Do not edit: rerun the sweep. -->

Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.

One stream (tok/s):

| model                    | eager | vLLM       | vLLM + static taps |
| ------------------------ | ----- | ---------- | ------------------ |
| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |

8 concurrent requests (aggregate tok/s):

| model                    | eager | vLLM        | vLLM + static taps |
| ------------------------ | ----- | ----------- | ------------------ |
| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |

<!-- THROUGHPUT:END -->

`backend="vllm-static"` is opt-in, and serves only the tap set it declared — `static_points="auto"` by default, or a list you name. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.

## Correctness

We verify correctness in two main ways:

1. A test suite that checks results across several models - what each check is designed to catch is in [INTERNALS.md](docs/INTERNALS.md#correctness).
2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at [`validator/`](validator/).

## Why use an Interpretability Engine instead of building from scratch?

1. **Speed**: Get performance without sacrificing correctness.
2. **Standardization + Verification**: Eliminate ambiguity when referring to points, plus a full test suite included.
3. **Faster Dev / Fewer Tokens Used**: You could spend ten million tokens and have your AI write, test, and make production-ready an interpretability engine. Or you could just `pip install interp-engine[vllm]`.

## Development / Contributing

Activate the shared git hooks once per clone — they format staged Python, rebuild the generated files, and run CI's static checks before a push. Details in [CONTRIBUTING.md](CONTRIBUTING.md).

```bash
make hooks    # or: git config core.hooksPath .githooks
```

## Contact

Bugs and feature requests belong in [issues](https://github.com/decoderesearch/interp-engine/issues). For anything else: [johnny@neuronpedia.org](mailto:johnny@neuronpedia.org).

## License

Apache 2.0
