Metadata-Version: 2.4
Name: shrinklens
Version: 0.1.0
Summary: Quantize a real image classifier FP32->FP16->INT8->INT4 and print real measured size/accuracy/latency numbers.
Author: Vichruth M
License-Expression: MIT
License-File: LICENSE
Keywords: edge-ai,model-compression,pytorch,quantization,transformers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Requires-Dist: accelerate>=0.30
Requires-Dist: bitsandbytes>=0.43
Requires-Dist: datasets>=2.19
Requires-Dist: huggingface-hub>=0.23
Requires-Dist: pillow>=10.0
Requires-Dist: rich>=13.0
Requires-Dist: torch>=2.2
Requires-Dist: torchvision
Requires-Dist: transformers>=4.40
Description-Content-Type: text/markdown

<div align="center">

# ShrinkLens

**See exactly what compression costs a model — in real megabytes, real accuracy, and real milliseconds.**

[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)
[![Numbers: 100% real](https://img.shields.io/badge/numbers-100%25%20real-brightgreen.svg)](#the-one-rule-this-tool-runs-on)

`pip install shrinklens` → point it at a real image classifier → watch it get
actually quantized, FP32 → FP16 → INT8 → INT4, with every size, accuracy, and
latency number measured live on your own machine.

</div>

---

## Table of contents

- [Quickstart](#quickstart)
- [The one rule this tool runs on](#the-one-rule-this-tool-runs-on)
- [Sample output](#sample-output)
- [What it does](#what-it-does)
- [Why a Vision Transformer, not ResNet/MobileNet](#why-a-vision-transformer-not-resnetmobilenet)
- [Requirements](#requirements)
- [Development](#development)
- [Roadmap](#roadmap)
- [License](#license)

## Quickstart

```bash
pip install shrinklens
shrinklens
```

That's it. No config file, no API key, no dataset to download by hand — the
first run pulls a real ImageNet-pretrained Vision Transformer and a real
labeled image set, quantizes the model four ways, and prints a report.

```bash
shrinklens --samples 200      # more images = a tighter accuracy estimate
shrinklens --device cpu       # FP32/FP16 only — INT8/INT4 need a CUDA GPU
shrinklens --debug            # full traceback on failure, for bug reports
```

## The one rule this tool runs on

> **Every number this tool prints comes from a real model actually being
> quantized and actually run — on your machine, right now.**

No cached numbers. No simulated hardware dial. No mocked accuracy. `pip
install` it and verify that for yourself: the model streams from the Hugging
Face Hub, each quantized variant is a real artifact sitting in GPU memory, the
accuracy is scored against live images, and the latency is timed with
`time.perf_counter()` around real forward passes. If a number here is wrong,
it's a bug, not a placeholder — [open an issue](issues).

## Sample output

Real output, unedited, from a `shrinklens --samples 200` run on an RTX 4050
laptop GPU (6GB VRAM):

```
                    Real measured results (200 live images)
┏━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Level ┃ Size (MB) ┃ Accuracy ┃ Latency (ms) ┃ Peak GPU mem  ┃ Fits on        ┃
┃       ┃           ┃          ┃              ┃ (MB)          ┃                ┃
┡━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ FP32  │     330.2 │    81.0% │         9.27 │         346.4 │ Raspberry Pi 4 │
│ FP16  │     165.1 │    81.0% │         3.90 │         185.2 │ Raspberry Pi 4 │
│ INT8  │      85.0 │    81.0% │        31.01 │         107.6 │ Raspberry Pi 4 │
│ INT4  │      44.2 │    80.0% │        14.59 │          77.9 │ Raspberry Pi 4 │
└───────┴───────────┴──────────┴──────────────┴───────────────┴────────────────┘

MONEYSHOT: sample #49 -- correct at FP16, wrong at INT4. Model shrank 3.7x
(165.1MB -> 44.2MB), overall accuracy 81.0% (FP16) -> 80.0% (INT4), now
Raspberry Pi 4 (4GB) (1500 MB budget).
```

Two things worth calling out, both real and left in on purpose instead of
smoothed over:

- **INT8 is slower than INT4 here.** `bitsandbytes` 8-bit matmul carries more
  per-call dequantization overhead than its 4-bit (NF4) path on this GPU and
  batch size. Counterintuitive, but that's what the hardware actually does.
- **The "moneyshot" sample isn't guaranteed on every run.** A well-tuned INT4
  method (NF4) can hold accuracy remarkably well on small batches — finding a
  real image that flips from correct to wrong took ~200 samples, not the
  first 40. ShrinkLens reports whichever is true, not whichever is punchier.

## What it does

1. Loads [`google/vit-base-patch16-224`](https://huggingface.co/google/vit-base-patch16-224),
   a real Vision Transformer pretrained on ImageNet-1k.
2. Produces four real quantized variants of it:

   | Level | Method |
   |---|---|
   | **FP32** | full-precision baseline |
   | **FP16** | native PyTorch half precision |
   | **INT8** | `bitsandbytes` 8-bit quantization |
   | **INT4** | `bitsandbytes` NF4 4-bit quantization |

3. Evaluates every variant on the same batch of real photos from
   [Imagenette](https://github.com/fastai/imagenette) (a 10-class real-photo
   subset of ImageNet), measuring:
   - **Size (MB)** — actual parameter + buffer byte count of the loaded model
   - **Accuracy** — top-1 accuracy on the live image batch
   - **Latency (ms)** — average per-image inference time on your hardware
   - **Peak GPU memory (MB)** — measured via `torch.cuda.max_memory_allocated`
4. Maps the measured size to a real device memory budget (Raspberry Pi 4,
   Jetson Nano, Nothing Phone 3a Pro, or a 6GB-VRAM laptop GPU) and reports
   which tier it now fits in.
5. Surfaces the "moneyshot": the first real image that's classified correctly
   at FP16 but wrong at INT4, alongside the exact size and accuracy delta.

```
 HuggingFace ViT checkpoint
          │
          ▼
   ┌────────────┐    ┌────────────┐    ┌─────────────┐    ┌─────────────┐
   │    FP32    │    │    FP16    │    │    INT8     │    │    INT4     │
   │  baseline  │    │ torch .half│    │ bitsandbytes│    │ bitsandbytes│
   └─────┬──────┘    └─────┬──────┘    └──────┬──────┘    └──────┬──────┘
         │                 │                  │                  │
         └─────────────────┴────────┬─────────┴──────────────────┘
                                     ▼
                    same real Imagenette images, every level
                                     ▼
                 size (MB) · accuracy · latency (ms) · GPU mem (MB)
                                     ▼
                     device-tier fit (Pi 4 / Jetson / phone)
```

## Why a Vision Transformer, not ResNet/MobileNet

Real INT4 quantization libraries (`bitsandbytes`, `optimum`) only give genuine
gains on `nn.Linear`-heavy architectures. There's no honest off-the-shelf INT4
path for a conv-heavy CNN like ResNet or MobileNet — the tooling just isn't
there yet. A ViT is Linear-heavy throughout, so every quantization level here
is backed by an existing, well-supported library path. Nothing hand-rolled,
nothing approximated.

## Requirements

- Python 3.10+
- An NVIDIA GPU for INT8/INT4 (`bitsandbytes` requires CUDA) — FP32/FP16 also
  run on CPU via `--device cpu`

## Development

```bash
git clone https://github.com/vichruth/QuantLens.git
cd QuantLens
python -m venv .venv && source .venv/bin/activate
pip install -e .
shrinklens
```

## Roadmap

ShrinkLens v0.1 (this repo) is the engine: a real, installable CLI. Next up —
an interactive web visualizer built directly on top of this same engine (no
rewrite), showing the same FP16-correct/INT4-wrong moment live in a browser
instead of a terminal. Not started yet; this CLI ships first.

## License

[MIT](LICENSE) © Vichruth M — [@vichruth](https://github.com/vichruth)
