Metadata-Version: 2.4
Name: zynthe
Version: 0.2.5
Summary: Universal knowledge distillation toolkit for text, code, vision, and multimodal models
Project-URL: Homepage, https://github.com/lakshin7/zynthe
Project-URL: Repository, https://github.com/lakshin7/zynthe
Project-URL: Issues, https://github.com/lakshin7/zynthe/issues
Author-email: Lakshin <lakshin7@users.noreply.github.com>
License-Expression: MIT
License-File: LICENSE
Keywords: deep-learning,knowledge-distillation,model-compression,pytorch,quantization,transformers
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: accelerate>=0.23.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: omegaconf>=2.3.0
Requires-Dist: psutil>=5.9.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.5.2
Requires-Dist: torch>=2.1.0
Requires-Dist: tqdm>=4.66.0
Requires-Dist: transformers>=4.34.0
Provides-Extra: all
Requires-Dist: datasets>=2.14.0; extra == 'all'
Requires-Dist: evaluate>=0.4.0; extra == 'all'
Requires-Dist: matplotlib>=3.7.0; extra == 'all'
Requires-Dist: onnx>=1.15.0; extra == 'all'
Requires-Dist: onnxruntime>=1.16.0; extra == 'all'
Requires-Dist: pandas>=2.0.0; extra == 'all'
Requires-Dist: pillow>=9.0; extra == 'all'
Requires-Dist: scikit-learn>=1.3.0; extra == 'all'
Requires-Dist: seaborn>=0.12.0; extra == 'all'
Requires-Dist: torchvision>=0.16.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: black>=23.0; extra == 'dev'
Requires-Dist: flake8>=6.0; extra == 'dev'
Requires-Dist: isort>=5.12; extra == 'dev'
Requires-Dist: mypy>=1.5; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: eval
Requires-Dist: datasets>=2.14.0; extra == 'eval'
Requires-Dist: evaluate>=0.4.0; extra == 'eval'
Requires-Dist: matplotlib>=3.7.0; extra == 'eval'
Requires-Dist: pandas>=2.0.0; extra == 'eval'
Requires-Dist: scikit-learn>=1.3.0; extra == 'eval'
Requires-Dist: seaborn>=0.12.0; extra == 'eval'
Provides-Extra: onnx
Requires-Dist: onnx>=1.15.0; extra == 'onnx'
Requires-Dist: onnxruntime>=1.16.0; extra == 'onnx'
Provides-Extra: vision
Requires-Dist: pillow>=9.0; extra == 'vision'
Requires-Dist: torchvision>=0.16.0; extra == 'vision'
Description-Content-Type: text/markdown

# Zynthé 🧠 ⚡

> **Zynthé** — *From "Synthesize". Knowledge distillation is the process of synthesizing knowledge from a massive, complex model down into a smaller, more focused one.*

**Universal Knowledge Distillation Toolkit**  
A Python library for compressing large language models into smaller, deployable student models — across text, code, vision, and multimodal architectures.

[![PyPI version](https://badge.fury.io/py/zynthe.svg)](https://badge.fury.io/py/zynthe)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

Large transformer models (BERT-110M, GPT-2-124M, ViT-86M) deliver state-of-the-art accuracy but are often too expensive for edge deployment, mobile inference, and cost-sensitive production. **Zynthé** solves this by training a smaller "student" model to replicate the behavior of a larger "teacher" model using a unified, multi-stage distillation pipeline.

---

## 🌟 Key Features

- **Modality Agnostic**: Works out-of-the-box with Text, Code, Vision, VLM, and Multimodal models via an auto-detecting `AdapterRegistry`.
- **Multi-Stage Distillation**: Supports sequential distillation stages:
  - **Logit Distillation** (KD Hinton)
  - **Feature Alignment** (Hidden representations)
  - **Attention Transfer** (Attention patterns)
  - **Structural Similarity** (Cross-layer relationships)
- **Zero-Config Presets**: 7 battle-tested distillation plans (`quick_start`, `balanced`, `compression_max`, etc.).
- **Preflight Analysis**: Validates configurations, hardware compatibility, and estimates memory before training starts.
- **Post-Training Quantization (PTQ)**: Further compress your distilled models to INT8.
- **Rich Visualization**: Auto-generates training curves, distillation gap plots, and metric grids.

---

## 📦 Installation

Install Zynthé via pip:

```bash
# Core installation
pip install zynthe

# With evaluation and plotting dependencies
pip install zynthe[eval]

# For computer vision support
pip install zynthe[vision]
```

---

## 🚀 Quickstart

Zynthé makes complex multi-stage distillation as simple as 5 lines of code:

<p align="center">
  <img src="docs/assets/typing_demo.svg" alt="Zynthé Typing Demo" width="100%">
</p>

---

## 📊 Example Results

**BERT-base (110M) → DistilBERT (66M) on SST-2 (Single NVIDIA T4)**

| Metric | Teacher | Student | Distilled Student |
|--------|---------|---------|-------------------|
| **Accuracy** | 92.1% | ~81.5% (scratch) | **86.8%** |
| **Model Size** | 418 MB | 255 MB | **255 MB** |
| **Quantized (PTQ)** | - | - | **64 MB** (6.5x smaller) |

---

## 🛠️ Architecture

Zynthé uses a modular, plugin-based architecture:

- `DistillationToolkit`: The primary user-facing facade.
- `AdapterRegistry`: Automatically normalizes I/O across HuggingFace architectures.
- `PipelineBuilder`: Constructs deterministic, multi-stage training plans.
- `PreflightAnalyzer`: Catches OOMs and shape mismatches before training.
- `Evaluator` & `ModelComparator`: Side-by-side student vs. teacher analysis.

---

## 📚 Documentation

- [Usage Guide](docs/usage_guide.md) — Full examples for Text, Code, Vision, CLIP, and GPT models.
- [Quickstart Guide](docs/quickstart.md) — Get up and running in minutes.
- [API Reference](docs/api_reference.md) — Comprehensive guide to the 44+ public API symbols.
- [Adapters System](docs/adapters.md) — How to add custom model architectures.
- [Kaggle Examples](examples/kaggle_t4_test.py) — End-to-end notebooks for T4/P100 environments.

---

## 📄 License

This project is licensed under the **MIT License**. 

You are free to use, modify, distribute, and use this software for commercial purposes. See the [LICENSE](LICENSE) file for the full text.

> **Note:** Models distilled using Zynthé inherit the license of their respective teacher models and datasets. Please ensure you comply with the licensing terms of the models you are distilling.