Metadata-Version: 2.4
Name: doppix
Version: 0.1.0
Summary: Modular perceptual-hash based image deduplication library.
Author: Tannous Geagea
License: MIT
Project-URL: Homepage, https://github.com/tannousgeagea/doppix
Project-URL: Repository, https://github.com/tannousgeagea/doppix
Project-URL: Bug Tracker, https://github.com/tannousgeagea/doppix/issues
Keywords: image,deduplication,perceptual-hash,computer-vision,clustering
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Pillow>=9.0
Requires-Dist: imagehash>=4.3
Requires-Dist: tqdm>=4.60
Requires-Dist: matplotlib>=3.5
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Dynamic: license-file

# Doppix 🔍

**Modular, production-grade image deduplication via perceptual hashing.**

Doppix finds exact and near-duplicate images in large collections — fast, with zero ML inference overhead. It clusters images by perceptual similarity and gives you full control over what to do with duplicates: visualise, move, or delete them.

---

## Features

| | |
|---|---|
| 🔀 **4 hash algorithms** | `ahash`, `phash`, `dhash`, `whash` — swap at any time |
| ⚡ **Parallel hashing** | Thread-pool I/O for large datasets |
| 📊 **Rich result object** | `ClusterSet` with stats, per-cluster metadata, summary |
| 🖼️ **Contact sheets** | One PNG per cluster for easy visual review |
| 📦 **Safe IO** | Move duplicates, delete them, or just `dry_run` first |
| 🔧 **CLI + Python API** | Use as a script or import into your own pipeline |
| 🧩 **Modular** | Every layer is independently importable and replaceable |

---

## Installation

```bash
pip install doppix
# or from source:
git clone https://github.com/tannousgeagea/doppix.git
cd doppix && pip install -e .
```

---

## Quick start

### Python API

```python
from doppix import Doppix

dp = Doppix(threshold=5, hasher="phash", num_workers=8)
result = dp.run(
    "./my_photos",
    recursive=True,
    visualize=True,          # save cluster contact sheets
    transfer=True,           # move duplicates to archive/
    destination_folder="./archive",
)
print(result.summary())
```

One-liner shortcut (no IO side-effects):

```python
from doppix import Doppix

result = Doppix.find_duplicates("./my_photos", threshold=3)
print(f"Found {result.num_duplicates} duplicates in {result.num_clusters} clusters")
```

### CLI

```bash
# Basic — print summary only
doppix ./my_photos

# Use perceptual hash, scan recursively, save contact sheets
doppix ./my_photos --hasher phash --recursive --visualize

# Move duplicates to a separate folder (preview first)
doppix ./my_photos --transfer --destination ./archive --dry-run
doppix ./my_photos --transfer --destination ./archive

# Permanently delete duplicates (use with care!)
doppix ./my_photos --delete

# All options
doppix --help
```

---

## Concepts

### Threshold

The `threshold` controls how different two images can be (in [Hamming distance](https://en.wikipedia.org/wiki/Hamming_distance)) and still be grouped together.

| Threshold | Catches |
|-----------|---------|
| 0 | Byte-identical images only |
| 1–3 | Near-identical (minor compression artefacts) |
| 5–8 | Visually similar (slight crops, brightness changes) |
| 10+ | Broadly similar (same subject, different conditions) |

### Hash algorithms

| Name | Speed | Robustness | Best for |
|------|-------|------------|---------|
| `ahash` | ⚡⚡⚡ | Medium | Large datasets, good default |
| `phash` | ⚡⚡ | High | Minor edits, JPEG re-compression |
| `dhash` | ⚡⚡⚡ | Medium | Cropped/shifted images |
| `whash` | ⚡ | High | Perceptually accurate matching |

### ClusterSet

```python
result.num_clusters      # int — number of distinct groups
result.num_duplicates    # int — images that are not representatives
result.duplicate_ratio   # float — fraction of total that are duplicates
result.summary()         # pretty-printed stats string

for cluster in result:
    print(cluster.representative)  # path to the kept image
    print(cluster.duplicates)      # list of paths to deduplicate
    print(cluster.size)            # total images in this group
```

---

## Architecture

```
doppix/
├── hashing/
│   └── hashers.py       # BaseHasher + AverageHasher, PHasher, DHasher, WHasher
├── clustering/
│   ├── models.py        # Cluster, ClusterSet data classes
│   └── cluster.py       # cluster_images() — parallel hash + greedy scan
├── io/
│   ├── loader.py        # collect_images() — recursive directory scan
│   └── transfer.py      # transfer_images(), delete_duplicates()
├── visualization/
│   └── plot.py          # visualize_clusters() — matplotlib contact sheets
├── pipeline/
│   └── runner.py        # Doppix — high-level orchestrator
└── __main__.py          # CLI entry point
```

Each layer is independently importable:

```python
from doppix.hashing import get_hasher
from doppix.clustering import cluster_images
from doppix.io import collect_images, transfer_images
from doppix.visualization import visualize_clusters
```

---

## Advanced usage

### Custom hasher

```python
from doppix.hashing.hashers import BaseHasher
import imagehash
from PIL import Image

class MyHasher(BaseHasher):
    name = "myhash"
    def compute(self, path):
        with Image.open(path) as img:
            return imagehash.colorhash(img)

from doppix.clustering import cluster_images
result = cluster_images(image_paths, hasher=MyHasher(), threshold=3)
```

### Pipeline without the runner

```python
from doppix.io import collect_images
from doppix.clustering import cluster_images
from doppix.io import transfer_images

images = collect_images("/data/raw", recursive=True)
result = cluster_images(images, threshold=5, hasher_name="phash", num_workers=16)
transfer_images(result, "/data/archive", dry_run=False)
```

### Multiple folders

```python
from doppix import Doppix

folders = ["amk_bunker_impurity", "site_b_waste", "site_c_material"]
dp = Doppix(threshold=3, hasher="phash")

for folder in folders:
    dp.run(
        f"/data/{folder}",
        visualize=True,
        transfer=True,
        output_dir=f"/data/{folder}/clusters",
        destination_folder=f"/data/{folder}/archive",
    )
```

---

## Running tests

```bash
pip install -e ".[dev]"
pytest
```

---

## Roadmap

- [ ] ANN-backed clustering (FAISS) for 1M+ image datasets  
- [ ] GPU hashing support  
- [ ] JSON/CSV export of cluster results  
- [ ] Web UI for cluster review  
- [ ] Docker image  

---

## License

MIT © Tannous Geagea
