Metadata-Version: 2.4
Name: coala_embeddings
Version: 0.1.2
Summary: Coarse-to-fine Optimized Asymmetric Loss Architecture for fast and efficient embedding retrieval.
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: torch
Requires-Dist: faiss-cpu
Requires-Dist: beir

# Embedding Compression Benchmark

All compression techniques evaluated with 4-bit quantization.

## Method Overview

| Method | When it Trains | Training Hardware / Time | Data Needed |
| :--- | :--- | :--- | :--- |
| **MRL** | During model training | Full training pipeline | Base training data |
| **SVD** | Post-hoc | CPU | ~40k samples |
| **Proposed** | Post-hoc | GPU (<20s on A100) | ~40k samples |

---

## 1. Ultra-Low Dimensions (32 dim)
Ours is consistently SOTA and surpassing previous baselines by a significant measure.

| Dataset | Metric | MRL | SVD | Proposed | Gain vs. SVD |
| :--- | :--- | :---: | :---: | :---: | :---: |
| **MXBAI FIQA** | Recall@200 | 0.41 | 0.59 | **0.63** | **+6.8%** |
| **NOMIC FIQA** | Recall@200 | 0.42 | 0.47 | **0.55** | **+17.0%** |
| **NOMIC TREC-COVID** | Recall@200 | 0.12 | 0.11 | **0.14** | **+27.3%** |

---

## 2. Low Dimensions (64 dim)
Although the benefit is less then at 32 dim, ours is still significantly better then MRL and SVD.

| Dataset | Metric | MRL | SVD | Proposed | Gain vs. SVD |
| :--- | :--- | :---: | :---: | :---: | :---: |
| **MXBAI FIQA** | Recall@200 | 0.57 | **0.71** | **0.71** | **0.0%** |
| **NOMIC FIQA** | Recall@200 | 0.60 | 0.62 | **0.65** | **+4.8%** |
| **NOMIC TREC-COVID** | Recall@200 | **0.17** | 0.15 | **0.17** | **+13.3%** |

---

## 3. Full-Stage Retrieval (NDCG@10)
Pipeline: 32-dim IVF index to get top 200 and then 256 dim retrieves top 10. 

Once again, ours is SOTA while retaining over 94% of the overall quality.

| Dataset | Original (Full) | MRL | SVD | Proposed | % Retention of Original |
| :--- | :---: | :---: | :---: | :---: | :---: |
| **MXBAI FIQA** | 0.45 | 0.34 | 0.40 | **0.43** | **95.6%** |
| **NOMIC FIQA** | 0.37 | 0.30 | 0.31 | **0.35** | **94.1%** |
| **NOMIC TREC-COVID** | 0.70 | **0.70** | 0.69 | **0.70** | **100.0%** |
