shrinkvec report

glove-100. GloVe word vectors, 1.18M words (ann-benchmarks). PDDL (GloVe).

vectors 1,183,514dimensions 100metric cosinequeries 1,000k 10seed 42
Recommendation for recall@10 ≥ 0.95

tq-mse-3>tq-mse-8 keeps recall@10 at 0.986 with 42 bytes per vector to search (9.5x smaller than float32). The rerank reads more precise data for each candidate, so 146 bytes per vector stay stored; only the searched part has to be in memory. In Postgres: pg_turbovec.

Codec lab: what the codes can do

Every option scans all codes exactly, so recall reflects the compression itself, not an index. Ground truth is exact search in float64. Hollow markers rerank the best candidates with a more precise codec.

Recall@10 against bytes per vector for every compression option0.10.20.30.40.50.60.70.80.91.0102050100200500bytes per vector to search (log scale)recall@10target 0.95Full precision (float32) recall@10 1.000 400 B/vector (1.0x)Half precision (float16) recall@10 0.999 200 B/vector (2.0x)TurboQuant 3-bit (mse), rerank top 100 with TurboQuant 8-bit (mse) recall@10 0.986 42 B/vector (9.5x)8-bit scalar, per-dimension range recall@10 0.981 100 B/vector (4.0x)RaBitQ 4-bit (extended) recall@10 0.878 72 B/vector (5.6x)TurboQuant 4-bit (mse) recall@10 0.855 54 B/vector (7.4x)TurboQuant 3-bit (mse) recall@10 0.737 42 B/vector (9.5x)RaBitQ 1-bit, rerank top 100 with Full precision (float32) recall@10 0.736 24 B/vector (16.7x)TurboQuant 4-bit (prod) recall@10 0.631 55 B/vector (7.3x)TurboQuant 2-bit (mse) recall@10 0.539 29 B/vector (13.8x)1 sign bit per dimension, Hamming distance, rerank top 100 with Full precision (float32) recall@10 0.348 13 B/vector (30.8x)RaBitQ 1-bit recall@10 0.329 24 B/vector (16.7x)1 sign bit per dimension, Hamming distance recall@10 0.136 13 B/vector (30.8x)halfvectq-mse-3>tq-mse-8int8rabitq-4tq-mse-4tq-mse-3rabitq-1>float32tq-prod-4tq-mse-2binary>float32rabitq-1binarypgvector typesRaBitQTurboQuantOther+ rerankbest per size
pipelinewhat it isrecall@10true top-10 in candidatesbytes/vector searchedbytes/vector storedvs float32in Postgres
float32Full precision (float32)1.0004004001.0xpgvector vector
halfvecHalf precision (float16)0.9992002002.0xpgvector halfvec
int88-bit scalar, per-dimension range0.9811001004.0xnot in pgvector
tq-mse-4TurboQuant 4-bit (mse)0.855 ± 0.00354547.4xpg_turbovec
tq-prod-4TurboQuant 4-bit (prod)0.631 ± 0.00355557.3xpg_turbovec
tq-mse-3TurboQuant 3-bit (mse)0.737 ± 0.00242429.5xpg_turbovec
tq-mse-2TurboQuant 2-bit (mse)0.539 ± 0.003292913.8xpg_turbovec
rabitq-4RaBitQ 4-bit (extended)0.878 ± 0.00172725.6xVectorChord vchordrq
rabitq-1RaBitQ 1-bit0.329 ± 0.003242416.7xVectorChord vchordrq
binary1 sign bit per dimension, Hamming distance0.136131330.8xpgvector binary_quantize
tq-mse-3>tq-mse-8TurboQuant 3-bit (mse), rerank top 100 with TurboQuant 8-bit (mse)0.986 ± 0.0000.999421469.5xpg_turbovec
rabitq-1>float32RaBitQ 1-bit, rerank top 100 with Full precision (float32)0.736 ± 0.0010.7362442416.7xVectorChord vchordrq
binary>float321 sign bit per dimension, Hamming distance, rerank top 100 with Full precision (float32)0.3480.3481341330.8xpgvector binary_quantize

Reproduce

pip install 'shrinkvec[pg,datasets]'
shrinkvec lab --dataset glove-100 --repeats 3 --out docs/results/glove-100