shrinkvec report

dbpedia-openai-1536. DBpedia entities embedded with OpenAI text-embedding-ada-002. MIT.

vectors 100,000dimensions 1536metric cosinequeries 1,000k 10seed 42
Recommendation for recall@10 ≥ 0.95

VectorChord vchordrq, codes only (rerank_in_table), vchordrq.probes=20: recall@10 0.957, p50 1.69 ms, index 30.6 MiB (321 bytes per vector). The smallest index that meets 0.95.

CREATE EXTENSION IF NOT EXISTS vchord CASCADE;
CREATE INDEX ON items USING vchordrq (embedding vector_cosine_ops) WITH (options = $$
rerank_in_table = true
residual_quantization = true
[build.internal]
lists = [200]
spherical_centroids = true
$$);
SET vchordrq.probes = '20';
SELECT id FROM items ORDER BY embedding <=> $1 LIMIT 10;

Codec lab: what the codes can do

Every option scans all codes exactly, so recall reflects the compression itself, not an index. Ground truth is exact search in float64. Hollow markers rerank the best candidates with a more precise codec.

Recall@10 against bytes per vector for every compression option0.70.80.91.02005001,0002,0005,000bytes per vector to search (log scale)recall@10target 0.95Full precision (float32) recall@10 1.000 6,144 B/vector (1.0x)RaBitQ 1-bit, rerank top 100 with Full precision (float32) recall@10 0.999 200 B/vector (30.7x)Half precision (float16) recall@10 0.999 3,072 B/vector (2.0x)TurboQuant 3-bit (mse), rerank top 100 with TurboQuant 8-bit (mse) recall@10 0.996 580 B/vector (10.6x)8-bit scalar, per-dimension range recall@10 0.994 1,536 B/vector (4.0x)1 sign bit per dimension, Hamming distance, rerank top 100 with Full precision (float32) recall@10 0.992 192 B/vector (32.0x)RaBitQ 4-bit (extended) recall@10 0.972 776 B/vector (7.9x)TurboQuant 4-bit (mse) recall@10 0.953 772 B/vector (8.0x)TurboQuant 3-bit (mse) recall@10 0.916 580 B/vector (10.6x)TurboQuant 2-bit (mse) recall@10 0.850 388 B/vector (15.8x)RaBitQ 1-bit recall@10 0.834 200 B/vector (30.7x)TurboQuant 4-bit (prod) recall@10 0.817 772 B/vector (8.0x)1 sign bit per dimension, Hamming distance recall@10 0.707 192 B/vector (32.0x)float32rabitq-1>float32halfvectq-mse-3>tq-mse-8int8rabitq-4tq-mse-4tq-mse-3tq-mse-2rabitq-1tq-prod-4binarypgvector typesRaBitQTurboQuantOther+ rerankbest per size
pipelinewhat it isrecall@10true top-10 in candidatesbytes/vector searchedbytes/vector storedvs float32in Postgres
float32Full precision (float32)1.0006,1446,1441.0xpgvector vector
halfvecHalf precision (float16)0.9993,0723,0722.0xpgvector halfvec
int88-bit scalar, per-dimension range0.9941,5361,5364.0xnot in pgvector
tq-mse-4TurboQuant 4-bit (mse)0.953 ± 0.0017727728.0xpg_turbovec
tq-prod-4TurboQuant 4-bit (prod)0.817 ± 0.0037727728.0xpg_turbovec
tq-mse-3TurboQuant 3-bit (mse)0.916 ± 0.00258058010.6xpg_turbovec
tq-mse-2TurboQuant 2-bit (mse)0.850 ± 0.00038838815.8xpg_turbovec
rabitq-4RaBitQ 4-bit (extended)0.972 ± 0.0017767767.9xVectorChord vchordrq
rabitq-1RaBitQ 1-bit0.834 ± 0.00320020030.7xVectorChord vchordrq
binary1 sign bit per dimension, Hamming distance0.70719219232.0xpgvector binary_quantize
tq-mse-3>tq-mse-8TurboQuant 3-bit (mse), rerank top 100 with TurboQuant 8-bit (mse)0.996 ± 0.0001.0005802,12010.6xpg_turbovec
rabitq-1>float32RaBitQ 1-bit, rerank top 100 with Full precision (float32)0.999 ± 0.0000.9992006,34430.7xVectorChord vchordrq
binary>float321 sign bit per dimension, Hamming distance, rerank top 100 with Full precision (float32)0.9920.9921926,33632.0xpgvector binary_quantize

Postgres: real indexes on this server

PostgreSQL 18.3 (Debian 18.3-1.pgdg12+1) with vector 0.8.2, vchord 1.1.1. Each index is built on a copy of the vectors, then every query runs once to warm up and once timed, from one client. Latency includes the round trip.

Recall@10 against p50 latency for each Postgres index, over its search knob0.80.91.012510p50 latency per query, ms (log scale)recall@10pgvector HNSW on vector (float32) hnsw.ef_search=40 recall@10 0.941 p50 1.74 ms, p95 2.23 mspgvector HNSW on vector (float32) hnsw.ef_search=80 recall@10 0.959 p50 2.34 ms, p95 3.07 mspgvector HNSW on vector (float32) hnsw.ef_search=160 recall@10 0.972 p50 3.55 ms, p95 4.51 mspgvector HNSW on vector (float32) hnsw.ef_search=320 recall@10 0.982 p50 5.83 ms, p95 8.60 mshnsw-vectorpgvector HNSW on halfvec (expression index) hnsw.ef_search=40 recall@10 0.940 p50 1.73 ms, p95 2.27 mspgvector HNSW on halfvec (expression index) hnsw.ef_search=80 recall@10 0.957 p50 2.31 ms, p95 3.89 mspgvector HNSW on halfvec (expression index) hnsw.ef_search=160 recall@10 0.969 p50 3.71 ms, p95 5.41 mspgvector HNSW on halfvec (expression index) hnsw.ef_search=320 recall@10 0.978 p50 5.35 ms, p95 7.10 mshnsw-halfvecpgvector HNSW on binary_quantize, rerank with vector candidates=100 recall@10 0.963 p50 2.83 ms, p95 4.97 mspgvector HNSW on binary_quantize, rerank with vector candidates=200 recall@10 0.977 p50 4.98 ms, p95 7.07 mspgvector HNSW on binary_quantize, rerank with vector candidates=400 recall@10 0.986 p50 7.65 ms, p95 9.80 mspgvector HNSW on binary_quantize, rerank with vector candidates=800 recall@10 0.994 p50 13.69 ms, p95 16.52 mshnsw-binary-rerankVectorChord vchordrq (IVF + RaBitQ, vectors kept in the index) vchordrq.probes=4 recall@10 0.854 p50 1.04 ms, p95 1.42 msVectorChord vchordrq (IVF + RaBitQ, vectors kept in the index) vchordrq.probes=10 recall@10 0.916 p50 1.31 ms, p95 1.77 msVectorChord vchordrq (IVF + RaBitQ, vectors kept in the index) vchordrq.probes=20 recall@10 0.949 p50 1.71 ms, p95 2.00 msVectorChord vchordrq (IVF + RaBitQ, vectors kept in the index) vchordrq.probes=40 recall@10 0.981 p50 2.58 ms, p95 3.27 msvchordrqVectorChord vchordrq, codes only (rerank_in_table) vchordrq.probes=4 recall@10 0.853 p50 1.07 ms, p95 1.26 msVectorChord vchordrq, codes only (rerank_in_table) vchordrq.probes=10 recall@10 0.920 p50 1.30 ms, p95 1.51 msVectorChord vchordrq, codes only (rerank_in_table) vchordrq.probes=20 recall@10 0.957 p50 1.69 ms, p95 2.01 msVectorChord vchordrq, codes only (rerank_in_table) vchordrq.probes=40 recall@10 0.985 p50 2.43 ms, p95 2.87 msvchordrq-codes
Size on disk of each index and of the tabletable (the vectors themselves)table (the vectors themselves): 795.1 MiB795.1 MiBhnsw-vectorhnsw-vector: 781.3 MiB781.3 MiBhnsw-halfvechnsw-halfvec: 390.6 MiB390.6 MiBhnsw-binary-rerankhnsw-binary-rerank: 47.9 MiB47.9 MiBvchordrqvchordrq: 812.4 MiB812.4 MiBvchordrq-codesvchordrq-codes: 30.6 MiB30.6 MiB
indexsettingrecall@10p50 msp95 msQPS (1 client)index MiBindex B/vectorbuild s
hnsw-vectorhnsw.ef_search=400.9411.742.23570781.38,19283.1
hnsw-vectorhnsw.ef_search=800.9592.343.07423781.38,19283.1
hnsw-vectorhnsw.ef_search=1600.9723.554.51288781.38,19283.1
hnsw-vectorhnsw.ef_search=3200.9825.838.60169781.38,19283.1
hnsw-halfvechnsw.ef_search=400.9401.732.27569390.64,09670.6
hnsw-halfvechnsw.ef_search=800.9572.313.89398390.64,09670.6
hnsw-halfvechnsw.ef_search=1600.9693.715.41264390.64,09670.6
hnsw-halfvechnsw.ef_search=3200.9785.357.10188390.64,09670.6
hnsw-binary-rerankcandidates=1000.9632.834.9732047.950232.6
hnsw-binary-rerankcandidates=2000.9774.987.0719447.950232.6
hnsw-binary-rerankcandidates=4000.9867.659.8012947.950232.6
hnsw-binary-rerankcandidates=8000.99413.6916.527347.950232.6
vchordrqvchordrq.probes=40.8541.041.42916812.48,51913.7
vchordrqvchordrq.probes=100.9161.311.77735812.48,51913.7
vchordrqvchordrq.probes=200.9491.712.00578812.48,51913.7
vchordrqvchordrq.probes=400.9812.583.27380812.48,51913.7
vchordrq-codesvchordrq.probes=40.8531.071.2691830.632110.6
vchordrq-codesvchordrq.probes=100.9201.301.5175830.632110.6
vchordrq-codesvchordrq.probes=200.9571.692.0158330.632110.6
vchordrq-codesvchordrq.probes=400.9852.432.8740530.632110.6

Reproduce

pip install 'shrinkvec[pg,datasets]'
shrinkvec run --dataset dbpedia-openai-1536 -n 100000 --repeats 3 --out docs/results/dbpedia-openai-1536