Size math
raw size ≈ vectors × dimensions × bytes per value
10M chunks × 1536 dims × 4 bytes (float32) ≈ 61 GB (before index overhead)
HNSW graphs add more on top (see the HNSW tuning post).
Lever 1: fewer dimensions
Some embedding models let you request fewer dimensions with modest quality loss:
emb = client.embeddings.create(model="text-embedding-3-large", input=texts, dimensions=1024)
Re-embedding is required when you change dimensions (see the embedding migration post).
Lever 2: quantization
| Type | Size vs float32 | Typical effect |
|---|---|---|
| float16 / halfvec | 1/2 | Minimal recall loss |
| int8 (scalar) | 1/4 | Small recall loss |
| Binary (1 bit) | 1/32 | Larger loss; needs rescoring |
| Product quantization | Configurable | Depends on settings |
Examples:
-- pgvector: half-precision storage and index
ALTER TABLE chunks ADD COLUMN emb_half halfvec(1536);
UPDATE chunks SET emb_half = embedding::halfvec(1536);
CREATE INDEX ON chunks USING hnsw (emb_half halfvec_cosine_ops);
// Qdrant collection with scalar quantization kept in RAM, originals on disk
{
"vectors": { "size": 1536, "distance": "Cosine", "on_disk": true },
"quantization_config": { "scalar": { "type": "int8", "always_ram": true } }
}
Azure AI Search supports scalar and binary compression on vector fields with options to keep or discard originals; check current settings and tier limits.
Lever 3: rescoring (oversampling)
Search with the compressed index for more candidates than needed (e.g. 4× k), then rescore them with full-precision vectors and keep the top k. This recovers most recall lost to quantization.
Measure recall
- Build a test set of queries with known relevant chunks (or use exact search results as ground truth).
- Compare recall@k and answer quality for: original, reduced dims, quantized, quantized + rescoring.
- Record memory/storage and latency for each.
| Config | Memory | Recall@10 | p95 latency |
|---|---|---|---|
| float32 1536 | baseline | baseline | baseline |
| int8 + rescoring | measure | measure | measure |
| binary + rescoring | measure | measure | measure |
Tip
int8 quantization with rescoring is a safe first step on most stores. Binary quantization gives the biggest savings but needs careful recall testing with your data and embedding model.
Checklist
- Current index size and growth calculated
- Dimension reduction evaluated
- Quantization type chosen per store
- Rescoring/oversampling enabled
- Recall and answer quality measured
- Memory and cost savings recorded