Vector index operations: HNSW tuning, memory sizing and measuring recall

#The three HNSW knobs

Reasonable starting points: M = 16, ef_construction = 200, ef_search = 100. Then tune using measured recall.

Important

Not every engine lets you change ef_search per query or after the build. Some fix it in the index metadata at creation time. Measure recall after every configuration change, and don't assume a parameter change took effect.

#Memory sizing

Raw vectors: N × dimensions × 4 bytes (float32).

The HNSW graph adds about N × M × 2 × 4 bytes (10 M vectors at M=16 ≈ 1.3 GB), plus metadata and engine overhead. Plan for 1.5–2× the raw vector size in RAM for the index to be fast.

Ways to shrink it:

• Scalar quantization (int8): about 4× smaller, usually with a small recall loss.
• Binary quantization: about 32× smaller. It needs re-scoring with the full vectors.
• Smaller embedding dimensions: many current models support truncated (Matryoshka) dimensions.

#pgvector example

CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 200);
SET maintenance_work_mem = '8GB';          -- build in memory; much faster
SET max_parallel_maintenance_workers = 7;

-- per session / transaction at query time
SET hnsw.ef_search = 100;
SELECT id FROM docs ORDER BY embedding <=> $1 LIMIT 10;

If you also filter (WHERE tenant_id = …), the index returns ef_search candidates before the filter is applied, so you may get fewer than 10 rows back. Use hnsw.iterative_scan (pgvector 0.8+), partial indexes per large tenant, or raise ef_search.

#Measuring recall

Recall@k is the share of the true top-k neighbours, from exact search, that the index returns.

import numpy as np

def recall_at_k(ann_ids, exact_ids, k=10):
    return np.mean([len(set(a[:k]) & set(e[:k])) / k for a, e in zip(ann_ids, exact_ids)])

# exact_ids: brute-force results for ~500 sample queries (numpy dot product, or the engine's exact/flat mode)
# ann_ids:   the same queries through the HNSW index

Plot recall against p95 latency for several ef_search values, and choose the lowest value that meets your target (often 0.95–0.98).

#Day-2 operations

• Deletes: HNSW usually marks deleted nodes with tombstones. After heavy churn (more than about 20% deleted), recall and latency get worse. Rebuild or compact the index on a schedule.
• Re-embedding: a new embedding model means new vectors that can't be mixed with the old ones. Build a second index, backfill it, run your recall and quality tests, then switch with an alias.
• Warm-up: a cold index after a restart serves its first queries from disk. Preload it (pg_prewarm, engine warm-up APIs) before you send traffic.
• Monitor: query p95/p99, recall on a nightly golden set, index size against RAM, and the number of vectors waiting to be indexed.