Vector database high availability: replicas, shards and zone placement across common engines

#1. Principles

• Replicas give availability and read throughput. Shards give capacity.
• At least 2 replicas for every shard, on different nodes, ideally in different zones.
• Writes should be acknowledged by more than one replica, or be replayable from a queue (see the RAG pipeline HA post).

#2. HA by engine

#3. Qdrant example

curl -X PUT "https://qdrant:6333/collections/kb_docs" -H "api-key: $KEY" -H "Content-Type: application/json" -d '{
  "vectors": { "size": 1536, "distance": "Cosine" },
  "shard_number": 6,
  "replication_factor": 2,
  "write_consistency_factor": 2
}'

• 3–6 nodes across 3 zones. Use shard_number as a multiple of the node count, so shards spread evenly.
• write_consistency_factor: 2 means writes succeed only once 2 replicas have them.

#4. Azure AI Search example

az search service update -n srch-prod -g rg-ai-prod --replica-count 3 --partition-count 1

• Partitions increase storage and indexing capacity. Replicas increase query throughput and availability.
• Indexers run on the service. A failed indexer run doesn't affect queries on the existing index.

#5. Sizing replicas for failure

If peak query load needs N replicas, run N+1, so one node or zone can fail without the remaining replicas saturating. Check p95 latency with one replica removed.

#6. Client behaviour

• Use the engine's SDK with retries and timeouts (for example a 2–5 s search timeout, then retry once).
• Graceful degradation: if vector search fails, fall back to keyword search, or answer "I can't search right now" rather than letting the model answer without context.

#7. Tests

Tip

Many vector stores depend on other components (etcd, object storage, Kafka for Milvus; PostgreSQL for pgvector). The vector layer is only as available as its least available dependency, so list them all and check each one.