Running Qdrant as a distributed cluster: shards, replication factor, write consistency and moving shards to new nodes

#Concepts in one table

Collection metadata (create, delete, shard placement) goes through Raft consensus. Point writes go directly to the replicas of the relevant shard.

#Step 1: enable cluster mode

In config.yaml (or env QDRANT__CLUSTER__ENABLED=true):

cluster:
  enabled: true
  p2p:
    port: 6335

Start the first peer with its own URI, and the others pointing at it:

# first peer
qdrant --uri http://qdrant-0.qdrant-headless:6335

# every other peer
qdrant --bootstrap http://qdrant-0.qdrant-headless:6335 --uri http://qdrant-1.qdrant-headless:6335

On Kubernetes the Qdrant Helm chart does this for you with a StatefulSet and headless service. Each peer needs its own persistent volume.

Check the cluster:

curl -s -H "api-key: $QDRANT_API_KEY" http://qdrant:6333/cluster | jq

You should see every peer, one Raft leader, and no pending operations.

Note

Raft needs a majority of peers. Run 3 or more peers for a cluster that survives a node failure. With 2 peers, losing one stops collection-level changes.

#Step 2: choose shard and replica counts

Shard count can't be changed later on self-hosted Qdrant (resharding is a Qdrant Cloud feature); changing it means creating a new collection and re-uploading. So plan for growth:

curl -X PUT http://qdrant:6333/collections/kb_chunks \
  -H "api-key: $QDRANT_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "vectors": {"size": 1536, "distance": "Cosine"},
    "shard_number": 6,
    "replication_factor": 2,
    "write_consistency_factor": 1
  }'

#Step 3: check where shards are

curl -s -H "api-key: $QDRANT_API_KEY" http://qdrant:6333/collections/kb_chunks/cluster | jq

The response lists local_shards and remote_shards with peer IDs and state (Active, Partial, Dead…), plus any shard_transfers in progress.

#Step 4: add a node and move data onto it

A new peer joins empty; data isn't rebalanced automatically on self-hosted. Move shards explicitly:

curl -X POST http://qdrant:6333/collections/kb_chunks/cluster \
  -H "api-key: $QDRANT_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "move_shard": {
      "shard_id": 4,
      "from_peer_id": 381894127,
      "to_peer_id": 467122995,
      "method": "stream_records"
    }
  }'

Transfer methods:

Watch progress in the shard_transfers section of the collection cluster info. Move one or two shards at a time; transfers use network, disk and CPU on both peers.

To add a replica instead of moving:

-d '{"replicate_shard": {"shard_id": 4, "from_peer_id": 381894127, "to_peer_id": 467122995}}'

To remove a replica from a peer:

-d '{"drop_replica": {"shard_id": 4, "peer_id": 381894127}}'

Never drop the last active replica of a shard.

#Step 5: reads and consistency

Reads can ask for stronger consistency per request:

curl -X POST "http://qdrant:6333/collections/kb_chunks/points/query?consistency=majority" ...

Values: a number of replicas, majority, quorum or all. Default reads from one replica, which is fastest but can briefly return stale data right after a write.

#Step 6: when a peer fails

Remove a dead peer once its shards are re-replicated:

curl -X DELETE "http://qdrant:6333/cluster/peer/381894127" -H "api-key: $QDRANT_API_KEY"

The call refuses to remove a peer that still holds shards unless forced; don't force it unless you've confirmed the data exists elsewhere.

#Step 7: Kubernetes specifics

• Use a PodDisruptionBudget (maxUnavailable: 1) so node drains don't take down several peers.
• Spread peers across zones with topology spread constraints, and remember a replica on the same zone doesn't protect against a zone outage.
• Don't scale the StatefulSet down without first moving shards off the last peer; its PVC holds data.

#Checklist

#Prevent problems next time

• Decide shard counts with a 2–3 year growth estimate; recreating collections later is expensive.
• Script shard moves and test them in staging.
• Alert on dead or partial replicas from /collections/{name}/cluster.
• Take collection snapshots before cluster changes.