Redis Cluster operations: hash slots, resharding, rebalancing and finding hot or big keys

Redis Cluster splits the keyspace into 16,384 hash slots spread over primary shards, each with replicas. Scaling is moving slots. Performance problems are usually one shard doing far more than its share.

#Keys, slots and hash tags

redis-cli -c -h node1 cluster keyslot order:1042
redis-cli -c -h node1 cluster keyslot "{customer:7}:orders"
redis-cli -c -h node1 cluster keyslot "{customer:7}:profile"

Slot = CRC16(key) mod 16384. If the key contains {...}, only the part in braces is hashed. That is a hash tag: keys sharing it land in the same slot.

Multi-key commands (MGET, MSET, transactions, Lua scripts touching several keys) only work when all keys are in the same slot:

(error) CROSSSLOT Keys in request don't hash to the same slot

Use hash tags for keys that must be used together, but sparingly: a tag shared by millions of keys puts them all on one shard.

#See the cluster

redis-cli -h node1 cluster info | grep -E "cluster_state|cluster_slots_ok|cluster_known_nodes|cluster_size"
redis-cli -h node1 cluster nodes
redis-cli --cluster check node1:6379
redis-cli --cluster info node1:6379          # keys and slots per primary

cluster_state:ok and all 16,384 slots assigned is the baseline.

#Add a shard and move slots to it

# 1. add a new empty primary
redis-cli --cluster add-node new1:6379 node1:6379

# 2. add a replica for it
redis-cli --cluster add-node new1r:6379 node1:6379 --cluster-slave --cluster-master-id <new1-node-id>

# 3. rebalance slots, including empty masters
redis-cli --cluster rebalance node1:6379 --cluster-use-empty-masters --cluster-pipeline 100

Or move a fixed number of slots:

redis-cli --cluster reshard node1:6379 \
  --cluster-from <source-node-id> --cluster-to <new1-node-id> \
  --cluster-slots 2000 --cluster-yes --cluster-pipeline 100

Moving a slot migrates its keys one batch at a time with MIGRATE. Clients that support cluster mode follow ASK and MOVED redirects during and after the move.

Warning

Very large keys (a hash or sorted set of hundreds of MB) block the source and target while they migrate, causing latency spikes and possible timeouts. Find big keys first (below) and move their slots at quiet times, or split the keys.

#Remove a shard

redis-cli --cluster reshard node1:6379 --cluster-from <old-node-id> --cluster-to <other-node-id> \
  --cluster-slots <all-its-slots> --cluster-yes
redis-cli --cluster del-node node1:6379 <old-node-id>

Remove its replicas first or reassign them.

#Find big keys

redis-cli -h node3 --bigkeys -i 0.01        # samples with a pause between SCAN batches
redis-cli -h node3 --memkeys -i 0.01        # by memory usage (Redis 7+)
redis-cli -h node3 MEMORY USAGE "session:index" SAMPLES 0

Run against each primary (or its replica, to keep load off the primary). -i throttles the scan.

Fixes for big keys: split into buckets (session:index:{0..63}), set TTLs, or move the data to a store better suited (a database or object storage) if it is not cache-shaped.

#Find hot keys

redis-cli -h node3 --hotkeys              # requires maxmemory-policy to be an LFU policy

Without LFU, sample traffic briefly:

timeout 10 redis-cli -h node3 MONITOR | awk '{print $5}' | sort | uniq -c | sort -rn | head

Caution

MONITOR streams every command and can roughly halve throughput on a busy node. Run it for seconds, not minutes, and preferably not on the hottest primary during peak.

Per-node CPU and ops tell you which shard is hot:

for n in node1 node2 node3 node4; do echo -n "$n "; redis-cli -h $n info stats | grep instantaneous_ops_per_sec; done
redis-cli -h node3 info commandstats | sort -t= -k3 -rn | head

Fixes for hot keys:

#Slot coverage and failures

If a primary and all its replicas fail, its slots are unserved and by default the whole cluster reports cluster_state:fail (cluster-require-full-coverage yes). For caches where partial availability is better, consider:

cluster-require-full-coverage no
cluster-allow-reads-when-down yes

Fix slots left in migrating/importing state after an interrupted reshard:

redis-cli --cluster fix node1:6379

#Prevent it next time

• Key naming standard with deliberate hash tags; review new key patterns.
• Weekly --bigkeys/--memkeys scan per shard, alert on keys above a size threshold.
• Monitor ops/sec and memory per shard; imbalance above ~30% triggers investigation.
• Use LFU eviction if you want --hotkeys available.