Home / Data Platform / Streaming & CDC

Data PlatformHow-toAdvanced

Kafka high availability: replication factor, min.insync.replicas, acks and rack awareness

Configure Kafka so that a broker or availability zone can fail without data loss or downtime, with the right replication settings, rack awareness, controller quorum, producer settings and monitoring.

Deepak Bhardwaj2 min read

Applies toApache Kafka 3.x/4.x (KRaft), Confluent Platform, Amazon MSK, Azure HDInsight Kafka, OCI Streaming with Kafka API (concepts)

The core trio

Setting Recommended Why
replication.factor 3 Each partition stored on 3 brokers
min.insync.replicas 2 A write is accepted only when at least 2 replicas have it
Producer acks all Producer waits for all in-sync replicas

With these, one broker (or one AZ) can fail with no data loss and no write outage. If two replicas of a partition are down, writes to it fail (NOT_ENOUGH_REPLICAS) instead of silently losing durability.

# broker defaults
default.replication.factor=3
min.insync.replicas=2
unclean.leader.election.enable=false
offsets.topic.replication.factor=3
transaction.state.log.replication.factor=3
transaction.state.log.min.isr=2

# producer
acks=all
enable.idempotence=true
retries=2147483647
delivery.timeout.ms=120000

Rack (AZ) awareness

Tell brokers which AZ they're in so replicas spread across AZs:

broker.rack=ap-south-1a

Consumers can read from the closest replica (fewer cross-AZ costs) with replica.selector.class=org.apache.kafka.common.replica.RackAwareReplicaSelector on brokers and client.rack on consumers.

Controller quorum (KRaft)

  • 3 controllers (tolerates 1 failure) or 5 (tolerates 2), each in a different AZ.
  • Dedicated controller nodes for medium/large clusters.

Monitoring

Metric Alert when
UnderReplicatedPartitions > 0 for 5 minutes
UnderMinIsrPartitionCount > 0 (writes failing)
OfflinePartitionsCount > 0 (partitions unavailable)
ActiveControllerCount (sum) ≠ 1
Disk usage per broker > 75% 
kafka-topics.sh --bootstrap-server broker1:9092 --describe --under-replicated-partitions
kafka-topics.sh --bootstrap-server broker1:9092 --describe --under-min-isr-partitions

Rolling restarts without downtime

Restart one broker at a time; wait until UnderReplicatedPartitions returns to 0 before the next. Controlled shutdown moves leadership away first (controlled.shutdown.enable=true, default).

Capacity for failure

With 3 AZs, losing one AZ adds the lost brokers' leadership and replication catch-up traffic to the rest. Keep CPU, network and disk headroom (around 30–40%).

Managed services

Amazon MSK with 3 AZs and these settings follows the same rules; Confluent Cloud and MSK Serverless handle broker placement for you, but topic replication and producer acks are still your responsibility.

Warning

replication.factor=3 with min.insync.replicas=1 and acks=1 (common defaults in old configs) can lose acknowledged messages when a leader fails. Check existing topics, not just broker defaults.

Related fixes