MongoDB replica set high availability: members, elections, write concern and maintenance

#1. Layout

Spread members across AZs or racks. Keep an odd number of voting members.

#2. Initiate

rs.initiate({
  _id: "rs-sales",
  members: [
    { _id: 0, host: "mongo1.corp.local:27017", priority: 2 },
    { _id: 1, host: "mongo2.corp.local:27017", priority: 1 },
    { _id: 2, host: "mongo3.corp.local:27017", priority: 1 }
  ]
})

# mongod.conf (each member)
replication:
  replSetName: rs-sales
security:
  authorization: enabled
  keyFile: /etc/mongo/keyfile      # or x.509 member auth
net:
  bindIp: 0.0.0.0
  tls: { mode: requireTLS, certificateKeyFile: /etc/mongo/tls/server.pem, CAFile: /etc/mongo/tls/ca.pem }

#3. Write concern and durability

db.adminCommand({ setDefaultRWConcern: 1, defaultWriteConcern: { w: "majority" } })

• w: "majority" (the default since 5.0 for most setups): an acknowledged write survives failover.
• w: 1 is faster, but writes not yet replicated are rolled back when a new primary takes over.
• Journaling is always on in modern versions.

#4. Read preference

Connection string:

mongodb://app:...@mongo1:27017,mongo2:27017,mongo3:27017/sales?replicaSet=rs-sales&tls=true&retryWrites=true&w=majority

retryWrites=true (the default in modern drivers) retries a write once after a failover.

#5. Elections

• When the primary is unreachable for electionTimeoutMillis (10 s by default), the secondaries hold an election. A new primary usually takes over within about 12 seconds.
• Priorities decide which member is preferred, not whether it's allowed. A member with priority: 0 never becomes primary (use this for DR or analytics members).

#6. Rolling maintenance

1. Restart or patch each secondary, and wait until it's in SECONDARY state and caught up.
2. rs.stepDown(120) on the primary.
3. Patch the old primary.

rs.status().members.map(m => ({ name: m.name, state: m.stateStr, lag: m.optimeDate }))
rs.printSecondaryReplicationInfo()

#7. Monitoring

• Replication lag (alert above 30–60 s), state changes, elections count.
• Oplog window (rs.printReplicationInfo()): keep it above 24–72 h, so a member that falls behind can catch up without a full resync.
• Connections, cache usage (wiredTiger.cache), and disk.

#8. Backups

Replica sets aren't backups. Use mongodump (small), filesystem snapshots with journaling on the same volume, Percona Backup for MongoDB, or Ops Manager / Atlas backups for point-in-time recovery.

Tip

Replacing an arbiter with a third data-bearing member is the most valuable change you can make in a P-S-A replica set. With one data node down, majority writes keep working.