RDS read replica lagging or replication stopped: find the cause and fix it

#Symptom

Reports or read-only API calls served from an RDS read replica return stale data. The ReplicaLag CloudWatch metric on the replica is climbing steadily, flat at a high value, or reporting -1. Sometimes the console shows the replication state as Error or Stopped.

There are two different problems behind this, and the first check tells you which one you have:

• Replication is running but the replica can't keep up. Lag grows during write bursts and may or may not recover.
• Replication has stopped. Lag never recovers, and on MySQL and MariaDB ReplicaLag reports -1.

This post covers RDS read replicas. Aurora replicas share storage with the writer and use AuroraReplicaLag instead. See the Aurora high availability post for that case.

#Check 1: is replication running?

Start with the replica's status info and recent events:

aws rds describe-db-instances \
  --db-instance-identifier app-db-replica1 \
  --query 'DBInstances[0].{src:ReadReplicaSourceDBInstanceIdentifier,class:DBInstanceClass,status:DBInstanceStatus,repl:StatusInfos}'

aws rds describe-events \
  --source-type db-instance \
  --source-identifier app-db-replica1 \
  --duration 1440 \
  --query 'Events[].[Date,Message]' --output table

StatusInfos with StatusType: read replication shows replicating, error, stopped or terminated. The events usually name the reason, for example a replication error, the replica running out of storage, or replication stopped for a long time.

Then pull the lag over the incident window:

aws cloudwatch get-metric-statistics \
  --namespace AWS/RDS --metric-name ReplicaLag \
  --dimensions Name=DBInstanceIdentifier,Value=app-db-replica1 \
  --start-time 2026-11-10T06:00:00Z --end-time 2026-11-10T12:00:00Z \
  --period 300 --statistics Maximum --output table

#Check 2: replication stopped (MySQL and MariaDB)

Connect to the replica and read the replication status. On MySQL 8.0.22 and later use the REPLICA syntax; older versions and MariaDB use SHOW SLAVE STATUS.

SHOW REPLICA STATUS\G
-- Look at:
--   Replica_IO_Running / Replica_SQL_Running
--   Last_IO_Error, Last_SQL_Error, Last_SQL_Errno
--   Seconds_Behind_Source

Common errors:

RDS doesn't give you SUPER, so use the stored procedures it provides:

CALL mysql.rds_start_replication;
CALL mysql.rds_stop_replication;
CALL mysql.rds_skip_repl_error;   -- skips ONE failing event

Caution

mysql.rds_skip_repl_error throws away the event that failed. If the replica is already out of sync, skipping hides the drift and reports will quietly return wrong data. Only skip when you know exactly which row is affected. Otherwise rebuild the replica.

Set binlog retention on the source so a replica can catch up after a short outage:

-- on the source
CALL mysql.rds_set_configuration('binlog retention hours', 24);
CALL mysql.rds_show_configuration;

Longer retention uses more storage on the source. Watch FreeStorageSpace after you change it.

To rebuild a broken replica, create a new one and switch readers to it before you delete the old one:

aws rds create-db-instance-read-replica \
  --db-instance-identifier app-db-replica2 \
  --source-db-instance-identifier app-db \
  --db-instance-class db.r6g.xlarge

#Check 3: the replica can't keep up

When replication is running but lag keeps growing, the replica is applying changes slower than the source produces them.

Compare the replica's resources with the source's. A replica on a smaller instance class or with lower storage IOPS than the source will fall behind during write peaks. Compare CPUUtilization, WriteIOPS, WriteLatency and DiskQueueDepth on both instances. If the replica is at its IOPS or throughput limit, increase its gp3 IOPS/throughput or move it to a bigger class:

aws rds modify-db-instance \
  --db-instance-identifier app-db-replica1 \
  --db-instance-class db.r6g.2xlarge \
  --apply-immediately

MySQL: check parallel apply. By default the replica applies changes with a small number of worker threads. On a write-heavy source, increase the parallel workers in the replica's parameter group. On MySQL 8.0.26 and later the parameter is replica_parallel_workers; older versions use slave_parallel_workers. Names vary by version, so check your parameter group. Keep replica_preserve_commit_order on if your readers depend on commit order.

MySQL: tables without primary keys. With row-based replication, every update or delete on a table without a primary key can force a full table scan on the replica. This is one of the most common causes of lag that only appears on the replica. Find them:

SELECT t.table_schema, t.table_name, t.table_rows
FROM information_schema.tables t
LEFT JOIN information_schema.table_constraints c
  ON c.table_schema = t.table_schema
 AND c.table_name = t.table_name
 AND c.constraint_type = 'PRIMARY KEY'
WHERE t.table_type = 'BASE TABLE'
  AND c.constraint_name IS NULL
  AND t.table_schema NOT IN ('mysql','sys','information_schema','performance_schema')
ORDER BY t.table_rows DESC;

Large transactions and DDL. A single DELETE of 50 million rows or a long ALTER TABLE is applied on the replica as one unit, and lag rises for as long as it takes. Split bulk deletes and updates into batches on the source. Run big schema changes outside business hours, or use an online schema change tool.

PostgreSQL: check WAL volume. On the source, TransactionLogsGeneration shows how fast WAL is produced. On the replica, measure apply delay directly:

-- on the replica
SELECT now() - pg_last_xact_replay_timestamp() AS replay_delay;

-- on the source
SELECT client_addr, state, write_lag, flush_lag, replay_lag
FROM pg_stat_replication;

replay_delay also grows when the source is idle, so read it together with replay_lag on the source.

#Check 4: PostgreSQL query conflicts

Long queries on a PostgreSQL replica can conflict with WAL replay. max_standby_streaming_delay sets how long replay waits before cancelling the query. While it waits, lag grows. When it cancels, users see errors such as canceling statement due to conflict with recovery.

-- on the replica
SELECT datname, confl_snapshot, confl_lock, confl_bufferpin, confl_tablespace
FROM pg_stat_database_conflicts;

Choose what matters more for this replica:

• Fresh data matters more: keep max_standby_streaming_delay low and move long reports to a separate replica.
• Long reports must finish: raise max_standby_streaming_delay and accept the lag, or set hot_standby_feedback = 1.

Warning

hot_standby_feedback = 1 stops vacuum on the source from removing rows that replica queries still need. Long replica queries then cause table bloat on the primary. Watch dead tuples and the source's storage after you turn it on.

#Prevent it next time

• Alarm on ReplicaLag per replica, with a threshold your readers can live with, and a second alarm for the -1 (stopped) case:

aws cloudwatch put-metric-alarm \
  --alarm-name rds-app-db-replica1-lag \
  --namespace AWS/RDS --metric-name ReplicaLag \
  --dimensions Name=DBInstanceIdentifier,Value=app-db-replica1 \
  --statistic Maximum --period 60 --evaluation-periods 5 \
  --threshold 300 --comparison-operator GreaterThanThreshold \
  --alarm-actions arn:aws:sns:eu-west-1:111122223333:dba-oncall

• Subscribe to RDS events for the read replica category so a stopped replica pages someone instead of failing quietly.
• Size replicas like the source, including storage IOPS, if they serve production reads or are your failover target.
• Keep read_only on for MySQL replicas and don't give application users write privileges there.
• Add primary keys to every replicated MySQL table and batch large data changes.
• Make the application check lag, or fall back to the primary, for reads that must be current.