Home / Data Platform / Streaming & CDC

Data PlatformTroubleshootingAdvanced

Debezium CDC lag and stuck connectors: a troubleshooting guide

Check connector status, source-side replication slots and binlogs, and the sink, to find where change-data-capture lag builds up and recover without losing changes.

Deepak Bhardwaj2 min read

Applies toDebezium 2.x on Kafka Connect (PostgreSQL, MySQL, SQL Server connectors)

1. Is the connector running?

curl -s http://connect:8083/connectors/orders-cdc/status | jq
  • FAILED task → the trace field holds the exception. Fix the cause, then restart:
curl -X POST "http://connect:8083/connectors/orders-cdc/restart?includeTasks=true&onlyFailed=true"

2. Measure lag

JMX metric MilliSecondsBehindSource (streaming metrics MBean) is the delay between the change in the database and Debezium processing it. Also watch the sink consumer group lag:

kafka-consumer-groups.sh --bootstrap-server broker1:9092 --describe --group connect-orders-sink

Lag high at Debezium → source or connector problem. Lag low at Debezium but high at the sink → the sink is slow.

3. PostgreSQL: replication slot

SELECT slot_name, active, wal_status,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)) AS lag
FROM pg_replication_slots;

Warning

An inactive slot keeps WAL on the primary forever, and can fill its disk and take the database down. If the connector will be down for long, set max_slot_wal_keep_size (PG 13+) as a safety net, or drop the slot and plan a re-snapshot.

Also: a low-traffic database with a busy other database on the same server can make the slot lag. Enable heartbeat.interval.ms and heartbeat.action.query.

4. MySQL: binlog retention

If the connector was stopped longer than binlog retention, it fails with The connector is trying to read binlog starting at ... but this is no longer available on the server.

SHOW BINARY LOGS;
SHOW VARIABLES LIKE 'binlog_expire_logs_seconds';

Recovery: a new snapshot (snapshot.mode=when_needed, or an incremental snapshot via a signal table). Prevent it by keeping binlogs longer than your longest expected outage.

5. Large transactions and snapshots

  • A single huge transaction (a bulk UPDATE of millions of rows) produces a burst that takes time to drain. It's expected; check it's draining.
  • Initial snapshots of big tables can take hours. Use incremental snapshots so streaming continues while tables are snapshotted in chunks.

6. Throughput settings

Setting Effect
max.batch.size, max.queue.size Bigger batches from the source (raise both together)
Producer linger.ms, batch.size, compression.type=lz4 More efficient writes to Kafka
Sink tasks.max Parallel sink writes, up to the number of topic partitions

Tip

Alert on replication slot size (PostgreSQL) or binlog position age (MySQL) as well as connector status. The database disk is what fails first if CDC stops quietly.

Related fixes