CDC Bootstrapping & Schema Evolution

CDC Bootstrapping and Schema Evolution

Change data capture streams every insert, update, and delete from a database's transaction log to other systems — search indices, caches, warehouses, other services. Two moments decide whether a CDC pipeline is pleasant or painful. The first is bootstrapping: the log only contains recent changes, so consumers first need a consistent copy of the existing data, and the copy must line up exactly with the point where streaming begins. The second is schema evolution: the source tables will change — new columns, renamed fields, new types — while the pipeline keeps running and consumers keep reading.

This page covers both, using Debezium and Kafka as the running example; the principles apply to managed CDC services and other connectors as well.

TL;DR

A CDC pipeline starts with a snapshot of existing rows, then streams changes from the log position captured at snapshot time.
Incremental snapshots read tables in chunks while streaming continues, and can re-snapshot single tables on demand.
Replication slots (PostgreSQL) and binlog retention (MySQL) must outlast any connector downtime — and stuck slots can fill the disk.
Additive changes (new nullable columns) are safe; renames, drops, and type changes need expand-and-contract migrations.
Use a schema registry with a compatibility mode so incompatible changes fail fast at the producer.
Downstream consumers apply changes as idempotent upserts by primary key, ordered by source log position.

Quick Example

A Debezium PostgreSQL connector that snapshots two tables, then streams changes, with a signal table for on-demand incremental snapshots:

Later, after adding a table or fixing a downstream bug, re-snapshot one table without stopping the stream:

Core Concepts

Snapshot Plus Stream

A CDC consumer's view of a table is built from two sources: a snapshot of rows that already existed, and the stream of changes from the transaction log. The pipeline is correct only if the stream starts at exactly the log position the snapshot represents — earlier, and changes are applied twice; later, and changes are lost.

Log Positions and Offsets

Every change carries a source position: a PostgreSQL LSN, a MySQL binlog file and position or GTID, or an equivalent. The connector stores the last position it delivered as its offset, and resumes from it after restarts. Consumers can use the same positions to order events and ignore stale ones.

Schemas Travel With Events

Each change event is serialized with the table's schema at the time of the change. Schema changes in the source therefore become schema changes in every topic and consumer — which is why DDL needs the same care as an API change.

Bootstrapping

Initial Snapshots

A traditional snapshot records the current log position, reads every row of the captured tables inside a consistent read, emits each row as a "read" event, and then starts streaming from the recorded position. Changes made during the snapshot are not lost, because streaming begins from before the snapshot finished. Large tables make this slow, and a failure midway may force the snapshot to start over.

Incremental Snapshots

Incremental snapshots read tables in primary-key chunks while streaming continues. Watermarks written to the signal table mark the start and end of each chunk, and rows changed by streaming events during a chunk are deduplicated so the newest value wins. This makes snapshots resumable, avoids long-running transactions on the source, and allows re-snapshotting individual tables after adding them to the capture list or repairing a consumer.

Log Retention and Replication Slots

Monitor slot lag and log retention, alert long before limits, set max_slot_wal_keep_size in PostgreSQL to cap risk, and drop slots of decommissioned connectors. See PostgreSQL Logical Replication.

Schema Evolution

What Happens When the Source Changes

CDC connectors observe DDL: PostgreSQL's logical decoding includes relation metadata, and MySQL connectors keep a schema history topic of DDL statements. Each change event is serialized with the table's current schema, so a column change produces a new schema version for the topic.

Schema Registry Compatibility

A schema registry stores every schema version per topic and rejects new versions that violate the configured compatibility mode:

With registry checks in place, an incompatible DDL change makes the connector fail loudly instead of silently breaking consumers. See Event Schema Evolution.

Expand and Contract

Make breaking source changes in compatible steps:

Expand: add the new column, nullable or with a default; write to both old and new.
Migrate: backfill the new column; update consumers to read the new field.
Contract: once every consumer has moved, stop writing and drop the old column.

Raw Tables Are Not Public Contracts

Streaming raw tables couples every consumer to the source's internal schema. For data shared across teams, publish curated events — through the outbox pattern or a transformation layer — and keep raw CDC topics for internal replication.

Applying Changes Downstream

Upsert by primary key and delete on delete events; snapshot "read" events and stream "create" events are handled the same way.
Order by source position (LSN, binlog position, or the connector's source timestamp and sequence) so late-arriving snapshot rows never overwrite newer changes.
Tombstones (null-value messages after deletes) let compacted Kafka topics drop deleted keys — keep them if downstream uses compaction.
Idempotency makes restarts, rebalances, and re-snapshots safe. See Delivery Guarantees and Consumer Idempotency.

Best Practices

Plan the First Snapshot Like a Migration

Estimate table sizes and snapshot duration, run it off-peak or from a replica where supported, and prefer incremental snapshots for large tables.

Alert on Slot Lag and Connector Health

Connector failures are quiet until disks fill or logs are purged. Alert on replication lag, connector task state, and slot size.

Put DDL Through Review With CDC in Mind

Add a check to migration review: "Does this change break CDC consumers?" Renames and drops need an expand-and-contract plan.

Document Re-Snapshot Procedures

Keep a runbook for re-snapshotting a table, resetting a connector, and recovering from purged logs, and practice it before you need it.

Common Mistakes

Abandoned Replication Slots

Renaming a Column in Place

Snapshot Rows Overwriting Newer Changes

FAQ

What is a CDC snapshot?

A consistent copy of existing table data taken when a CDC pipeline starts, emitted as events so consumers have the full current state before streaming changes from the transaction log.

What is an incremental snapshot?

A snapshot that reads a table in chunks while change streaming continues, deduplicating rows that change during each chunk. It is resumable and can be triggered for specific tables on demand.

Why is my PostgreSQL disk filling up with WAL?

Usually because a replication slot is inactive or lagging, so PostgreSQL keeps all WAL since the slot's confirmed position. Restart or drop the consumer and cap retention with max_slot_wal_keep_size.

How do I rename a column without breaking CDC consumers?

Use expand and contract: add the new column, backfill and write to both, move consumers to the new field, then remove the old column in a later release.

Should consumers read raw CDC topics directly?

For internal replication, yes. For data shared across teams, publish curated events or apply a transformation layer so the source table's internals are not a public contract.