Skill community advanced

CDC Merge Designer

Design a correct upsert/delete merge from a change stream.

Produces a MERGE (or Delta/Iceberg merge) that applies inserts, updates and deletes from a CDC stream correctly, including out-of-order events and tombstones.

npx de-agent-platform add cdc-merge-designer

At a glance

What	Design a correct upsert/delete merge from a change stream.
Where	CDC · tech-debezium · tech-delta · tech-iceberg

Learning objectives

Order change events deterministically
Handle deletes (tombstones) without losing history
Make the merge idempotent under replay

Security scan

Last sweep 2026-10-24: 0 finding(s), 0 marked as examples.

Details

CDC Merge Designer

Ordering

Events arrive out of order. Rank by (lsn, ts_ms) — never by ingestion time.

Deduplicate before merging

Keep the latest event per primary key with a window function, then merge once.

Deletes

A delete is an event with op = 'd'. Two valid choices:

hard delete on merge, or
soft delete + _deleted_at (required if you need history or audit).

Idempotency

Re-running the same micro-batch must produce the same table. Ranking by LSN guarantees this; ranking by arrival time does not.

Registry stats

Verification	community · Sep 15, 2026
Completeness	100
Trust multiplier	×1
Composite	39.5 · adequate
Freshness	39 days since last source check
Difficulty	advanced
Version	0.2.0
License	Apache-2.0
Security flags	1

Marketplace Quality Score

adequate (39.5/100)

Rule-based scoring across 7 dimensions. Higher is better. How it works

performance /100

completeness 100/100

test coverage 5/100

security 100/100

documentation 55/100

maintainability 30/100

community /100

Metadata

Category: CDC
Stage: Streaming
Version: 0.2.0
License: Apache-2.0
Author: de-agent
Publisher: DE//AGENT Registry
Last Verified: 2026-09-15
Source: https://github.com/XMIRTUNJAY/de-agent-platform/tree/main/data/curated/skills/cdc-merge-designer
Files: SKILL.md, examples/merge.sql, references/tombstone-handling.md

Security Posture (1)

A listing is not an endorsement. Review the flags before connecting.

runs code: No
network access: No
credentials: No
filesystem access: Yes
external apis: No

Provenance

Verification: community
Evidence: 1/10 checks · earned community
Source File: data/curated/skills/06-streaming.yaml

Before you trust this

Limitations: No documented limitations — treat the unknown as unknown, not as safe.
Prerequisites: SQL Review
Evidence checklist: source reachable: unclaimedmetadata validated: passedlicense reviewed: unclaimedtutorial generated: unclaimedstatic checks passed: unclaimedbuild passed: unclaimedtests passed: unclaimedsmoke passed: unclaimedoutput validated: unclaimedsecurity reviewed: unclaimed

Prerequisites (1)

SQL Review (Skill)

Relationships

related to: cat-cdcPostgres CDC to Lakehouse CDC stopped

Related in the graph (5)

Postgres → CDC → Kafka → Lakehouse (Project)
Postgres CDC to Lakehouse (Pattern)
Debezium Documentation (Reference)
CDC stopped (Runbook)
Stream changes, not tables — CDC from Postgres into the lakehouse (Tutorial)

Computed from shared taxonomy — not curator endorsements.

Technologies (3)

Debezium (cat-cdc)
Delta Lake (cat-lakehouse)
Apache Iceberg (cat-lakehouse)

Roadmap Alignment

Streaming (platforms)

This artifact teaches skills for the stages above in the 13-stage Data Engineering roadmap.

Did this solve your problem?

CDC Merge Designer