idempotency

DE//AGENT currently lists 8 resources under idempotency: 5 tutorials, 1 skill, 1 runbook. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Also present: 1 pattern.

Documented learning objectives across these resources:

• Make every task safe to re-run — via Airflow DAG Reviewer
• Choose retries and timeouts deliberately — via Airflow DAG Reviewer
• Predict what a backfill will do before running it — via Airflow DAG Reviewer
• Count duplicates per key with one GROUP BY query — via Deduplicate records without rewriting the table
• Correlate the first duplicate date with a deploy — via Deduplicate records without rewriting the table
• Rewrite affected partitions instead of the whole table — via Deduplicate records without rewriting the table
• Apply the idempotency test to every task in a DAG — via Review a DAG for idempotency and retry safety
• Justify retry, timeout and concurrency settings per task — via Review a DAG for idempotency and retry safety

9 more objectives live on the individual resource pages.

These resources map to stages: Data Quality (2), Orchestration (2), Data Processing (2), Reliability (1), Streaming (1). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

How scores are computed: source quality score.

Highest composite score first:

1. Airflow DAG Reviewer score 39.5 — intermediate
2. Duplicate records score 35.25 — intermediate
3. Idempotent Partition Overwrite score 34.75 — beginner

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Start here if you're new: Idempotent Partition Overwrite, Deduplicate records without rewriting the table, Review a DAG for idempotency and retry safety.

Security-flagged entries: Airflow DAG Reviewer (1 flag), Duplicate records (1 flag). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

Known limits: Streaming writes with no partition boundary (use MERGE/upsert instead) (Idempotent Partition Overwrite)

• Airflow DAG Reviewer — Checks a DAG for non-idempotent tasks, unsafe retries, missing catchup policy, ambiguous catchup/backfill semantics and unbounded concurrency. details
• Duplicate records — Row counts are inflated and metrics double-counted after a retry, a fan-out join or a backfill. details
• Deduplicate records without rewriting the table — Row counts are inflated and revenue double-counted after a retry or a fan-out join. Work the duplicate-records runbook: find the first duplicate date, correlate it with a deploy, d… details
• Review a DAG for idempotency and retry safety — Walk the Pipeline Reviewer's four-step pass over a DAG: apply the idempotency test to every task, check retry, timeout and concurrency settings, predict backfill behaviour, and pro… details

Real install commands from the registry — read the source before running any of them:

npx de-agent-platform add airflow-dag-reviewer

Airflow DAG Reviewer — review first

Adjacent topics: tutorial (5 shared), data-quality (2 shared), batch (2 shared), partitions (2 shared), dag (2 shared), duplicates (2 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.