Capstone — wire a dataset's quality gates end to end

At a glance

Learning objectives

• Run the full checklist — contract, CI gate, promotion gate, quarantine, alerts — against one dataset
• Point at the registry artifact that enforces each gate
• Name the monitoring signals and prevention paths that keep the gates honest

What you need

• The eight earlier lessons of this series, or their registry artifacts
• [object Object]
• A dataset with a consumer and an SLA worth defending

Steps

1. Write the six-part contract — owner, schema, semantics, SLA, breaking-change policy, enforcement. (from skill-data-contract-author )
2. Run the contract as code in the producer's CI — compatibility check, then build, then publish. (from pattern-contracts-quality-gate )
3. After a pipeline writes to the staging namespace, run the model's dbt/GE tests against the staged output; promote only if all critical tests pass. (from hook-pre-deployment-dq-check )
4. Split every batch into publish and quarantine on a quality contract — a row never disappears. (from pattern-quarantine-publish )
5. Alert when quarantine rate > 1% for 2 consecutive runs, and ship a reprocess command that re-runs the quarantined rows after a fix. (from project-pipeline-dq-quarantine )
6. Watch the signals — expectation pass rate, quarantine row counts, freshness. (from runbook-data-quality-failure )
7. Close the loop with consumer-driven contract tests in the producer's pipeline. (from runbook-data-contract-broken )

Check yourself

• The checklist names the CI test that fails on violation — enforcement, part six of the contract.
• A not-null test failing on 412 rows in staging blocks promotion and leaves the previous production data in place.
• The quarantine alert fires on rate — 1% for 2 consecutive runs — not only on a hard failure.

Worked example

1. Owner (a team, with an on-call).
2. Schema (typed, versioned).
3. Semantics (what each field *means*, units, nullability).
4. SLA (freshness, completeness).
5. Breaking-change policy (notice period, deprecation path).
6. Enforcement (which CI test fails on violation).

trigger: pre-deploy
purpose: Run the quality suite against a staged run before promoting to production.
failure_behavior: block

expectation pass rate, quarantine row counts, freshness

Sourced from

• Data Contract Author
• Contracts as a Quality Gate
• Pre-deployment DQ Check
• Quarantine then Publish
• Pipeline → DQ → quarantine → publish
• Data quality failure
• Data contract broken

Security scan

Last sweep 2026-10-24: 0 finding(s), 0 marked as examples.

Common gotchas

• Tier the expectations: critical (block) vs warning (alert only) — blocking on a warning-tier expectation turns small blips into outages.
• Quarantine never reviewed → add a scheduled review and a reprocess path.
• The gate lives where the change is made (producer CI), not in the consumer's runtime — the cheapest place to catch it is before merge.

Production considerations

• Alert when quarantine rate > 1% for 2 consecutive runs.
• Contracts get CI enforcement and a dated deprecation path.

Diagram

capstone checklist
   |
   |- contract (six parts) --> CI compatibility check --> build --> publish
   |
   |- staged run --> dbt/GE tests --> all critical pass --> promote
   |                                  any critical fail --> block --> production untouched
   |
   |- batch --> pass --> published
   |             fail --> quarantine (+reason) --> rate > 1% x2 runs --> alert --> review --> reprocess
   |
   |- signals: expectation pass rate, quarantine row counts, freshness
   |
   |- consumer-driven contract tests in the producer's pipeline

Registry stats