Pattern community intermediate
Quarantine then Publish
One bad batch of rows can poison a whole table and every downstream metric.
At a glance
| What | One bad batch of rows can poison a whole table and every downstream metric. |
|---|---|
| When | External/partner data; Any table with an SLA; High-value downstream metrics |
| Where | Data Quality |
Problem
One bad batch of rows can poison a whole table and every downstream metric.
Solution
Split every batch into publish and quarantine on a quality contract. pass ─▶ published table (SLA-bearing) fail ─▶ quarantine table (retained, alertable, reprocessable) A row never disappears: it is either published or quarantined with a reason. Alert on quarantine *rate*, not just on a hard failure — a rising rate is the early signal of an upstream problem.
When not to use
- Throwaway exploratory pipelines
Security scan
Last sweep 2026-10-24: 0 finding(s), 0 marked as examples.
Context
A pipeline where a known fraction of incoming rows may be malformed.
Forces
- Consumers need clean data
- Bad rows must not be silently dropped
- Reprocessing bad rows must be possible
Tradeoffs
| Choose this when | Costs & caveats |
|---|---|
| consumers always read clean data; failures are visible and recoverable | extra storage and a reprocessing path to maintain |
- watch: a quarantine nobody reviews is just a slower data loss
Diagram
batch ─▶ [contract check] ─┬─ pass ─▶ published ─▶ consumers
└─ fail ─▶ quarantine (+reason) ─▶ alert/reprocess
Registry stats
| Verification | community · Sep 27, 2026 |
|---|---|
| Completeness | 75 |
| Trust multiplier | ×1.25 |
| Composite | 37.25 · adequate |
| Freshness | 27 days since last source check |
| Difficulty | intermediate |
| Version | 1.1.0 |
| License | CC-BY-4.0 |
| Security flags | none |
Did this solve your problem?
Quarantine then Publish