DE//AGENT
Explore
Pattern community beginner

Medallion Zones (raw / curated / serving)

Object storage gives you a place to put files but no structure for trust or ownership.

Reference architecture canvas Sources feed processing stages; a data contract sits between processing and destinations; a failure path routes invalid output to a dead-letter zone. OLTP DBs cdc Event log stream Processing transform · model idempotent writes versioned code Data contract schema · grain · SLA validated at boundary Warehouse analytics Feature/API agents · apps Dead letter quarantined output

At a glance

What Object storage gives you a place to put files but no structure for trust or ownership.
When Multiple consumers with different quality needs; You need to reprocess history; You need an audit trail to the source
Where Data Lakes

Problem

Object storage gives you a place to put files but no structure for trust or ownership.

Solution

Three zones with explicit contracts between them. raw/ — immutable, exactly as received; never edited; schema-on-read. curated/ — cleaned, typed, deduplicated; partitioned; quality-gated. serving/ — modelled marts optimised for consumption; SLA-bearing. Each zone has a different retention, quality bar and owner. Promotion between zones is a pipeline step with a gate, not a copy.

When not to use

  • A single small pipeline with one consumer
  • When storage cost dominates and copies are wasteful

Security scan

Last sweep 2026-10-24: 0 finding(s), 0 marked as examples.

Context

A lake or lakehouse where many teams write and read, and consumers need to know how far to trust a table.

Forces

  • Reprocessing needs the original bytes
  • Consumers want clean, typed, tested tables
  • Cost grows with redundant copies
  • Ownership must be legible

Tradeoffs

Choose this when Costs & caveats
clear trust boundary, cheap reprocessing from raw multiple copies, more storage and more pipelines to run
  • watch: don't let curated become a dumping ground; enforce the gate

Diagram

source ─▶ raw (immutable) ─▶ [validate] ─▶ curated ─▶ [model+test] ─▶ serving
              ▲                                              │
              └──────────────── reprocess ◀──────────────────┘

Registry stats

Verification community · Sep 28, 2026
Completeness 75
Trust multiplier ×1.25
Composite 37.25 · adequate
Freshness 26 days since last source check
Difficulty beginner
Version 1.1.0
License CC-BY-4.0
Security flags none