Roadmap · stage 5/13

Lakehouse

Stage 5 of 13 · 20 resources · 2 categories

What this stage is

Lakehouse is stage 5 of 13 in the DE//AGENT roadmap. It currently holds 20 resources: 10 tutorials, 4 references, 2 skills across 2 categories.

What it teaches

ACID tables on object storage — Delta, Iceberg, Hudi.

What you can do after this stage

Choose between Delta, Iceberg and Hudi
Plan compaction and snapshot expiry

Exit gate

Gate: You can run and maintain an open table format at scale. The stage deliverable: skill-iceberg-maintenance.

Gates are self-assessed — the roadmap does not track your progress; it tells you what to be able to do before moving on.

Categories in this stage

Data Lakes (2), Lakehouse (18)

Registry resources for this stage

Sorted by composite score — provenance, completeness and trust, per the scoring model:

Resource	Type	Difficulty	Verification	Score
Medallion Zones (raw / curated / serving)	Pattern	beginner	community	37.25
Iceberg Table Maintenance	Skill	advanced	community	36.5
Delta Optimization Advisor	Skill	intermediate	community	34.9
Decide when a Delta table needs OPTIMIZE or Z-ORDER	Tutorial	beginner	community	34.15
The lakehouse mental model — lake vs warehouse vs table formats vs zones	Tutorial	beginner	community	34.15
Your first lakehouse table — create, query, measure, compact	Tutorial	beginner	community	34.15
ACID and concurrent writes — why compaction and ingestion collide	Tutorial	intermediate	community	34.15
Capstone — the standing lakehouse checklist	Tutorial	intermediate	community	34.15
Check lakehouse table health with Lakehouse Guard	Tutorial	intermediate	community	34.15
Diagnose a small-files explosion	Tutorial	intermediate	community	34.15
Run the Iceberg maintenance routine	Tutorial	intermediate	community	34.15
Schema evolution and time travel — change the table, keep the past	Tutorial	intermediate	community	34.15
Table format mechanics — snapshots, manifests, and the metadata tables	Tutorial	intermediate	community	34.15
Small files explosion	Runbook	intermediate	community	29.25
Compaction & File-Size Management	Pattern	intermediate	community	28.75
Lakehouse Guard	Plugin	advanced	community	28.6
Apache Iceberg Documentation	Reference	—	community	27.7
Delta Lake Documentation	Reference	—	community	27.7
A DuckDB Database with No Data in It	Reference	—	community	23.55
Query across accounts and table formats with multi-catalog in Amazon EMR	Reference	—	community	23.55

Documented learning objectives

Read an Iceberg metadata table to size the problem
Pick a compaction strategy and rewrite interval
Expire snapshots without breaking time travel
Decide if a table needs OPTIMIZE from file stats
Choose Z-ORDER columns from query predicates
Know when liquid clustering beats partitioning
Contrast lake, warehouse, and lakehouse by what each does to raw files
Name the table format's jobs — snapshots, manifests, hidden partitioning, schema evolution

Continue the progression

← Warehousing · Orchestration →

← Full roadmap · All learning topics