Roadmap · stage 10/13

Observability

Stage 10 of 13 · 20 resources · 3 categories

What this stage is

Observability is stage 10 of 13 in the DE//AGENT roadmap. It currently holds 20 resources: 10 tutorials, 5 references, 1 skill across 3 categories.

What it teaches

Freshness, volume, lineage and cataloguing.

What you can do after this stage

Detect drift automatically
Trace a column to its source

Exit gate

Gate: You can tell what a pipeline did without reading its code. The stage deliverable: ref-openlineage-docs.

Gates are self-assessed — the roadmap does not track your progress; it tells you what to be able to do before moving on.

Categories in this stage

Data Observability (10), Data Lineage (6), Data Catalog (4)

Registry resources for this stage

Sorted by composite score — provenance, completeness and trust, per the scoring model:

Resource	Type	Difficulty	Verification	Score
DataHub MCP Server	MCP server	intermediate	community	42.2
Lineage Explainer	Skill	intermediate	community	35.5
Express a freshness check as a metric and an alert	Tutorial	beginner	community	34.15
Hold the observability mental model: infrastructure metrics versus data signals	Tutorial	beginner	community	34.15
Capstone: the data-platform observability checklist	Tutorial	intermediate	community	34.15
Column-level lineage: the concepts and the event standard	Tutorial	intermediate	community	34.15
Instrument pipelines with OpenTelemetry: traces for data jobs	Tutorial	intermediate	community	34.15
Walkthrough: the lineage-explainer skill, from question to impact set	Tutorial	intermediate	community	34.15
Catalogs and ownership: who owns this table and is it safe to use?	Tutorial	beginner	community	31.15
Incident drill: a broken upstream caught by freshness and lineage	Tutorial	intermediate	community	31.15
Lineage tooling in practice: DataHub MCP and the lineage-graph plugin	Tutorial	intermediate	community	31.15
Production gotchas: alert fatigue, missing contracts, no-data silence	Tutorial	intermediate	community	31.15
Freshness Pre-Merge Check	Hook	intermediate	community	31.1
Data Catalog Assistant	Agent	beginner	community	30.5
Lineage Graph	Plugin	intermediate	community	28.6
Grafana Documentation	Reference	—	community	27.7
OpenTelemetry Documentation	Reference	—	community	27.7
Prometheus Documentation	Reference	—	community	27.7
DataHub Documentation	Reference	—	community	26.7
OpenLineage Documentation	Reference	—	community	26.7

Documented learning objectives

Read the lineages of an asset from a lineage store
Compute a downstream impact set
Distinguish technical from business lineage
Declare a freshness SLA and compute worst-case lag from a schedule
Express staleness as a metric and write an alerting rule
Configure no-data handling so silence is an alert, not a pass
Separate infrastructure metrics from data signals — freshness, volume, schema, distribution
Place lineage and the catalog on the observability map, next to metrics and traces

Continue the progression

← Data Quality · Reliability →

← Full roadmap · All learning topics