Roadmap · stage 1/13

Foundations

Stage 1 of 13 · 16 resources · 2 categories

What this stage is

Foundations is stage 1 of 13 in the DE//AGENT roadmap. It currently holds 16 resources: 10 tutorials, 6 references across 2 categories.

How storage, compute, encoding and networks actually behave.

Gate: You can describe a data system's lifecycle and where cost and latency come from. The stage deliverable: ref-fundamentals-de.

Gates are self-assessed — the roadmap does not track your progress; it tells you what to be able to do before moving on.

Sorted by composite score — provenance, completeness and trust, per the scoring model:

Resource	Type	Difficulty	Verification	Score
Hold the row-versus-column mental model	Tutorial	beginner	community	34.15
Open the Parquet file — row groups, pages, encodings	Tutorial	beginner	community	34.15
The twelve-factor vocabulary for pipeline jobs	Tutorial	beginner	community	34.15
Arrow — columnar memory and zero-copy interchange	Tutorial	intermediate	community	34.15
Avro and schema evolution for event payloads	Tutorial	intermediate	community	34.15
Capstone — the foundations checklist	Tutorial	intermediate	community	34.15
Choosing file formats — Parquet, Avro, and Arrow's role	Tutorial	intermediate	community	34.15
Compression and encodings — the tradeoffs behind the file	Tutorial	intermediate	community	34.15
Format-driven performance incidents — small scans and wrong layouts	Tutorial	intermediate	community	34.15
Reading data files — from bytes to columnar memory	Tutorial	intermediate	community	34.15
Apache Arrow Documentation	Reference	—	community	27.7
Apache Parquet Format Documentation	Reference	—	community	27.7
The Twelve-Factor App	Reference	—	community	27.7
Apache Avro Documentation	Reference	—	community	27.2
Apache Parquet Documentation & Spec	Reference	—	community	26.7
Fundamentals of Data Engineering (O'Reilly)	Reference	—	community	26.7

Say why analytics engines read columnar files — columnar files scan less data
Place file-format decisions on the storage stage of the data engineering lifecycle
Point at the sources that make columnar claims exact — layout, statistics, memory
Name the parts of a Parquet file — row groups, column chunks, pages, statistics
Explain what per-row-group statistics buy a reader through predicate pushdown
Reach the authoritative wording instead of paraphrasing the format
Name the twelve-factor principles that map onto pipeline jobs
Treat config, backing services, logs, and disposability as shared vocabulary