Docker

DE//AGENT currently lists 3 resources under Docker: 2 templates, 1 project. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Documented learning objectives across these resources:

• Handle pagination, retries and rate limits correctly — via API → S3 → Spark → Parquet
• Write an idempotent partition overwrite — via API → S3 → Spark → Parquet
• Test a Spark transform without a cluster — via API → S3 → Spark → Parquet

These resources map to stages: Cloud (1), Data Processing (2). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

How scores are computed: source quality score.

Highest composite score first:

1. Local Docker Data Stack score 34.1 — beginner
2. PySpark Project Template score 34.1 — intermediate
3. API → S3 → Spark → Parquet score 32.15 — intermediate

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Start here if you're new: Local Docker Data Stack.

Security-flagged entries: Local Docker Data Stack (2 flags), PySpark Project Template (2 flags), API → S3 → Spark → Parquet (5 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

• Local Docker Data Stack — A docker-compose stack (Postgres, Kafka, MinIO, Spark, and a DuckDB CLI) for developing and testing pipelines entirely offline. details
• PySpark Project Template — A testable PySpark project: src/transforms, tests with a local Spark session, a Dockerfile, and a Makefile that runs lint + tests offline. details
• API → S3 → Spark → Parquet — Land a paginated REST API into S3 raw, transform with PySpark into partitioned Parquet, and make the whole thing idempotent and testable offline. details

Real install commands from the registry — read the source before running any of them:

amp template new de-agent/docker-local-stack ./local

Local Docker Data Stack — review first

amp template new de-agent/pyspark-project ./my-pipeline

PySpark Project Template — review first

Official source: Docker (license: Apache-2.0) — start there for the canonical documentation, then come back for the vetted registry entries.

Adjacent topics: Apache Spark (3 shared), PySpark (2 shared), template (2 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.