Learn
ci
13 registry resources · 5 tutorials, 2 skills · roadmap: Data Quality, Python + SQL, Reliability, Warehousing
What this topic covers
DE//AGENT currently lists 13 resources under ci: 5 tutorials, 2 skills, 2 hooks. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.
Also present: 2 templates, 1 plugin, 1 pattern.
What you can learn here
Documented learning objectives across these resources:
- Test a transform without a Spark cluster — via Pipeline Test Author
- Assert invariants instead of golden files — via Pipeline Test Author
- Make a pipeline test deterministic — via Pipeline Test Author
- Capture cost-relevant metrics from a Spark run — via Spark Cost Regression Guard
- Store and compare a baseline — via Spark Cost Regression Guard
- Explain which stage caused a regression — via Spark Cost Regression Guard
- Configure a pre-commit check over staged schema files — via Enforce contracts in CI with the schema change check
- State what lets a breaking change pass — MIGRATION.md or a deprecated_at annotation — via Enforce contracts in CI with the schema change check
13 more objectives live on the individual resource pages.
Where this fits in the learning roadmap
These resources map to stages: Data Quality (6), Python + SQL (1), Reliability (4), Warehousing (2). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.
Compare the registry's options
Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.
| Resource | Type | Difficulty | Verification | Score |
|---|---|---|---|---|
| Pipeline Test Author | Skill | intermediate | community flags | 38 |
| Spark Cost Regression Guard | Skill | advanced | community flags | 37 |
| Contracts as a Quality Gate | Pattern | intermediate | community | 34.75 |
| Enforce contracts in CI with the schema change check | Tutorial | beginner | community | 34.15 |
| Fix SQL lint with the SQLFluff core rule pack | Tutorial | beginner | community | 34.15 |
| Install cost guardrails before the invoice | Tutorial | beginner | community | 34.15 |
| Run the dbt warehouse tests starter end to end | Tutorial | beginner | community | 34.15 |
| Gate promotion with a pre-deploy DQ check | Tutorial | intermediate | community | 34.15 |
| Contract Kit | Plugin | intermediate | community flags | 34.1 |
| Data Contract Template | Template | intermediate | community flags | 33.1 |
| Spark Cost Check | Hook | advanced | community flags | 31.1 |
| dbt Project Template | Template | beginner | community flags | 28.1 |
1 more in the full list below.
How scores are computed: source quality score.
Where to start
Highest composite score first:
- Pipeline Test Author score 38 — intermediate
- Spark Cost Regression Guard score 37 — advanced
- Contracts as a Quality Gate score 34.75 — intermediate
Scores rank provenance, completeness and trust — read how verification works before adopting anything.
Good fits
Start here if you're new: Enforce contracts in CI with the schema change check, Fix SQL lint with the SQLFluff core rule pack, Install cost guardrails before the invoice, Run the dbt warehouse tests starter end to end.
Reach for these with production experience: Spark Cost Regression Guard, Spark Cost Check.
Cautions and limitations
Security-flagged entries: Pipeline Test Author (2 flags), Spark Cost Regression Guard (2 flags), Contract Kit (2 flags), Data Contract Template (2 flags), Spark Cost Check (2 flags), dbt Project Template (4 flags), Pre-deployment DQ Check (4 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.
Known limits: A single team owning the whole chain (Contracts as a Quality Gate)
Production use cases
- Pipeline Test Author — Builds a local DuckDB/pyarrow harness with tiny fixtures so pipeline logic is testable in CI without a cluster, and asserts on invariants, not snapshots. details
- Spark Cost Regression Guard — Compares a job's shuffle/spill/scan metrics against a stored baseline and fails the change if cost-per-run regresses beyond a threshold, with the diff explained. details
- Enforce contracts in CI with the schema change check — Install the Schema Change Check as a pre-commit gate over staged schema files so a breaking change exits non-zero unless a migration plan or deprecation annotation rides along with… details
- Fix SQL lint with the SQLFluff core rule pack — Install the SQLFluff core rule pack, point it at your warehouse dialect, and lint a fixture that fails on purpose: indentation, keyword capitalisation, SELECT star, and unused CTEs… details
Install and inspect
Real install commands from the registry — read the source before running any of them:
npx de-agent-platform add pipeline-test-authorPipeline Test Author — review first
npx de-agent-platform add spark-cost-regression-guardSpark Cost Regression Guard — review first
amp plugin add de-agent/contract-kitContract Kit — review first
amp template new de-agent/data-contract-template ./contractsData Contract Template — review first
Further study
Adjacent topics: tutorial (5 shared), cost (3 shared), beginner (3 shared), hook (3 shared), testing (3 shared), contracts (3 shared).
Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.
Full resource list
| Resource | Type | Score |
|---|---|---|
| Pipeline Test Author | Skill | 38 |
| Spark Cost Regression Guard | Skill | 37 |
| Contracts as a Quality Gate | Pattern | 34.75 |
| Enforce contracts in CI with the schema change check | Tutorial | 34.15 |
| Fix SQL lint with the SQLFluff core rule pack | Tutorial | 34.15 |
| Install cost guardrails before the invoice | Tutorial | 34.15 |
| Run the dbt warehouse tests starter end to end | Tutorial | 34.15 |
| Gate promotion with a pre-deploy DQ check | Tutorial | 34.15 |
| Contract Kit | Plugin | 34.1 |
| Data Contract Template | Template | 33.1 |
| Spark Cost Check | Hook | 31.1 |
| dbt Project Template | Template | 28.1 |