data-quality

DE//AGENT currently lists 18 resources under data-quality: 8 tutorials, 2 skills, 2 runbooks. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Also present: 2 references, 1 rule pack, 1 hook, 1 agent, 1 pattern.

Documented learning objectives across these resources:

• Identify the five classes of silent failure in a Spark transform — via Spark Data Quality Review
• Add pre/post row-count and null assertions to a job — via Spark Data Quality Review
• Recognise join fan-out caused by non-unique keys — via Spark Data Quality Review
• Rank tests by the cost of the failure they prevent — via dbt Test Writer
• Write a custom singular test — via dbt Test Writer
• Wire freshness on a source — via dbt Test Writer
• Configure SQLFluff for your warehouse dialect — via SQLFluff Core Rule Pack
• Customize rules for team standards — via SQLFluff Core Rule Pack

24 more objectives live on the individual resource pages.

These resources map to stages: Python + SQL (1), Data Quality (13), Warehousing (2), Data Processing (2). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

6 more in the full list below.

How scores are computed: source quality score.

Highest composite score first:

1. Spark Data Quality Review score 41 — intermediate
2. dbt Test Writer score 40.5 — beginner
3. SQLFluff Core Rule Pack score 39.7 — beginner

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Start here if you're new: dbt Test Writer, SQLFluff Core Rule Pack, Deduplicate records without rewriting the table, Name the four dimensions of data quality.

Security-flagged entries: Spark Data Quality Review (1 flag), dbt Test Writer (1 flag), SQLFluff Core Rule Pack (1 flag), Duplicate records (1 flag), Data quality failure (2 flags), Data Quality Reviewer (2 flags), Pre-deployment DQ Check (4 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

Known limits: Throwaway exploratory pipelines (Quarantine then Publish)

• Spark Data Quality Review — A review skill an agent runs against a Spark/PySpark transform. It checks null propagation, join fan-out, silent type coercion, non-deterministic row counts and missing post-write … details
• dbt Test Writer — Proposes dbt tests ranked by blast radius: uniqueness/not-null on the grain key, accepted values on enums, relationships on foreign keys, and freshness on sources. details
• SQLFluff Core Rule Pack — A curated SQLFluff configuration with rules optimized for modern data engineering workflows. Includes dialect-specific rules for dbt (Jinja), Spark SQL, Snowflake, BigQuery, and Re… details
• Duplicate records — Row counts are inflated and metrics double-counted after a retry, a fan-out join or a backfill. details

Real install commands from the registry — read the source before running any of them:

npx de-agent-platform add spark-data-quality-review

Spark Data Quality Review — review first

npx de-agent-platform add dbt-test-writer

dbt Test Writer — review first

pip install sqlfluff==3.0.0 sqlfluff-templater-dbt

SQLFluff Core Rule Pack — review first

Adjacent topics: tutorial (8 shared), testing (3 shared), quarantine (3 shared), Apache Spark (2 shared), dbt (2 shared), beginner (2 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.