Learn
performance
17 registry resources · 7 tutorials, 4 skills · roadmap: Data Processing, Foundations, Warehousing, Python + SQL, Reliability, Lakehouse
What this topic covers
DE//AGENT currently lists 17 resources under performance: 7 tutorials, 4 skills, 3 runbooks. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.
Also present: 2 agents, 1 plugin.
What you can learn here
Documented learning objectives across these resources:
- Read a plan bottom-up and name the expensive node — via SQL Query Plan Explainer
- Detect cardinality misestimation from stale statistics — via SQL Query Plan Explainer
- Decide between an index, a rewrite, or a stats refresh — via SQL Query Plan Explainer
- Read task-level duration histograms to confirm skew — via Spark Skew Diagnosis
- Choose between AQE skew join, salting and broadcast — via Spark Skew Diagnosis
- Reproduce skew locally with a bounded dataset — via Spark Skew Diagnosis
- Capture cost-relevant metrics from a Spark run — via Spark Cost Regression Guard
- Store and compare a baseline — via Spark Cost Regression Guard
20 more objectives live on the individual resource pages.
Where this fits in the learning roadmap
These resources map to stages: Data Processing (7), Foundations (1), Warehousing (4), Python + SQL (2), Reliability (2), Lakehouse (1). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.
Compare the registry's options
Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.
| Resource | Type | Difficulty | Verification | Score |
|---|---|---|---|---|
| SQL Query Plan Explainer | Skill | intermediate | community flags | 38.5 |
| Spark Skew Diagnosis | Skill | advanced | community flags | 38.5 |
| Spark Cost Regression Guard | Skill | advanced | community flags | 37 |
| Spark job is slow | Runbook | advanced | community flags | 36.75 |
| Warehouse Clustering Advisor | Skill | advanced | community flags | 36.5 |
| Spark Optimizer | Agent | advanced | community flags | 34.5 |
| Diagnose a skewed Spark stage | Tutorial | intermediate | community | 34.15 |
| Format-driven performance incidents — small scans and wrong layouts | Tutorial | intermediate | community | 34.15 |
| Incident — the dashboard query that went slow | Tutorial | intermediate | community | 34.15 |
| Make queries prune — clustering keys and scan diagnosis | Tutorial | intermediate | community | 34.15 |
| Read a Spark event log with the Spark Lens plugin | Tutorial | intermediate | community | 34.15 |
| Read an EXPLAIN plan and pick the one fix | Tutorial | intermediate | community | 34.15 |
5 more in the full list below.
How scores are computed: source quality score.
Where to start
Highest composite score first:
- SQL Query Plan Explainer score 38.5 — intermediate
- Spark Skew Diagnosis score 38.5 — advanced
- Spark Cost Regression Guard score 37 — advanced
Scores rank provenance, completeness and trust — read how verification works before adopting anything.
Good fits
Reach for these with production experience: Spark Skew Diagnosis, Spark Cost Regression Guard, Spark job is slow.
Cautions and limitations
Security-flagged entries: SQL Query Plan Explainer (1 flag), Spark Skew Diagnosis (2 flags), Spark Cost Regression Guard (2 flags), Spark job is slow (1 flag), Warehouse Clustering Advisor (2 flags), Spark Optimizer (1 flag), Spark Lens (2 flags), Small files explosion (3 flags), Warehouse query is slow (3 flags), Cost Optimizer (5 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.
Production use cases
- SQL Query Plan Explainer — Reads EXPLAIN / EXPLAIN ANALYZE output and explains scans, joins, spills and estimated-vs-actual row skew, then proposes the single highest-impact change. details
- Spark Skew Diagnosis — Walks an agent through reading the Spark UI/event log to distinguish key skew, partition skew, and shuffle-write amplification, then selects between salting, AQE skew join, broadca… details
- Spark Cost Regression Guard — Compares a job's shuffle/spill/scan metrics against a stored baseline and fails the change if cost-per-run regresses beyond a threshold, with the diff explained. details
- Spark job is slow — A batch Spark job's runtime has jumped — find whether it is skew, partition count, spill or over-scan. details
Install and inspect
Real install commands from the registry — read the source before running any of them:
npx de-agent-platform add sql-query-plan-explainerSQL Query Plan Explainer — review first
npx de-agent-platform add spark-skew-diagnosisSpark Skew Diagnosis — review first
npx de-agent-platform add spark-cost-regression-guardSpark Cost Regression Guard — review first
npx de-agent-platform add warehouse-clustering-advisorWarehouse Clustering Advisor — review first
Further study
Adjacent topics: tutorial (7 shared), cost (5 shared), Apache Spark (2 shared), Databricks (2 shared), incident (2 shared), plugin (2 shared).
Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.
Full resource list
| Resource | Type | Score |
|---|---|---|
| SQL Query Plan Explainer | Skill | 38.5 |
| Spark Skew Diagnosis | Skill | 38.5 |
| Spark Cost Regression Guard | Skill | 37 |
| Spark job is slow | Runbook | 36.75 |
| Warehouse Clustering Advisor | Skill | 36.5 |
| Spark Optimizer | Agent | 34.5 |
| Diagnose a skewed Spark stage | Tutorial | 34.15 |
| Format-driven performance incidents — small scans and wrong layouts | Tutorial | 34.15 |
| Incident — the dashboard query that went slow | Tutorial | 34.15 |
| Make queries prune — clustering keys and scan diagnosis | Tutorial | 34.15 |
| Read a Spark event log with the Spark Lens plugin | Tutorial | 34.15 |
| Read an EXPLAIN plan and pick the one fix | Tutorial | 34.15 |