Learn
cost
24 registry resources · 12 tutorials, 4 skills · roadmap: Reliability, Cloud, Data Processing, Warehousing
What this topic covers
DE//AGENT currently lists 24 resources under cost: 12 tutorials, 4 skills, 2 plugins. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.
Also present: 2 hooks, 2 agents, 1 runbook, 1 pattern.
What you can learn here
Documented learning objectives across these resources:
- Size Glue workers from data volume, not habit — via AWS Glue Job Reviewer
- Use job bookmarks correctly — via AWS Glue Job Reviewer
- Avoid full-table reprocessing from catalogue mistakes — via AWS Glue Job Reviewer
- Capture cost-relevant metrics from a Spark run — via Spark Cost Regression Guard
- Store and compare a baseline — via Spark Cost Regression Guard
- Explain which stage caused a regression — via Spark Cost Regression Guard
- Attribute compute cost to a pipeline run — via Compute unit economics for a pipeline
- Define a unit metric that resists regressions — via Compute unit economics for a pipeline
31 more objectives live on the individual resource pages.
Where this fits in the learning roadmap
These resources map to stages: Reliability (11), Cloud (8), Data Processing (3), Warehousing (2). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.
Compare the registry's options
Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.
| Resource | Type | Difficulty | Verification | Score |
|---|---|---|---|---|
| AWS Glue Job Reviewer | Skill | intermediate | community flags | 37 |
| Spark Cost Regression Guard | Skill | advanced | community flags | 37 |
| Spark job suddenly costs 4x | Runbook | advanced | community flags | 36.75 |
| Spark Optimizer | Agent | advanced | community flags | 34.5 |
| Compute unit economics for a pipeline | Tutorial | beginner | community | 34.15 |
| Cut Athena bytes scanned with partitions and projection | Tutorial | beginner | community | 34.15 |
| Install cost guardrails before the invoice | Tutorial | beginner | community | 34.15 |
| Review a Glue job for sizing, bookmarks, and catalogue drift | Tutorial | beginner | community | 34.15 |
| Attribute a 4x Spark cost regression | Tutorial | intermediate | community | 34.15 |
| Attribute spend with the Cost Lens plugin | Tutorial | intermediate | community | 34.15 |
| Govern cloud spend — right-sizing, resource monitors, and unit economics | Tutorial | intermediate | community | 34.15 |
| Incident — the job that quadrupled in cost overnight | Tutorial | intermediate | community | 34.15 |
12 more in the full list below.
How scores are computed: source quality score.
Where to start
Highest composite score first:
- AWS Glue Job Reviewer score 37 — intermediate
- Spark Cost Regression Guard score 37 — advanced
- Spark job suddenly costs 4x score 36.75 — advanced
Scores rank provenance, completeness and trust — read how verification works before adopting anything.
Good fits
Start here if you're new: Compute unit economics for a pipeline, Cut Athena bytes scanned with partitions and projection, Install cost guardrails before the invoice, Review a Glue job for sizing, bookmarks, and catalogue drift.
Reach for these with production experience: Spark Cost Regression Guard, Spark job suddenly costs 4x, Spark Optimizer.
Cautions and limitations
Security-flagged entries: AWS Glue Job Reviewer (2 flags), Spark Cost Regression Guard (2 flags), Spark job suddenly costs 4x (1 flag), Spark Optimizer (1 flag), Spark Lens (2 flags), Athena Partition Advisor (3 flags), dbt Full-Refresh Guard (1 flag), Spark Cost Check (2 flags), Cost Unit-Economics Reporter (4 flags), Cost Guardrails (3 flags), Cost Lens (5 flags), Cost Optimizer (5 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.
Known limits: Negligible fixed-cost workloads (Cost Guardrails)
Production use cases
- AWS Glue Job Reviewer — Reviews Glue/Spark jobs for worker sizing, job bookmarks, partition pruning, and the catalogue/schema pitfalls that cause silent full reprocessing. details
- Spark Cost Regression Guard — Compares a job's shuffle/spill/scan metrics against a stored baseline and fails the change if cost-per-run regresses beyond a threshold, with the diff explained. details
- Spark job suddenly costs 4x — One job's compute cost quadrupled without an obvious functional change — attribute it and reverse the regression. details
- Spark Optimizer — Diagnose and fix slow or expensive Spark jobs from their metrics. details
Install and inspect
Real install commands from the registry — read the source before running any of them:
npx de-agent-platform add aws-glue-job-reviewerAWS Glue Job Reviewer — review first
npx de-agent-platform add spark-cost-regression-guardSpark Cost Regression Guard — review first
amp plugin add de-agent/spark-lensSpark Lens — review first
npx de-agent-platform add athena-partition-advisorAthena Partition Advisor — review first
Further study
Adjacent topics: tutorial (12 shared), finops (7 shared), performance (5 shared), plugin (4 shared), attribution (4 shared), regression (4 shared).
Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.
Full resource list
| Resource | Type | Score |
|---|---|---|
| AWS Glue Job Reviewer | Skill | 37 |
| Spark Cost Regression Guard | Skill | 37 |
| Spark job suddenly costs 4x | Runbook | 36.75 |
| Spark Optimizer | Agent | 34.5 |
| Compute unit economics for a pipeline | Tutorial | 34.15 |
| Cut Athena bytes scanned with partitions and projection | Tutorial | 34.15 |
| Install cost guardrails before the invoice | Tutorial | 34.15 |
| Review a Glue job for sizing, bookmarks, and catalogue drift | Tutorial | 34.15 |
| Attribute a 4x Spark cost regression | Tutorial | 34.15 |
| Attribute spend with the Cost Lens plugin | Tutorial | 34.15 |
| Govern cloud spend — right-sizing, resource monitors, and unit economics | Tutorial | 34.15 |
| Incident — the job that quadrupled in cost overnight | Tutorial | 34.15 |