Compute unit economics for a pipeline

At a glance

Learning objectives

• Attribute compute cost to a pipeline run
• Define a unit metric that resists regressions
• Set an alert on cost-per-run, not on total spend

What you need

• A billing/usage export and pipeline and query logs
• Per-run compute granularity from the cluster or warehouse
• [object Object]

Steps

1. Attribute spend to pipelines and tables. (from agent-cost-optimizer )
2. Rank by cost and by wasteful ratio (scan vs needed). (from agent-cost-optimizer )
3. Pick a unit metric with a stable denominator — $ per pipeline run, $ per 1M rows processed, or DBU/credits per model build. (from skill-cost-unit-economics )
4. Total spend rises with growth and hides regressions; cost-per-unit is flat when growth is healthy and spikes the day a join starts fanning out. (from skill-cost-unit-economics )
5. Define the unit metric to watch afterwards. (from agent-cost-optimizer )
6. Set an alert on cost-per-run, not on total spend. (from skill-cost-unit-economics )

Check yourself

• Cost-per-run stays flat while volume grows and total spend rises in the same period — proving the denominator resists growth.
• A deliberate join fan-out in a test run doubles cost-per-run and fires the alert while total spend still looks normal.

Worked example

Attribute spend to pipelines and tables.
Rank by cost and by wasteful ratio (scan vs needed).
Propose changes ordered by savings / risk.
Define the unit metric to watch afterwards.

- $ per pipeline run (stable denominator)
- $ per 1M rows processed
- DBU/credits per model build

Total spend rises with growth and hides regressions. Cost-per-unit is flat when
growth is healthy and spikes the day a join starts fanning out.

Sourced from

• Cost Unit-Economics Reporter
• Cost Optimizer

Security scan

Last sweep 2026-10-24: 0 finding(s), 0 marked as examples.

Common gotchas

• Total spend rises with growth and hides regressions — a healthy month can hide a 4x regression.
• Never accept a recommendation that could change results — the Cost Optimizer flags those for a reason.
• The optimizer never applies a change — apply savings through your own review, not through the agent.

Production considerations

• Define the unit metric to watch after every optimization.
• Validate savings estimates against historical billing after applying the change.

Diagram

billing export
   |
   |- attribute spend --> pipelines + tables
   |
   |- rank by cost + wasteful ratio --> top offenders
   |
   |- define unit metric --> $ per run / per 1M rows --> alert on cost-per-run

Registry stats