Data engineering skills

dbt Test Writer

• Rank tests by the cost of the failure they prevent
• Write a custom singular test

Incident Postmortem Writer

• Write a blameless timeline
• Separate trigger from contributing factors

SQL Review

A review pass that catches correctness bugs before the query reaches production.

Airflow DAG Reviewer

• Make every task safe to re-run
• Choose retries and timeouts deliberately

Athena Partition Advisor

• Read the scanned-bytes metric per query
• Apply partition projection for high-cardinality columns

AWS Glue Job Reviewer

• Size Glue workers from data volume, not habit
• Use job bookmarks correctly

Cost Unit-Economics Reporter

• Attribute compute cost to a pipeline run
• Define a unit metric that resists regressions

Data Contract Author

• Specify a freshness and completeness SLA
• Encode a contract as executable tests

dbt Model Review

• State a model's grain and match it to its unique key test
• Choose table vs incremental vs view deliberately

Delta Optimization Advisor

• Decide if a table needs OPTIMIZE from file stats
• Choose Z-ORDER columns from query predicates

Lineage Explainer

• Read the lineages of an asset from a lineage store
• Compute a downstream impact set

PII Classifier

• Classify columns into PII categories
• Choose masking vs hashing vs tokenisation

Pipeline Test Author

• Test a transform without a Spark cluster
• Assert invariants instead of golden files

Schema Change Contract Review

• Apply backward/forward compatibility rules to a diff
• Detect a breaking change hidden behind a "safe" rename

Spark Data Quality Review

• Identify the five classes of silent failure in a Spark transform
• Add pre/post row-count and null assertions to a job

SQL Query Plan Explainer

• Read a plan bottom-up and name the expensive node
• Detect cardinality misestimation from stale statistics

Backfill Planner

• Scope a backfill by date and by dependency
• Detect which downstream tables must also be rebuilt

CDC Merge Designer

• Order change events deterministically
• Handle deletes (tombstones) without losing history

Iceberg Table Maintenance

• Read an Iceberg metadata table to size the problem
• Pick a compaction strategy and rewrite interval

Kafka Consumer Lag Triage

• Read per-partition lag to separate skew from throughput
• Detect a rebalance loop from consumer-group state

MCP Tool Contract Auditor

• Read a tool schema and predict its side effects
• Classify tools by blast radius

Spark Cost Regression Guard

• Capture cost-relevant metrics from a Spark run
• Store and compare a baseline

Spark Skew Diagnosis

• Read task-level duration histograms to confirm skew
• Choose between AQE skew join, salting and broadcast

Warehouse Clustering Advisor

• Separate scan inefficiency from compute inefficiency
• Choose a clustering key from predicate selectivity