Operations

Production Operations

Day-two work: monitoring, on-call, backfills, deployments and capacity.

6 resources 5 types

Inside this category

It currently holds 4 resources: 1 plugin, 1 hook, 1 runbook. Mapped to roadmap stage: Reliability. Verification mix: 4 community — check each entry's evidence before adopting.

Where to start: Runbook Runner (48) , Production Write Confirmation (47.29) , Pipeline failed (36.07)

Tutorial 34 adequate

Plan and execute a backfill that doesn't break production

Historical output is wrong or missing, and the fix touches a production table: define correct first, map the blast radius from lineage, then shadow-write in date batches smallest-first, validate a sample per batch, and swap atomically — behind a gate that demands a rollback path.

Details

intermediate tutorialreliability community

Tutorial 34 adequate

Production gotchas — silent failures and partial writes

The failures that become pages are the ones nobody was watching: a regression hiding in growth, a cron change that silently widens the freshness SLA, and appends that duplicate on retry — cap each with a guard, a computed worst-case lag, and an atomic partition replace.

Details

intermediate tutorialreliability community

Plugin 36 adequate

Runbook Runner

Loads the production playbooks into the agent, matches an alert to a runbook, and walks the operator through diagnosis and mitigation step by step.

Details

beginner pluginrunbooks community

Hook 34 adequate

Production Write Confirmation

Force human confirmation for any write against a production data target.

Details

beginner hookproduction community

Runbook 29 adequate

Pipeline failed

A task or DAG run has failed and downstream work is blocked — triage, classify and recover.

Details

beginner incidentorchestration community

Reference 24 unproven

Reading CI Failures Across Pipelines, Not One Build at a Time

Jenkins is good at explaining a single build.

Details

community