late-data

DE//AGENT currently lists 3 resources under late-data: 2 tutorials, 1 runbook. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Documented learning objectives across these resources:

• Bound incremental work with a watermark and a lookback window — via Incremental processing with a watermark
• Write incremental output idempotently so a re-run is safe — via Incremental processing with a watermark
• Absorb late-arriving files without re-running history — via Incremental processing with a watermark
• Gate full-refreshes on row thresholds and criticality before they run — via Warehouse production gotchas — cost, full refreshes, late data
• Size lookback windows against real late-arriving data — via Warehouse production gotchas — cost, full refreshes, late data
• Keep warehouse cost visible with weekly top-cost query reviews — via Warehouse production gotchas — cost, full refreshes, late data

These resources map to stages: Data Processing (2), Warehousing (1). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

How scores are computed: source quality score.

Highest composite score first:

1. Incremental processing with a watermark score 34.15 — intermediate
2. Warehouse production gotchas — cost, full refreshes, late data score 34.15 — advanced
3. Late files score 29.25 — intermediate

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Reach for these with production experience: Warehouse production gotchas — cost, full refreshes, late data.

Security-flagged entries: Late files (3 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

• Incremental processing with a watermark — Full reloads stop scaling once a table grows past millions of rows. Process a bounded window instead: a watermark ends it, a lookback absorbs late files, and an idempotent write (p… details
• Warehouse production gotchas — cost, full refreshes, late data — The three ways a healthy warehouse model hurts production: a full-refresh that rebuilds a 4B-row table unannounced, a lookback window too short to catch late-arriving orders, and c… details

Adjacent topics: tutorial (2 shared), batch (2 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.