partitions

DE//AGENT currently lists 5 resources under partitions: 3 tutorials, 1 skill, 1 pattern. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Documented learning objectives across these resources:

• Read the scanned-bytes metric per query — via Cut Athena bytes scanned with partitions and projection
• Apply partition projection for high-cardinality columns — via Cut Athena bytes scanned with partitions and projection
• Compact small files to cut per-request overhead — via Cut Athena bytes scanned with partitions and projection
• Name the four causes of a slow batch job: skew, partition count, spill, over-scan — via The batch processing mental model
• Explain what a shuffle does to a hot key — via The batch processing mental model
• State the idempotent-write invariant for re-runnable batch jobs — via The batch processing mental model
• Apply the idempotency test to every task before trusting retries — via Idempotency and reruns — make every task safe to run twice
• Replace appends with atomic partition overwrites keyed by the logical date, or MERGE — via Idempotency and reruns — make every task safe to run twice

1 more objectives live on the individual resource pages.

These resources map to stages: Cloud (2), Data Processing (2), Reliability (1). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

How scores are computed: source quality score.

Highest composite score first:

1. Idempotent Partition Overwrite score 34.75 — beginner
2. Cut Athena bytes scanned with partitions and projection score 34.15 — beginner
3. The batch processing mental model score 34.15 — beginner

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Start here if you're new: Idempotent Partition Overwrite, Cut Athena bytes scanned with partitions and projection, The batch processing mental model.

Security-flagged entries: Athena Partition Advisor (3 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

Known limits: Streaming writes with no partition boundary (use MERGE/upsert instead) (Idempotent Partition Overwrite)

• Cut Athena bytes scanned with partitions and projection — Walk one expensive Athena query through the Partition Advisor's levers in order: confirm the per-TB bill model, partition on the dominant filter column, replace MSCK REPAIR with pa… details
• The batch processing mental model — Batch processing in partition terms. Learn what a partition, a shuffle and a spill are, why "the job is slow" resolves into four named causes — skew, partition count, spill, over-s… details
• Idempotency and reruns — make every task safe to run twice — Retries are inevitable, so a task that appends duplicates rows every time it is cleared: hold each task to the idempotency test, replace its partition atomically instead of appendi… details
• Athena Partition Advisor — Recommends partition projection, bucketing or file-layout changes to reduce Athena bytes-scanned, and writes the projection DDL. details

Real install commands from the registry — read the source before running any of them:

npx de-agent-platform add athena-partition-advisor

Athena Partition Advisor — review first

Adjacent topics: tutorial (3 shared), cost (2 shared), idempotency (2 shared), batch (2 shared), projection (2 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.