DE//AGENT
Explore
Tutorial Lesson 4 of 10 · The Cloud Series community beginner

Cut Athena bytes scanned with partitions and projection

Walk one expensive Athena query through the Partition Advisor's levers in order: confirm the per-TB bill model, partition on the dominant filter column, replace MSCK REPAIR with partition projection, compact small files, and move to columnar compression. Every move is judged by a single number — bytes scanned.

At a glance

When Athena cost climbs with no change in query count; A partitioned table still scans everything because partition metadata relies on MSCK REPAIR
Where AWS
Why Bytes scanned is the only thing Athena bills, so applying the advisor's four levers in order is the shortest path to a smaller invoice on the next run.

Learning objectives

  • Read the scanned-bytes metric per query
  • Apply partition projection for high-cardinality columns
  • Compact small files to cut per-request overhead

What you need

  • An Athena workgroup and a table you can query
  • The scanned-bytes metric for recent queries
  • [object Object]

Steps

  1. Judge every recommendation by one question — does this reduce bytes scanned? You pay per TB scanned, and nothing else matters. (from skill-athena-partition-advisor )
  2. Partition on the dominant filter column (usually date). (from skill-athena-partition-advisor )
  3. Use partition projection instead of `MSCK REPAIR` for predictable layouts. (from skill-athena-partition-advisor )
  4. Compact small files — per-file overhead dominates at < 16 MB. (from skill-athena-partition-advisor )
  5. Go columnar + compression (Parquet + snappy/zstd) — 10-100x vs CSV. (from skill-athena-partition-advisor )

Check yourself

  • Re-running the expensive query after the levers land shows the scanned-bytes metric dropping to only the touched date partitions.
  • A table whose files average under 16 MB is compacted, and the per-query bill no longer carries per-file overhead.

Worked example

The four levers, in order — from skill-athena-partition-advisor 

## Levers, in order
1. Partition on the dominant filter column (usually date).
2. Partition projection instead of `MSCK REPAIR` for predictable layouts.
3. Compact small files — per-file overhead dominates at < 16 MB.
4. Columnar + compression (Parquet + snappy/zstd) — 10-100x vs CSV.

Bill model — you pay per TB scanned — from skill-athena-partition-advisor 

## Bill model
You pay per TB scanned. Every recommendation is judged by "does this reduce
bytes scanned?" — nothing else matters.

Sourced from

Security scan

Last sweep 2026-10-24: 0 finding(s), 0 marked as examples.

Common gotchas

  • Every recommendation is judged by bytes scanned — a change that adds complexity without cutting scan is a no.
  • Per-file overhead dominates at < 16 MB — small files inflate the bill even when pruning works.
  • Partition projection replaces MSCK REPAIR only for predictable layouts.

Production considerations

  • Track the scanned-bytes metric per query so layout regressions show up the day they land.
  • Write Parquet with snappy/zstd; CSV inputs scan 10-100x more bytes for the same data.

Diagram

athena bytes pass
   |
   |- read scanned-bytes metric --> baseline per query
   |
   |- partition on dominant filter column --> projection replaces MSCK REPAIR
   |
   |- compact small files (< 16 MB overhead) --> parquet + snappy/zstd
   |
   --> re-run query --> bytes scanned drops

Registry stats

Verification community · Oct 21, 2026
Completeness 81
Trust multiplier ×1.25
Composite 34.15 · adequate
Freshness 3 days since last source check
Difficulty beginner
Version 0.1.0
License CC-BY-4.0
Security flags none