Amazon S3

DE//AGENT currently lists 5 resources under Amazon S3: 2 skills, 2 projects, 1 template. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Documented learning objectives across these resources:

• Size Glue workers from data volume, not habit — via AWS Glue Job Reviewer
• Use job bookmarks correctly — via AWS Glue Job Reviewer
• Avoid full-table reprocessing from catalogue mistakes — via AWS Glue Job Reviewer
• Read the scanned-bytes metric per query — via Athena Partition Advisor
• Apply partition projection for high-cardinality columns — via Athena Partition Advisor
• Compact small files to cut per-request overhead — via Athena Partition Advisor
• Handle pagination, retries and rate limits correctly — via API → S3 → Spark → Parquet
• Write an idempotent partition overwrite — via API → S3 → Spark → Parquet

4 more objectives live on the individual resource pages.

These resources map to stages: Cloud (4), Data Processing (1). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

How scores are computed: source quality score.

Highest composite score first:

1. AWS Glue Job Reviewer score 37 — intermediate
2. Athena Partition Advisor score 33.4 — intermediate
3. API → S3 → Spark → Parquet score 32.15 — intermediate

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Reach for these with production experience: Terraform AWS Data Platform.

Security-flagged entries: AWS Glue Job Reviewer (2 flags), Athena Partition Advisor (3 flags), API → S3 → Spark → Parquet (5 flags), S3 → Glue → Athena (5 flags), Terraform AWS Data Platform (5 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

• AWS Glue Job Reviewer — Reviews Glue/Spark jobs for worker sizing, job bookmarks, partition pruning, and the catalogue/schema pitfalls that cause silent full reprocessing. details
• Athena Partition Advisor — Recommends partition projection, bucketing or file-layout changes to reduce Athena bytes-scanned, and writes the projection DDL. details
• API → S3 → Spark → Parquet — Land a paginated REST API into S3 raw, transform with PySpark into partitioned Parquet, and make the whole thing idempotent and testable offline. details
• S3 → Glue → Athena — A serverless, query-in-place pattern: land Parquet in S3, catalogue it with Glue, and query with Athena while keeping scanned bytes (and cost) low. details

Real install commands from the registry — read the source before running any of them:

npx de-agent-platform add aws-glue-job-reviewer

AWS Glue Job Reviewer — review first

npx de-agent-platform add athena-partition-advisor

Athena Partition Advisor — review first

amp template new de-agent/terraform-aws-data-platform ./infra

Terraform AWS Data Platform — review first

Official source: Amazon S3 — start there for the canonical documentation, then come back for the vetted registry entries.

Adjacent topics: AWS Glue (4 shared), Amazon Athena (3 shared), Apache Parquet (2 shared), Apache Spark (2 shared), Terraform (2 shared), cost (2 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.