small-files

DE//AGENT currently lists 4 resources under small-files: 1 tutorial, 1 skill, 1 runbook. Everything below is derived from the live registry — counts, capabilities, verification state and security posture come from the curated records, never from marketing copy.

Also present: 1 pattern.

Documented learning objectives across these resources:

• Read an Iceberg metadata table to size the problem — via Iceberg Table Maintenance
• Pick a compaction strategy and rewrite interval — via Iceberg Table Maintenance
• Expire snapshots without breaking time travel — via Iceberg Table Maintenance
• Recognize the small-files signature from file sizes and planning time — via Diagnose a small-files explosion
• Query metadata tables to quantify fragmentation — via Diagnose a small-files explosion
• Fix the layout, not just the symptoms — via Diagnose a small-files explosion

These resources map to stage: Lakehouse (4). Follow the four-phase roadmap to place this topic inside a progression instead of learning it in isolation.

Every entry, its shape and its trust state — sorted by the registry's composite score. This table is the honest version of a marketing comparison: no sponsored placements, no unverifiable claims.

How scores are computed: source quality score.

Highest composite score first:

1. Iceberg Table Maintenance score 36.5 — advanced
2. Diagnose a small-files explosion score 34.15 — intermediate
3. Small files explosion score 29.25 — intermediate

Scores rank provenance, completeness and trust — read how verification works before adopting anything.

Reach for these with production experience: Iceberg Table Maintenance.

Security-flagged entries: Iceberg Table Maintenance (2 flags), Small files explosion (3 flags), Compaction & File-Size Management (3 flags). Flags mean the record declares capabilities like credential use or write access — read the security posture on each page before connecting.

Known limits: Low-frequency bulk writes that already produce large files (Compaction & File-Size Management)

• Iceberg Table Maintenance — Diagnoses small-file and manifest bloat and generates the compaction, snapshot expiry and orphan-file cleanup plan for an Iceberg table. details
• Diagnose a small-files explosion — Follow the small-files runbook from symptom to prevention: confirm the tens-of-thousands-of-tiny-files signature, query the metadata tables for the average file size, compact to a … details
• Small files explosion — A table has fragmented into thousands of tiny files and query planning now dominates runtime. details

Real install commands from the registry — read the source before running any of them:

npx de-agent-platform add iceberg-table-maintenance

Iceberg Table Maintenance — review first

Adjacent topics: compaction (4 shared), lakehouse (3 shared).

Study path: the roadmap places this topic in a progression · the scoring model explains the numbers · the trust rule catalog shows every security check these entries passed through.