Compare
Spark / PySpark: 14 options
14 options compared · 7 tutorial, 2 skill, 1 plugin, 1 agent, 1 template, 1 runbook, 1 reference
Comparing 14 options
Spark / PySpark — Distributed dataframes: lazy DAGs, shuffles, skew, partitioning and physical plans. The registry currently holds 14 resources: 7 tutorials, 2 skills, 1 plugin, 1 agent, 1 template, 1 runbook, 1 reference.
Side by side
Every field below comes from the curated record — nothing is sponsored or ranked by payment:
| Resource | Type | Difficulty | Verification | Score |
|---|---|---|---|---|
| Spark Data Quality Review | Skill | intermediate | community | 41 |
| Spark Skew Diagnosis | Skill | advanced | community | 38.5 |
| Spark job is slow | Runbook | advanced | community | 36.75 |
| Spark Optimizer | Agent | advanced | community | 34.5 |
| Start a PySpark project from the template | Tutorial | beginner | community | 34.15 |
| Aggregations and windows without the skew | Tutorial | intermediate | community | 34.15 |
| Diagnose a skewed Spark stage | Tutorial | intermediate | community | 34.15 |
| Read a Spark event log with the Spark Lens plugin | Tutorial | intermediate | community | 34.15 |
| Review a PySpark transform for silent quality failures | Tutorial | intermediate | community | 34.15 |
| The 3am batch incident path | Tutorial | advanced | community | 34.15 |
| Tune a slow job — memory, partitions, baselines | Tutorial | advanced | community | 34.15 |
| Spark Lens | Plugin | advanced | community | 34.1 |
| PySpark Project Template | Template | intermediate | community | 34.1 |
| Apache Spark Documentation | Reference | — | community | 27.7 |
Trust breakdown
Verification mix: 14 community. No evidence gaps: every claim on file matches the record. Read how verification works — nothing here is independently audited yet, and the site says so everywhere it matters.
Security posture
Flagged entries: Spark Data Quality Review (1), Spark Skew Diagnosis (2), Spark job is slow (1), Spark Optimizer (1), Spark Lens (2), PySpark Project Template (2). Flags record declared capabilities (credentials, write access, network calls) — they are facts to review, not verdicts. Every entry also passes the 30-rule static trust scan; findings print on each detail page.
How to choose
How to choose: start with Start a PySpark project from the template for the gentlest on-ramp; defer Spark Skew Diagnosis, Spark job is slow until you have production context. Weight verification and evidence over score when two entries are close, and check the last-verified date on each page — the registry re-checks every 180 days.