The dataset engine: validated content that cannot render lies

How typed datasets, resolvers that throw, and a prebuild QA gate keep 200 pages honest.

• typescript
• validation
• content-pipeline
• beginners

What is a dataset page?

A dataset here is a JSON file describing pages: each entry holds the slug, title, search intent, and data hooks — references like “words from these categories” or “speaking rows from unit 3, lessons 1–2”. A resolver is the function that turns those hooks into real curriculum rows at build time. A QA gate is a script that fails the build when a dataset breaks the rules. Authors write data; the engine decides what ships.

The five families

Page datasets live in src/data/seo/*.json, one file per family:

Each entry carries a status of draft or published — only published pages render. Drafts are how half-finished ideas wait without leaking onto the site.

Resolvers that throw

Real code from src/lib/seo.ts (verbatim, trimmed to the word resolver):

export function wordsFor(page: SeoPage): Word[] {
  const words: Word[] = [];
  if (page.categories) words.push(...vocab.filter((w) => page.categories!.includes(w.category)));
  for (const id of page.wordIds ?? []) {
    const w = vocab.find((x) => x.id === id);
    if (!w) throw new Error(`seo/lib: unknown word id "${id}" on page "${page.slug}"`);
    if (!words.includes(w)) words.push(w);
  }
  return words;
}

Three sibling resolvers work the same way: speakingFor pulls lesson-script rows by unit and lesson number, grammarNotesFor verifies every picked lesson resolved exactly one row (count mismatch throws), and bridgesForPage pulls Hindi-bridge rows by unit. The rule is absolute: every word, phrase, or grammar row on a generated page traces to vocab.json or units.json, or it is page-level prose. Nothing in between.

The QA gate

Resolvers catch drift. The validator catches everything else. Real thresholds from tools/validate-content.mjs (verbatim):

const MIN = {
  vocabulary: { words: 10 },
  phrases: { phrases: 8 },
  'hindi-bridge': { bridges: 5 },
  'english-path': { sentences: 5 },
  grammar: { notes: 1 },
};

The same script also fails on schema violations, duplicate slugs/titles/descriptions, slug collisions with the curated taxonomy, broken internal links, Latin script in Marathi answer fields (and Devanagari in English fields), and published pages with no practice items. It warns — without failing — on near-duplicate intents and draft debt. And because package.json wires it into prebuild (node ./tools/validate-content.mjs && node ./tools/build-search-index.mjs), none of this is advisory: a red dataset stops the deploy.

The alternative it beat

The alternative was trusting the datasets — write JSON carefully, review carefully, ship. Careful works until the 40th page, then a renamed category silently empties three pages and nobody notices for months because the pages still render. They render empty. The throw-at-build layer exists because the failure mode of data-driven sites isn’t crashing — it’s confidently displaying nothing.