Data Engineering Mastery Platform · Part 3 of 6

API and data model: what each endpoint does for Priya

API and data model: what each endpoint does for Priya

Follow her clicks through sixteen router groups: what each API does, where her data lives, and why writes are decided server-side.

2026-09-05beginnerFastAPI, SQLAlchemy 2, Pydantic v2, PostgreSQL
  • python
  • fastapi
  • postgres
  • api-design
  • beginners

Her clicks, mapped to APIs

Part 2’s math is useless until Priya can touch it — and letting screens touch the database directly is how progress gets forged (the hub’s BUG-017 story). This part is the locked door between her clicks and the data. Every screen she touches calls one of sixteen router groups. In plain words:

She does this The API does this
Signs up / logs in auth: validates, hashes passwords, issues short + rotating tokens, rate-limits logins
Answers onboarding onboarding: stores her role/experience/exam date, returns a plan + week estimate
Opens dashboard progress: section mastery, streak, due reviews, weak topics
Reads a concept concepts: content + enrichment + her progress on it
Practices questions: next unmastered question; answers graded, progress updated, XP awarded
Reviews flashcards flashcards: SM-2 scheduling, XP for quality ≥ 4
Takes a mock exams: generate papers without answers; submit grades at 70%, XP on pass
Plans study study-plans: auto-distributes concepts across her weeks
Checks stats analytics: summaries, heatmap, weak/strength areas
Competes leaderboard, badges, game_state: XP, levels, ranks, streaks
Exports notes obsidian: builds her vault ZIP in the background
Pays billing: trials, portal, webhooks
Admins (staff) admin: pipeline jobs, publishing, users, audit logs
Health (ops) health: liveness plus an admin-only metrics endpoint

You are here: Part 3 of 6 — the serving layer. Part 2 built the math; this part puts it behind URLs with locked doors. Part 4 fills the tables these endpoints read, and Part 6 proves the whole chain running.

The rule behind the table

Router → service → model, no skips. Routers parse HTTP and authenticate; services hold logic with no SQL dialect and no status codes; models describe tables; Pydantic schemas validate in, serialize out, and generate docs for free. First principle: a layer you can’t test alone is a layer that doesn’t exist. For a weekend prototype this layering is pure overhead — layers scale with lifespan, and this project plans in years.

Concretely, in backend/app/: api/v1/ holds the 16 routers, services/ holds 11 logic modules (exam_service, question_service, progress_service, …), models/ holds the tables, schemas/ holds the Pydantic shapes, core/ holds config, database sessions, security, and rate limits. Learn that sentence and you can find anything in the backend in under a minute.

A request’s full journey

Follow one login click through every checkpoint:

Browser → rate limiter → auth check → Pydantic validation
→ route handler → service → database → service → schema → Browser

Each stop earns its place. The rate limiter throttles guessing at the door (auth_rate_limit on login). Auth verifies the token — short-lived access plus renewable refresh, so a stolen token dies quickly. Validation rejects malformed bodies with 400 before they touch logic. The handler is deliberately thin: parse, authenticate, delegate. The service thinks. The schema shapes the answer and becomes API documentation for free.

Remove any stop and name the consequence: no limiter → brute force; no auth → anyone reads Priya’s history; no validation → crashes deep inside calculations; fat handlers → logic you can only test over HTTP.

One request, every checkpoint
flowchart LR
  B["Browser"]
  R["Rate limit"]
  A["Auth"]
  V["Validate"]
  H["Handler"]
  S["Service"]
  D[("Postgres")]
  B --> R --> A --> V --> H --> S --> D
  D --> S --> B

Seven stops in, two stops out. Security and shape checks happen before any thinking.

One write, fully traced

Real code from backend/app/api/v1/exams.py (submit path):

Server-authoritative rewardsbackend/app/api/v1/exams.py
passed = score >= 0.7 # 70% passing threshold
exam.passed = passed
# BUG-017: award XP server-side on exam pass (backend is source of truth).
if passed:
from app.services.game_state_service import record_exam_pass
await record_exam_pass(db, current_user.id, 50 + round(score * 50))

What. 70% pass threshold; on pass, the backend awards 50 plus up to 50 scaled by score.

Why. The BUG-017 comment is the history: clients awarded XP before, clients can lie — the backend is now the source of truth.

Tradeoff. A deliberate import-cycle break inside the route; the alternative (client-trusted XP) is a cheat code waiting to happen. Lint noise versus incidents — take the noise.

Your turn: audit one write path in your project. Who decides — client or server? Move it server-side with a history comment exactly like this one. Failure mode prevented: forged progress and support tickets you cannot adjudicate.

Reading the data model without drowning

UUID-keyed tables in two families. Content: certifications, concepts with JSONB bodies, questions, graph edges, versions, pipeline jobs. People: users, 0–1000 mastery, attempts, SM-2 cards, plans, analytics buckets, game state, Stripe customers. Fourteen Alembic migrations, one linear chain — the schema’s replayable memory.

Two design notes for beginners. UUIDs (random-looking IDs) cost index size but buy merge safety and unguessable URLs — Priya can’t enumerate other learners by counting. JSONB columns hold evolving content without a migration per new field — flexibility fenced inside one column, while everything queried stays a real column. Flexibility with boundaries, not instead of them.

Errors are a designed surface

Four codes Priya may meet, each with a meaning and a fix: 400 — her request was malformed (check the shape); 401 — log in first (token missing or dead); 429 — slow down (rate limiter, especially login); 500 — our bug, retry later, and it is already in the structured logs. A fifth case matters most: validation errors name the field. “email: invalid format” teaches; “bad request” shrugs. Error messages are user interface — write them for the human on the other side.

Directing an AI agent at this API

You are adding an endpoint to this FastAPI backend.
Goal: <one sentence>
Constraints:
- Router → service → model. No SQL in the handler.
- Request/response bodies are Pydantic models (they generate docs).
- Writes the client must not control go server-side (see BUG-017).
- Add/extend tests; run make test-unit before finishing.
Before coding:
1. Name the router, service function, and tables involved.
2. List the error cases and their status codes.
3. Then implement.

Agents default to fat handlers and trusting clients. This prompt forbids both up front.

Failure modes of serving layers

  • Fat handlers. Logic in routes can only be tested over HTTP — slow, flaky, skipped. Defense: the layer rule, enforced in review.
  • Trusting the client. Forged XP, forged progress (BUG-017, resolved). Defense: server decides; client displays.
  • Leaky errors. Stack traces to browsers leak internals. Defense: shaped error responses; details go to structlog, never to Priya.
  • Missing rate limits. Login without throttling invites guessing. Defense: auth_rate_limit at the door (verified in source).
  • Schema drift. A renamed field breaks every screen at runtime. Defense: the contract test file plus Pydantic as the single shape definition.

Exercises

  1. Pick any row in the Topic 1 table. Find its router file, service, and tables. Time yourself — under five minutes means the layer rule works.
  2. Send a malformed submit request (bad JSON). Record the exact error. Is it helpful? Improve one message.
  3. Move one more client-side decision server-side in your own project. Write the BUG-style comment.
  4. List every 401 path: which endpoints require login, and what breaks if one is mislabeled public?
  5. Ask an AI agent to add a “retake exam” endpoint with the Topic 7 prompt. Check: no SQL in handler? Tests added?

Five things to take with you

  1. Router → service → model, no skips; testable layers are real layers.
  2. The server decides; the client displays — especially rewards.
  3. Validation at the boundary turns crashes into helpful 400s.
  4. UUIDs and JSONB buy specific freedoms at specific prices — name both.
  5. Error messages are UI: name the field, suggest the fix.

Next: Part 4 fills these tables — the pipeline turning syllabi into Priya’s curriculum.

What worked, what I’d change

Worked: the layer rule held across all sixteen routers — review found no SQL in handlers. Changed already (resolved): XP moved server-side per BUG-017. Principle: derive trust from verification, never from the client.