Repo structure and the spec-driven method

Follow one learner from signup to study plan while learning how specs, ADRs, and a Makefile keep a year-long build coherent.

• python
• fastapi
• architecture
• process
• beginners

Meet your learner

Priya has six months until her Databricks exam, one hour a day, and a shelf of PDFs she never opens. She signs up, answers onboarding questions (role, experience, exam date, daily minutes), and gets a study plan spread across the weeks to her exam date. Everything in this series exists to serve that journey — and everything in this part exists so a project this big survives long enough to serve her.

You are here: Part 1 of 6 — the map room. The hub told Priya’s story and this project’s scars; this part answers how a build this size survives its own year. Parts 2–6 build the study brain, the API, the content pipeline, the knowledge systems, then prove it all running. Each part ends with a handoff telling you exactly where to go next.

Why long projects die (and the fix)

Side projects rarely die from hard problems. They die around week six from drift: the code no longer matches the idea in your head, quality erodes silently, restarting feels easier than continuing. First principle: drift is a documentation failure, not a discipline failure. This repo fights it with three maps, each answering one question:

specs/  (SPEC-001…009)  → WHAT are we building?
docs/architecture/ (ADR-001…009) → WHY this way, and what did we reject?
docs/requirements/ (REQ-001…005) → WHAT must hold, and what ships first?
docs/progress/PROGRESS.md → WHERE are we right now?

A map nobody reads is decoration. The rule that makes these work: the progress tracker is updated weekly, and a CI job validates the specs — maps that check themselves.

How to read an ADR (and write one)

An ADR (Architecture Decision Record) is a one-page memory of a consequential choice. Every ADR in this repo follows one shape — copy it for every consequential decision you ever make:

# ADR-00X: <decision>
- Status: accepted | rejected | superseded (date)
- Context: what forced the choice
- Options: A, B, C with one-line costs each
- Decision: the pick
- Consequences: what gets harder because of it

The last line is the one most teams skip, and the most valuable: it tells your future self what pain you knowingly bought. When Priya’s app later strains somewhere, the ADR says whether that strain was foreseen — which changes the fix completely.

Your turn: pick one decision in your own project (database, framework, anything) and write it in this shape. If you cannot name the rejected options, you have found a decision made by default — the most dangerous kind.

The repository as Priya’s journey needs it

backend/app/
├── api/v1/        # 16 route groups — every screen Priya sees calls one
├── services/      # business logic: XP, badges, analytics, recommendations
├── models/        # tables: her answers, her progress, her reviews
├── schemas/       # shapes that validate her input and serialize responses
├── algorithms/    # 6 pure modules: the math that coaches her (Part 2)
├── pipeline/      # 7 stages turning syllabi into her curriculum (Part 4)
├── knowledge/     # search, graph, agent tools (Part 5)
└── core/          # config, database, login sessions, rate limits

Mental model for newcomers: dependencies point inward. Screens call services, services call math and tables, math imports nothing. When Priya’s dashboard number looks wrong, that arrow tells you exactly where to look — and in Part 2 you’ll trace one yourself.

Three directories deserve one extra sentence each. core/ holds the unglamorous machinery every request touches: configuration, the database connection, password sessions, and the rate limiter that throttles login guessing. schemas/ holds Pydantic shapes — the bouncers that reject malformed input at the door with a 400 before it can reach calculations. models/ holds the SQLAlchemy tables — the permanent memory; fourteen Alembic migrations form one linear chain, the schema’s replayable memory from empty database to today.

The five-command start

make dev-up      # postgres + redis + minio (her data's homes)
make install     # dependencies
make migrate     # build the schema from 14 migrations
make seed        # certifications + a demo learner like Priya
make backend-dev # hot-reload server

Why a Makefile and not docs: commands that execute can’t go stale the way wiki pages do. Priya’s onboarding flow runs against this exact setup. Your turn: clone any project you own and count the commands from zero to running. If it’s more than five or any live only in someone’s head, you found your first ADR.

Checkpoint for this part: after make seed, the demo learner exists and the server answers. If any command fails, stop — every later part assumes this ground.

The gate that protects Priya

Ten CI jobs — lint, strict types, pytest with an 80% coverage floor, integration against real services, contract tests, security audit, spec validation, frontend build, Docker smoke. Measured: 86/86 unit tests passing, engine modules at 93–100% each. What this means for her: the mastery math deciding her study plan is pinned by tests that run in 4 seconds, on every change, before merge.

Read that list again as promises to a learner: nobody can merge untested math that schedules her mornings. Process is care, expressed as automation.

Directing an AI agent in this repo

If you hand this repo to a coding agent, do not say “add a feature.” Say:

You are working in data-eng-mastery, a FastAPI + Postgres study app.

Goal: <one sentence>
Constraints:
- Router → service → model. No SQL in route handlers.
- Study math lives in backend/app/algorithms/ as pure functions.
- New behavior needs unit tests; run make test-unit before finishing.
- Check docs/architecture/ADRs for any decision this touches.

Before coding:
1. Show me the files you will touch and why.
2. State which layer each change belongs to.
3. Then implement, then run the tests.

The maps from Topic 2 are what make this prompt work — the agent reads the same ADRs you would. Repos without maps get confident, wrong agents.

When the start breaks (troubleshooting)

Usual suspects, in order — check these before anything clever:

1. Docker isn’t running. make dev-up fails with connection errors. Start Docker Desktop, wait, retry.
2. A port is taken. Postgres, Redis, or the API port answers something unexpected. Stop the other occupant or change the mapping.
3. Environment missing. Copy .env.example to .env — the app refuses to boot without secrets configured, which is correct behavior, not a bug.
4. Migration mismatch. make migrate complains about heads. Never edit an applied migration; add a new one (make migration).
5. Seed twice. Re-running seed duplicates demo rows. Reset first (make reset-db), then seed — teardown is part of the test, not decoration.

Exercises

1. Draw the dependency arrows for one screen (dashboard): screen → service → math/tables. Which arrow would break if you deleted schemas/?
2. Write one ADR for your own project in the Topic 3 shape. Show it to someone; if they can’t state what gets harder, rewrite the consequences.
3. Count zero-to-running commands in a project you own. More than five? Draft the Makefile that fixes it.
4. Ask an AI agent to explain what core/ does — then verify each claim against the files. Keep score of its hits and misses.
5. Read one ADR-00X and one SPEC in this repo. Do they agree with the code today? Evidence beats memory.

Five things to take with you

1. Drift kills projects; documentation defeats drift — but only if read weekly.
2. Dependencies point inward: screens → services → math/tables, math imports nothing.
3. Executable commands beat wiki pages; five commands from zero to running.
4. Gates (CI, coverage floors, spec validation) are care expressed as automation.
5. Maps make agents reliable: an agent that reads ADRs behaves like a teammate.

Next: Part 2 opens the coaching brain — six algorithms you’ll compute by hand.

What worked, what I’d change

Worked: the maps survived a year — specs still describe the code. I’d change: the tracker records stalled phases faithfully but nothing restarts them; process reveals, humans resume. Principle: write the map, read it weekly.