Recommender system — collaborative filtering at million-row scale

Resume bullet (when finished)

“Designed and deployed a hybrid (collaborative-filtering + content-based) recommender for a 1M-user × 100K-item dataset; served via FastAPI with vector lookups in <30 ms p95, evaluated offline against MAP@10 + nDCG, A/B-tested online via shadow deployment.”

Locked tech stack

No "choose your language" — analysis paralysis kills completion. Follow the stack to the letter on your first build.

Milestones (8 · ~62h)

1. M1~6h

   Dataset + baseline (popularity recommender)

   Load MovieLens 1M or similar. Build a 'recommend the most popular K items the user hasn't seen' baseline. Measure MAP@10.

   CHECK BEFORE MOVING ON:

   • Why start with a popularity baseline before fancy ML?
   • What does MAP@10 capture that simple accuracy doesn't?
   $ git commit -m "feat: MovieLens loader + popularity baseline + MAP@10 eval"
2. M2~10h

   Matrix factorization with PyTorch

   Implicit-feedback ALS or BPR loss. Train embeddings for users + items, dim=64. Beat popularity by ≥20% MAP@10.

   CHECK BEFORE MOVING ON:

   • Why implicit feedback for this dataset (ratings or clicks)?
   • What's the 'cold-start' problem and how does MF handle it?
   $ git commit -m "feat(ml): MF embeddings via PyTorch"
3. M3~8h

   Content-based hybrid component

   Tag/genre embeddings for items. Compute weighted mix of CF + content scores per item, weight tuned on val set.

   CHECK BEFORE MOVING ON:

   • When does CF beat content-based, and vice-versa?
   • Why hybrid for cold-start items?
   $ git commit -m "feat(ml): content-based blend"
4. M4~8h

   FAISS ANN index — sub-30ms top-K lookup

   Build a FAISS index on item embeddings. Per-user top-K served by FAISS search. Measure p95 latency.

   CHECK BEFORE MOVING ON:

   • Why ANN instead of exact k-NN at million-item scale?
   • What's the trade-off — recall@K vs latency vs index size?
   $ git commit -m "feat(perf): FAISS ANN index for top-K serving"
5. M5~8h

   FastAPI serving layer + Redis feature store

   GET /recommend?user_id=<id>&k=10 → 10 item ids. User embedding pulled from Redis, FAISS search, response under 30ms p95.

   CHECK BEFORE MOVING ON:

   • Why Redis between FastAPI and the model?
   • What goes wrong if you load embeddings from disk per request?
   $ git commit -m "feat(serving): FastAPI + Redis feature store"
6. M6~6h

   MLflow experiment tracking + model registry

   Every train run logged in MLflow with hyperparams + metrics + artifact. /recommend reads which model id is current.

   CHECK BEFORE MOVING ON:

   • Why register models with versions instead of pickling?
   • How would you roll back a bad release?
   $ git commit -m "feat(mlops): MLflow tracking + registry"
7. M7~8h

   Offline eval suite + drift dashboard

   Nightly batch computes MAP@10 + nDCG on held-out data. Grafana panel for drift (mean recommendation entropy week-over-week).

   CHECK BEFORE MOVING ON:

   • What's drift in recommenders and why does it matter?
   • Why nDCG in addition to MAP@10?
   $ git commit -m "feat(eval): nightly metric job + drift dashboard"
8. M8~8h

   Shadow deployment + A/B test plan

   New model serves shadow traffic for 24h. Statistical test outline (sample size, MDE, primary metric, guardrails) in docs.

   CHECK BEFORE MOVING ON:

   • Why shadow before rolling out?
   • What's a minimum-detectable-effect calc and why do PMs care?
   $ git commit -m "ops: shadow deploy + A/B test design doc"

60-second demo storyboard

What you say in the recruiter screen when they ask "tell me about your latest project." Practice it out loud.

1. 0-5s: 'I built an end-to-end recommender — model, serving, eval, the works.'
2. 5-20s: Show the architecture diagram (offline train → MLflow → registry → FAISS index → FastAPI → Redis).
3. 20-40s: Live latency demo — POST 1000 reqs, show p95 < 30ms in the Grafana panel.
4. 40-55s: Walk one MLops decision (MLflow registry vs git tags, e.g.) and the shadow-deploy plan.
5. 55-60s: 'Repo + writeup. Would love to talk about how you'd extend to multi-objective ranking.'

STAR talking points for behavioral round

Production references — how grown-up systems do this

Self-review rubric (before you claim done)