ML System Design Interview: A 7-Step Framework With Examples

A machine learning system design interview asks you to design a product feature that depends on a model, such as a video feed, an ad ranker or a fraud detector, and to defend every choice from the business goal down to how the model is monitored. It is not a modelling quiz. Interviewers want to see that you can turn a vague goal into a learnable problem, find the data and labels, pick a sensible model, serve it within a latency budget, prove it works and keep it working. This guide gives you a seven-step framework, a time plan for a 45-minute round, and what separates a pass from a strong pass.

The most common failure is jumping to a model architecture in minute two. Spend the first ten minutes on the problem, the metric and the data; most of the score is earned there.

How ML design rounds differ from backend rounds

You still need ordinary building blocks: the serving path is a low-latency service, features come from data pipelines, and candidates often come from a vector index. The ML round adds the question of decision quality on top.

The seven-step framework

1. Clarify requirements

Ask about the business objective (more bookings? less fraud loss?), the users and scale (daily active users, items in the catalogue, requests per second), latency (a feed must render in about 200 ms; a nightly email can take hours), constraints (on-device, privacy, regulated domain), and what data already exists. Write the answers down; you will refer back to them.

2. Frame the problem as an ML task

Translate the goal into something a model can learn: the input, the output, the label and the task type (binary classification, regression, ranking, retrieval). "Increase watch time" might become "predict the probability a user watches more than half of a video, and rank by expected watch time". Say whether ML is even needed; a rule may do. See ML problem framing.

3. Data and labels

Where do training examples come from, how are they labelled (natural labels like clicks, or human annotation), how much is there, how biased is it, and how will you split it without leaking the future into the past? See training data and data labeling.

4. Features and model

Name feature groups (user, item, context, user-item interaction), how each is computed and whether it's available at prediction time. Then propose a baseline (popularity, logistic regression) before a stronger model (gradient-boosted trees, a deep network, a two-tower retriever). Justify the choice by data size, latency and interpretability. See feature engineering and choosing a model.

5. Evaluation

Pick an offline metric that matches the task (PR-AUC for rare positives, NDCG for ranking, MAE for regression) and an online metric tied to the business goal, plus guardrails that must not get worse. Explain how an A/B test decides the launch. See ML evaluation.

6. Serving

Batch or online prediction? Where do features come from at request time (feature store)? For ranking, sketch the multi-stage funnel. Give a latency budget per stage and a fallback when the model is slow or down. See model serving.

7. Monitoring and iteration

Watch operational metrics (latency, errors), data health (missing features, distribution shift) and model quality (delayed labels, calibration). Decide the retraining cadence and how a new model is promoted. Mention feedback loops if the model shapes its own training data. See model monitoring.

A 45-minute time plan

In a 60-minute round, spend the extra time on the deep dive the interviewer picks. Interviewers often stop you after the overview and say "let's go deeper on features" or "how would you evaluate this"; that's a good sign, not an interruption.

What each level is expected to show

Common mistakes

• Model first. Naming a transformer before agreeing on the label. The label decides everything downstream.
• One metric. Only accuracy, on a problem with 0.1% positives, where predicting "never fraud" scores 99.9%.
• Features that don't exist at serving time. Using "total purchases this week" computed with data from after the prediction moment, a classic data leak.
• No baseline. Without one you can't say whether the complex model is worth its cost.
• Ignoring the serving path. A 200 ms budget can't run a heavy model over a million items; you need candidate generation first.
• Forgetting the loop. No word on drift, retraining or how the model's own outputs bias the next training set.

A worked opening: "Design a restaurant recommender"

Clarify   : goal = more orders from the home screen; 20M DAU; 500K restaurants;
            home screen must render in < 300 ms; we log impressions, clicks, orders.
Frame     : rank restaurants per user; label = ordered within the session after
            an impression (clicks are a secondary, noisier signal).
Data      : impression logs joined to orders; negatives = shown but not ordered;
            split by time (train on weeks 1-6, test on week 7).
Model     : candidates = nearby + open now + two-tower retrieval (~1,000);
            ranker = gradient-boosted trees, then a multi-task deep model.
Evaluate  : offline NDCG@10 and recall@100; online orders per session,
            guardrails = delivery time, cancellation rate, restaurant diversity.
Serve     : precomputed user embeddings, online ANN + ranker, cache per area;
            fallback = popular-nearby list if the ranker times out.
Monitor   : feature null rates, score distribution, daily retrain, weekly review
            of new-restaurant exposure (cold start).

Every line is a decision the interviewer can probe. Full walkthroughs of this shape are in the ML system design examples, and a list of what gets asked is in ML system design interview questions.