Design a Video Recommendation System Like YouTube: ML System Design

The question: design the recommendation system for a video platform's home page, like YouTube. Users should see videos they'll enjoy watching, new uploads should get a fair chance, and the page must load quickly for hundreds of millions of users. This walkthrough follows the ML system design framework from clarifying questions to monitoring.

1. Clarify requirements

2. Frame it as an ML problem

Clicks alone reward clickbait, so the ML objective is to rank videos by expected satisfied watch time. Concretely: for each (user, video) pair, predict several outcomes and combine them:

score(user, video) =   w1 · p(click)
                     + w2 · E[watch time | click]   (or p(watch > 50%))
                     + w3 · p(like) + w4 · p(share)
                     − w5 · p(not interested)
weights w are tuned by online experiments against long-term engagement

That's a multi-stage ranking problem: retrieval (candidate generation) then ranking, both learned from interaction logs. See ML problem framing.

3. Data

• Rows: one per impression (user saw video at position p, at time t, on device d), joined with outcomes within a window: click, watch seconds, like, share, dismiss.
• Labels: natural labels from behaviour; "not interested" and fast skips are explicit and implicit negatives.
• Volume: 500M users × ~40 impressions/day ≈ 20B impressions/day. Downsample unclicked impressions heavily for training.
• Splits: time-based (train on the last 30 days, evaluate on the next day). See training data.
• Biases: position bias and exposure bias; log position and the serving model version for correction. See feedback loops.

4. High-level architecture

5. Candidate generation

Retrieval must have high recall and be cheap. Combine several sources and merge them:

• Two-tower retrieval: a user tower over watch history, searches and demographics; a video tower over id, title, thumbnail and channel embeddings. Video vectors are precomputed and indexed in an ANN index; per request we embed the user and fetch the top 1,000 by dot product. See embeddings.
• Subscriptions: recent uploads from subscribed channels.
• Item-to-item: videos co-watched with the user's last few watches.
• Trending and fresh: popular in the user's region and an exploration source for new uploads (cold start).

6. Ranking

A light ranker (a small model on cheap features) cuts 2,000 candidates to 500. The heavy ranker is a multi-task deep network with heads for click, watch time, like, share and dismiss, sharing lower layers. See choosing a model and learning to rank.

Position is a training feature set to a constant at serving time, to separate relevance from placement bias.

7. Re-ranking

• Remove policy-violating, borderline and age-restricted videos, and anything already watched or dismissed.
• Diversity: limit videos per channel and per topic in the first screen.
• Freshness: a small boost for recent uploads, and reserved slots for exploration.

8. Serving and scale

Requests     : 500M DAU × ~10 home loads/day = 5B/day ≈ 58K RPS avg, ~150K peak
Retrieval    : ANN over 5B videos is too big; index only the ~500M eligible
               (recent or popular) videos: 500M × 128 dims × 1 B (int8) ≈ 64 GB per replica
Ranking      : 500 candidates × 150K RPS = 75M scores/s → GPU batching, ~2-5K GPU inference
               servers at a few thousand candidate-scores per ms each (illustrative)
Latency plan : features 20 ms | retrieval 30 ms | light 20 ms | heavy 60 ms | re-rank 10 ms
               → ~140 ms, leaving headroom under 200 ms p99
Fallback     : if the heavy ranker times out, serve the light ranker's order

• User embeddings are recomputed in near real time from recent watches, so the feed reacts within a session.
• Precompute recommendations for inactive users in batch and refresh on login, cutting peak load.
• Cache video features aggressively; they change slowly. See model serving.

9. Evaluation

Ship only after an A/B test shows gains in long-term metrics; keep a long-running holdout to measure cumulative effects. See ML evaluation.

10. Monitoring and iteration

• Retrain the ranker daily (incrementally) and the retrieval towers weekly, then rebuild the index; see continual learning.
• Monitor feature null rates, score distributions, catalogue coverage and impression concentration.
• Watch for feedback loops: falling new-video reach or rising concentration of impressions trigger review.

Follow-up questions to expect

• *How do you stop clickbait?* Multi-objective scoring with satisfaction signals and dismiss predictions, plus survey-based labels.
• *How does a brand-new video get its first views?* Content embeddings in the video tower, channel priors and exploration slots.
• *How do you recommend to a logged-out user?* Context and popularity in the region, then session-based signals.
• *How do you know watch time didn't rise only because videos got longer?* Track watch percentage and satisfaction, and compare by duration bucket.