Recommendation System Design: Candidate Generation and Ranking

A recommendation system picks the few items (videos, products, posts, songs) most likely to be useful to a user out of millions. At scale it's always a funnel: fast candidate generation narrows millions of items to a few thousand, a ranking model scores those candidates carefully, and a re-ranking step applies business rules such as diversity, freshness and safety. Each stage trades accuracy for speed differently.

The multi-stage funnel

Candidate generation

Retrieval must be fast and have high recall: it should rarely miss an item the user would like, even if it lets through many they wouldn't. Systems combine several sources and merge their results:

• Two-tower (dual encoder) models: one network embeds the user and context, another embeds items, trained so that a user's embedding is close to items they engaged with. Item embeddings are precomputed and indexed in a vector database; at request time you embed the user and run an approximate nearest-neighbour search.
• Collaborative filtering: "people who watched X also watched Y", from co-occurrence or matrix factorisation.
• Content-based: items similar in text, category or creator to what the user engaged with.
• Heuristic sources: trending, new from followed creators, popular in the user's region. These also help cold start for new users and items.

Ranking

The ranker scores each candidate with a richer model (gradient-boosted trees or a deep network) using many features: user history, item statistics, context like time and device, and user-item interaction features. It usually predicts several probabilities, such as click, watch time, like and hide, and combines them into one score with weights that reflect product goals.

• Features come from a feature store, with user features fetched once and item features batched.
• A heavy ranker may only fit a few hundred candidates within the latency budget, so a light pre-ranker sometimes sits between retrieval and ranking.
• Optimising only clicks rewards clickbait; combine engagement with satisfaction signals and long-term outcomes.

Re-ranking and policy

• Diversity: avoid ten items from the same creator or topic in a row.
• Freshness: boost new items so they get a chance to collect feedback (exploration).
• Business rules: remove out-of-stock products, respect age ratings, enforce sponsored-slot limits.
• Safety and integrity: filter policy-violating content and downrank low-quality items.
• Already seen: remove items the user has already viewed or dismissed.

Data, training and feedback loops

Training data comes from logs of what was shown and what happened next. That creates biases a strong answer addresses:

• Position bias: items shown first get clicked more regardless of quality. Log positions and correct for them.
• Feedback loops: the model only learns about items it already shows, so popular items get more popular. Reserve some traffic for exploration.
• Delayed and implicit labels: a purchase may come days after a click; skipping a song is a negative signal only sometimes.
• Freshness: user interests shift quickly, so retrain often and use recent behaviour as real-time features.

Evaluate offline with ranking metrics such as recall@k for retrieval and NDCG for ranking, then confirm with an A/B test on engagement and guardrail metrics. See ML evaluation.

Serving at scale

• Precompute candidates for inactive users in batch; compute fresh ones online for active sessions.
• Cache popular item embeddings and features; batch model inference on GPUs or optimised CPU runtimes.
• Set a latency budget per stage (for example 20 ms retrieval, 50 ms ranking) and degrade gracefully, such as serving trending items if ranking times out.