Choosing an ML Model: Baselines, GBDTs, Deep Nets and Multi-Task

How to choose a model in an ML system design interview: heuristic baselines, logistic regression, gradient-boosted trees, deep nets and multi-task models.

Intermediate 3 min read · Updated 29 October 2026

Interviewers don't want the fanciest architecture; they want a model that fits the data size, the latency budget, the need for interpretability, and the team's ability to run it, plus a reason to move to something heavier. The reliable path is: a heuristic baseline, then a simple learned model, then a stronger model only when the data and the gain justify it.

The progression to describe

Each step must beat the previous one on the agreed metric to earn its extra cost. Many production systems stop at gradient-boosted trees.

Model families compared

Family	Strengths	Weaknesses	Typical use
Heuristics	No training, transparent, instant	No personalisation, can't learn	Baseline, cold start, fallback
Logistic / linear regression	Fast to train and serve, calibrated, explainable	Needs manual crosses; limited capacity	First ads and risk models, strong baselines
Gradient-boosted trees (XGBoost, LightGBM)	Excellent on tabular features, little tuning, handles missing values	Poor with huge sparse ids, raw text or images	Fraud, ETA, ranking with dense features
Deep networks (MLP, wide-and-deep, DLRM)	Learns embeddings for ids, interactions, scale with data	Needs lots of data and GPUs, harder to debug	Large-scale ads, feeds, recommendations
Sequence models (transformers, RNNs)	Use order: session history, text, events	Costly at serving time	Next-item prediction, text classification
Two-tower / embedding models	Precomputable item side, fast retrieval	Limited user-item interaction	Candidate generation
Graph neural networks	Use relationships between entities	Complex training and serving	Friend suggestions, fraud rings
Pretrained foundation models	Strong out of the box on text and images	Expensive, slower, may need adaptation	Moderation, search, extraction; see AI engineering

Criteria for choosing

Data size and type: thousands of tabular rows favour trees; billions of sparse interactions favour deep models with embeddings.
Latency and cost: a model scoring 500 candidates in 50 ms on CPU limits depth; see model compression.
Interpretability: credit and some fraud decisions need reasons; trees and linear models are easier to explain.
Freshness: models that train incrementally (linear models, deep nets with online updates) suit fast-changing domains; see continual learning.
Calibration needs: auctions and thresholds need real probabilities; logistic regression is calibrated by construction, others may need recalibration.

Loss functions by task

Task	Loss	Note
Binary classification	Log loss (binary cross-entropy)	Produces probabilities; add class weights or focal loss for imbalance
Multiclass	Softmax cross-entropy	Use sigmoid per class for multilabel
Regression	MSE, MAE, Huber	MAE and Huber resist outliers; quantile loss gives P50 and P90
Ranking	Pairwise (RankNet), listwise (LambdaRank, softmax)	See learning to rank
Retrieval	Sampled softmax, contrastive	With in-batch and hard negatives; see embeddings

Multi-task models

Feeds and ads predict several outcomes at once. A shared bottom layer learns common representations; separate heads predict each objective. Mixture-of-experts variants let tasks share selectively when they conflict (for example clicks vs satisfaction).

A multi-task ranker: shared layers feed one head per objective; the serving layer combines them with weights tuned by experiments.

Ensembles and cascades

Stacking: trees on top of deep-model outputs, or the reverse; common in ads (trees transforming features for a linear model was a well-known early design).
Cascades: a cheap model filters, an expensive one decides; the same idea as the two-stage pattern.
Rules plus models: hard rules for known-bad cases (sanctions, blocked devices), a model for the rest. Standard in fraud.

Training choices to mention

From scratch vs fine-tuning a pretrained model (text and images almost always fine-tune).
Hyperparameter search (random or Bayesian) on the validation set, never the test set.
Regularisation: dropout, weight decay, early stopping on validation loss.
Scale: distributed training for large embedding tables or big models.

Where it’s used

Fraud

Rules for known patterns plus gradient-boosted trees on velocity features, chosen for accuracy on tabular data and explainable reasons.

Large-scale ads

Deep models with huge sparse embedding tables and multi-task heads for click and conversion.

ETA

Gradient-boosted trees as a strong baseline, then deep models that use route sequences and real-time traffic.

In an interview

Start with a heuristic and a simple learned baseline before proposing a deep model.
Justify the model by data type and size, latency, interpretability and calibration.
Name the loss function, and how it handles imbalance or multiple objectives.
Say what result would make you switch to a heavier model.

Frequently asked questions

Which model should I propose in an ML system design interview?

Start with a heuristic baseline and a simple learned model such as logistic regression or gradient-boosted trees, then propose a deep model only if the data size, feature types and expected gain justify the extra latency and complexity.

When are gradient-boosted trees better than deep learning?

On tabular data with dense features and moderate data sizes, such as fraud and ETA prediction. Deep models win with huge sparse id features, text, images and sequences.