Feedback Loops and Bias in ML Systems: Position and Selection Bias
How ML systems bias their own training data: degenerate feedback loops, position, selection and popularity bias, exploration, debiasing and fairness checks.
Most production models influence the data they're later trained on. A recommender decides what users see, users can only click what they see, and those clicks become tomorrow's training data. This feedback loop creates biases (position bias, selection bias, popularity bias) that make a model look better offline than it is and slowly narrow what it recommends. Senior ML design answers name the loop and say how they break it.
How a degenerate feedback loop forms
Symptoms: recommendations get more homogeneous, a few items take a growing share of impressions, new items never break through, and offline metrics keep improving while online metrics stall.
The biases to name
| Bias | What happens | Example |
|---|---|---|
| Position bias | Higher positions get more clicks regardless of relevance | Search result 1 gets many times the clicks of result 5 |
| Selection (exposure) bias | You only observe labels for items the system chose to show | No data at all on items the model never ranked highly |
| Popularity bias | Popular items get more exposure, so more data, so higher scores | The same blockbuster in every user's feed |
| Presentation bias | Thumbnails, badges and layout change behaviour | A "trending" badge inflates clicks |
| Survivorship bias | Only completed journeys are logged | Fraud labels exist only for transactions that were approved |
Fraud has a nasty version of selection bias: if the model blocks a transaction, you never learn whether it was really fraud. Without some exploration, the model can't learn from its own false positives.
Ways to break the loop
Exploration
Reserve a small share of traffic or slots to show items the model wouldn't have picked: random items, new items, or choices from a bandit. This creates unbiased data and gives new items a chance. Keep it small and safe (policy-compliant content only).
Position-bias correction
- Position as a training feature: include the displayed position in training, and set it to a fixed value (for example position 1) at serving time, so the model learns relevance separately from placement.
- Inverse propensity weighting: weight each click by 1 / P(examined at that position), estimated from randomisation experiments.
- Randomised slots: occasionally swap adjacent results to measure how much position alone changes click rates.
Holdouts and counterfactual logs
- Global holdout: a small group of users who never see the model (or see a simple baseline), so you can measure its true cumulative effect over months.
- Log propensities: store the probability each item had of being shown, enabling offline evaluation of new policies.
- Approve-and-review samples: in fraud, let a tiny random sample of high-score transactions through to a reviewer to measure false positives.
Diversity and freshness in re-ranking
Re-ranking rules that cap items per creator or topic and boost fresh items reduce homogeneity directly; see recommendation systems.
Fairness checks
Feedback loops can also amplify unfair outcomes: if a group of creators or users gets less exposure early, it generates less data and keeps getting less. Practical checks:
- Report metrics by slice (region, language, new vs established creators), not only the overall average.
- Track exposure share over time for groups you care about.
- Audit features for protected attributes and close proxies (postcode can proxy for ethnicity).
- Set guardrail metrics for the slices in experiments, so a launch can't silently harm one group.
Measuring whether the loop is hurting you
Signals to chart weekly
catalogue coverage : % of items with ≥1 impression (falling = narrowing)
Gini of impressions : concentration of exposure (rising = popularity bias)
new-item reach : impressions for items < 7 days old (falling = cold start worse)
holdout lift : model group vs global holdout (the honest effect)
exploration CTR gap : CTR of explored vs ranked items (how much you're missing)Where it’s used
Position-bias correction with click models and randomised swaps is standard for training rankers from click logs.
Exploration slots, diversity rules and long-running holdouts keep feeds from collapsing onto a few items.
Approved-sample reviews and reject inference estimate outcomes for decisions the model blocked.
In an interview
- Point out that the model's outputs shape its future training data.
- Name position and selection bias and one fix for each.
- Propose a small exploration budget and a long-term holdout.
- Add slice metrics and exposure tracking as fairness guardrails.
Frequently asked questions
What is a degenerate feedback loop?
When a model's outputs shape its future training data, for example a recommender that ranks an item first, so it gets more clicks, so the next model ranks it higher, regardless of real quality.
How do you correct position bias?
Include position as a training feature and fix it at serving time, weight clicks by the inverse probability of being examined at that position, or randomise some positions to measure the effect.
Further reading
- Unbiased Learning-to-Rank with Biased FeedbackJoachims et al., WSDM 2017
- Hidden Technical Debt in Machine Learning SystemsSculley et al., NeurIPS 2015