Bias-Variance Tradeoff — ML Technical Interview Questions & Answers

Balancing model simplicity against flexibility.

Bias-Variance Tradeoff

  • Simple meaning: Bias = the model is too stubborn (a straight line when the truth is a curve) → underfit. Variance = the model jumps around with every new sample of students → overfit. You want a middle.
  • Picture: Always predict 50 marks — high bias. A wild polynomial through every point — high variance. A gentle line or a shallow tree — usually better.
  • Error ≈ bias² + variance + noise you cannot remove. More complexity: bias down, variance up. Regularisation / more data: variance down.
  • Trap: saying “high bias means overfit” (it is the opposite). Trap: only talking formulas with no picture.

Technical Interview Tip: Stubborn model = bias. Jumpy model = variance. Aim for the middle.

Code Examples for Bias-Variance Tradeoff

Bias-Variance Tradeoff — working example

# Toy train vs test errors (higher = worse)
models = [
    {"name": "underfit", "train": 0.30, "test": 0.32},
    {"name": "good", "train": 0.08, "test": 0.10},
    {"name": "overfit", "train": 0.01, "test": 0.28},
]
for m in models:
    gap = m["test"] - m["train"]
    print(m["name"], "gap=", round(gap, 2))

# L2 idea: penalty += lambda * w**2  (discourages huge weights)

Bias-Variance Tradeoff — overfit = low train / high test gap; underfit = high error everywhere; regularization limits capacity.

How to Explain Bias-Variance Tradeoff to an Interviewer

High bias means?

Underfit Too simple

High variance means?

Overfit Too sensitive to the training sheet