Gradient Boosting (XGBoost/LightGBM) — ML Technical Interview Questions & Answers

Sequentially building trees that fix previous errors.

Gradient Boosting (XGBoost/LightGBM)

  • Simple meaning: Boosting builds trees one after another. Each new tree tries to fix the mistakes of the current team. XGBoost / LightGBM are fast versions of this idea.
  • Picture: First tree is a rough pass/fail. Second tree focuses on the students you got wrong. After many rounds, train error is tiny — stop early or you overfit.
  • Usually stronger than one forest on tables, but more knobs (learning rate, depth, rounds). Need a validation set or CV. Slower to tune than logistic.
  • Trap: running 1000 rounds because train accuracy is 100%. Trap: saying boosting is the same as random forest (forest = parallel bagging; boosting = sequential).

Technical Interview Tip: Trees in a queue, each fixing the last. Don’t train until 100% train.

Code Examples for Gradient Boosting (XGBoost/LightGBM)

Gradient Boosting (XGBoost/LightGBM) — working example

# ROC idea: vary threshold on scores
y_true = [0, 0, 1, 1]
scores = [0.1, 0.4, 0.35, 0.8]

def tpr_fpr(threshold):
    pred = [1 if s >= threshold else 0 for s in scores]
    tp = sum(t == 1 and p == 1 for t, p in zip(y_true, pred))
    fp = sum(t == 0 and p == 1 for t, p in zip(y_true, pred))
    fn = sum(t == 1 and p == 0 for t, p in zip(y_true, pred))
    tn = sum(t == 0 and p == 0 for t, p in zip(y_true, pred))
    tpr = tp / (tp + fn)
    fpr = fp / (fp + tn)
    return tpr, fpr

for thr in (0.3, 0.5, 0.75):
    print(thr, tpr_fpr(thr))
# Boosting: train trees sequentially on residuals / hard examples

Gradient Boosting (XGBoost/LightGBM) — rOC plots TPR vs FPR across thresholds; AUC summarizes ranking quality. Boosting builds strong learners from weak ones.

How to Explain Gradient Boosting (XGBoost/LightGBM) to an Interviewer

Boosting vs random forest?

Sequential vs parallel trees Fix errors vs vote

Name two libraries.

XGBoost LightGBM sklearn GradientBoosting