Trees, Ensembles & Gradient Boosting

Decision trees to XGBoost and LightGBM — still the strongest baseline on tabular data, and interviewers know it.

Module 03.4 · Intermediate
Free

10h · 8 lessons · 1 challenge

Before this: Regularization & Model Selection

About this module

If the data is tabular, gradient boosting is probably the answer, and being able to say why — and to tune it — is a real differentiator against candidates who reach for a neural network reflexively.

The module goes from a single tree through bagging and random forests to the boosting objective, then into practical tuning and interpretation with SHAP.

After this module you can

8 lessons

Lesson 1
Decision trees, impurity and splits
Kind
Concept
Length
50 min
Lesson 2
Bagging, random forests and variance reduction
Kind
Concept
Length
45 min
Lesson 3
Boosting: the objective and the update
Kind
Concept
Length
60 min
Lesson 4
XGBoost and LightGBM: what the implementations add
Kind
Concept
Length
45 min
Lesson 5
Tune a booster against a deep model on tabular data
Kind
Lab
Length
100 min
Lesson 6
Feature importance, SHAP and honest interpretation
Kind
Concept
Length
50 min
Lesson 7
Split Master: choose the split, grow the tree
Kind
Interactive
Length
30 min
Lesson 8
Checkpoint: tabular model selection
Kind
Checkpoint
Length
25 min

The challenge: Split Master

Grow a tree one split at a time against an opponent tree grown by the optimal criterion. See how far greedy gets you.

Worth 450 XP.

Related modules

All modules
Unsupervised Learning
8h · 7 lessons
Recommender Systems
8h · 7 lessons
Neural Networks from Scratch
12h · 8 lessons