Regularization & Model Selection

L1, L2, cross-validation, learning curves, and deciding what to do next from evidence rather than instinct.

Module 03.3 · Intermediate
Free

8h · 7 lessons · 1 challenge

Before this: Classification & Logistic Regression

About this module

"The model underfits, so what do you do?" is the most common diagnostic question in an ML interview, and the answer is a procedure, not a guess. This module teaches that procedure: learning curves, validation strategy, and the specific action each diagnosis implies.

Regularization is derived as a prior, which connects it back to the probability track and forward to weight decay in deep learning.

After this module you can

7 lessons

Lesson 1
Overfitting, underfitting and capacity
Kind
Concept
Length
45 min
Lesson 2
L1, L2 and elastic net, derived as priors
Kind
Concept
Length
55 min
Lesson 3
Cross-validation schemes and their failure modes
Kind
Concept
Length
50 min
Lesson 4
Learning Curve Doctor: prescribe the fix
Kind
Interactive
Length
35 min
Lesson 5
Hyperparameter search: grid, random, Bayesian
Kind
Concept
Length
45 min
Lesson 6
Nested CV and honest model selection
Kind
Lab
Length
70 min
Lesson 7
Checkpoint: what do you do next?
Kind
Checkpoint
Length
25 min

The challenge: Learning Curve Doctor

Training and validation curves, one prescription: more data, more capacity, more regularization, better features, or stop. Fifteen cases, and the game shows you the outcome of your choice.

Worth 400 XP.

Related modules

All modules
Unsupervised Learning
8h · 7 lessons
Recommender Systems
8h · 7 lessons