Initialization, normalization, dropout, weight decay and the vanishing gradient -- everything between a network that exists and one that converges.
10h · 8 lessons · 1 challenge
Before this: Neural Networks from Scratch
A deep network that will not train is the standard experience, and the fixes are not folklore — they follow from what happens to gradient and activation statistics as depth increases.
This module derives why Xavier and He initialization have the constants they do, what batch and layer normalization each stabilize, and how to tell from a training run which one you need.
Loss curves, gradient norms and activation histograms from a failing run. One fix per case, twelve cases, no re-runs.
Worth 550 XP.
