Data pipelines, curricula, stability tricks and the failure modes of long training runs.
10h · 8 lessons · 1 challenge
Before this: Attention & the Transformer
Training a transformer for a week is a different discipline from training one for an hour. Loss spikes, dead runs, corrupted checkpoints and data ordering effects all become real problems with real remedies.
This module is the practical craft: data mixing and deduplication, curriculum, checkpointing, and reading the telemetry of a run you cannot afford to restart.
A month-long run, compressed. Loss spikes arrive; roll back, skip the batch, lower the rate, or hold your nerve. Reach the end.
Worth 600 XP.
