SFT, reward models, RLHF, DPO and the constitutional methods — how a base model becomes an assistant.
10h · 9 lessons · 1 challenge
Before this: Reinforcement Learning Foundations, LLM Pretraining & Scaling Laws
A base model completes text; an assistant follows instructions and declines some of them. The gap is closed by supervised fine-tuning, a reward model, and a policy optimization step — and this module walks the whole pipeline with the RL foundations already in place.
DPO and its relatives get equal weight, since they are what most teams now reach for, and the tradeoff against PPO is a live interview question.
You are the policy. Maximize a reward model without doing the task it was meant to reward. Then patch the reward model against your own exploit.
Worth 650 XP.
