Self-attention, multi-head attention and a complete transformer implemented from the paper, with every shape written out.
14h · 10 lessons · 1 challenge
Before this: Tokenization & Embeddings
The center of the curriculum. You implement a transformer end to end — scaled dot-product attention, multiple heads, positional encoding, the residual and normalization structure, the feedforward block — and train it on a real task.
Every tensor shape is written on the board at every step. Being able to reproduce that diagram from memory is close to a guarantee of passing a transformer depth interview.
An unlabeled transformer block. Annotate every tensor shape before the timer runs out. The interview version of this question, made into a drill.
Worth 700 XP.
