The memory hierarchy, arithmetic intensity, fused kernels and writing your own in Triton.
10h · 8 lessons · 1 challenge
Before this: Training Infrastructure & Distributed Training
Almost every model is memory-bandwidth bound, and understanding why is what separates engineers who can make a model faster from engineers who file a ticket.
You profile a real workload, find the bound, and write a fused kernel in Triton. Flash Attention is studied as the worked example of exactly this reasoning.
Profiles from eight workloads. Name the bound — bandwidth, compute, launch overhead, synchronization — and pick the fix that buys the most.
Worth 650 XP.
