Full fine-tuning, LoRA, QLoRA and adapters — adapting a pretrained model on a budget you actually have.
10h · 8 lessons · 1 challenge
Before this: Attention & the Transformer
Almost nobody pretrains, and almost everybody adapts. This module covers the adaptation toolkit and, more importantly, the decision: full fine-tune, LoRA, prompt, or retrieve.
LoRA is derived from the low-rank material in the linear algebra module, which is a satisfying place for that thread to come back.
A model, a task, one 24GB card. Pick the method, the rank and the precision. The game runs your configuration and reports whether it fit and what it scored.
Worth 600 XP.
