Brief
GPT-2 small has 124 million parameters and has read a lot of the internet, but it doesn't follow commands. This project teaches it one trick of your choice with a few hundred examples. Continuing to train a pretrained model on a small dataset is called fine-tuning. When the examples are commands and answers, it is instruction tuning.
Full fine-tuning updates every weight, which needs memory for all of them plus the optimiser state. LoRA trains far less. It freezes a weight matrix of shape and learns an update made from two thin matrices, () and (), with a small rank :
That is trainable numbers instead of . starts at zero, so at step 0 the model is exactly GPT-2. For GPT-2's attention projection (768 in, 2304 out) at , that's about 25k parameters per layer instead of 1.8M.
The second new topic is evaluation. Loss alone hides a lot, so you'll also define a task score before training and compute it on a test set the model never trained on. Contamination is the classic trap: if test examples or near copies sit in the training data, every score lies. You'll check for that with n-gram overlap. You built the training loop in BL-21 and sped it up in BL-23; here the new work is freezing, wrapping and measuring.
DATA · a small instruction set you build
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- The best LoRA run's eval loss is within 5% of full fine-tuning while training under 1% of the parameters. Record it with `python tools/mlarena.py record BL-24 "<lora> vs <full>, <pct>% params"`.
- Both fine-tuned models beat base GPT-2 on your task score, and the contamination check finds no overlaps between test and train.
- Your LoRA's parameter count matches what the peft library reports for the same layers and rank, and the merged model matches to 1e-4.
Side quests
Terms in plain words
- Fine-tuning
- Continuing to train a pretrained model on a small, specific dataset so it picks up a new behaviour.
- LoRA
- Low-rank adaptation. Freeze the original weights and train two thin matrices whose product is added to each chosen layer.
- Rank
- The inner size r of the two LoRA matrices. It caps how much the update can change, and sets how many parameters you train.
- Loss masking
- Computing the loss only on the answer tokens, so the model is not trained to repeat the prompt.
- Contamination
- Test examples, or near copies of them, leaking into the training data. It makes scores look better than they are.