← All Base Form ML projects
BL-24NOT STARTEDBASE FORM · TRANSFORMERSMLPYTHON

Teach GPT-2 a trick

Fine-tune GPT-2 with LoRA so it follows your commands or writes in a style.

EST.8 h
REWARD+100 XP
AFTERHu et al.

ML-ARENA BENCHMARKS

YOUR NUMBER VS THE INDUSTRY'S
METRICINDUSTRYMINE
eval loss; trainable params
GOAL · within 5% of full FT with < 1% of params
full fine-tune · measured—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

GPT-2 small has 124 million parameters and has read a lot of the internet, but it doesn't follow commands. This project teaches it one trick of your choice with a few hundred examples. Continuing to train a pretrained model on a small dataset is called fine-tuning. When the examples are commands and answers, it is instruction tuning.

Full fine-tuning updates every weight, which needs memory for all of them plus the optimiser state. LoRA trains far less. It freezes a weight matrix WW of shape d×kd \times k and learns an update made from two thin matrices, BB (d×rd \times r) and AA (r×kr \times k), with a small rank rr:

h=Wx+αrBAxh = Wx + \frac{\alpha}{r} BAx

That is r(d+k)r(d + k) trainable numbers instead of dkdk. BB starts at zero, so at step 0 the model is exactly GPT-2. For GPT-2's attention projection (768 in, 2304 out) at r=8r = 8, that's about 25k parameters per layer instead of 1.8M.

The second new topic is evaluation. Loss alone hides a lot, so you'll also define a task score before training and compute it on a test set the model never trained on. Contamination is the classic trap: if test examples or near copies sit in the training data, every score lies. You'll check for that with n-gram overlap. You built the training loop in BL-21 and sped it up in BL-23; here the new work is freezing, wrapping and measuring.

VISUALFig. 1
x W d × k, frozen A r × k r B d × r × α/r + h frozen trained B starts at 0, so step 0 is exactly GPT-2 trained: r(d + k) full update: d · k
LoRA keeps W frozen and adds a low-rank path: A squeezes x down to r numbers, B expands them back. Why it's here: Objective 4 is building this wrapper; the shapes are the part that is easy to get wrong.
HINT · OBJECTIVE 4
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 1
Stuck? Hints stay hidden until you ask.

DATA · a small instruction set you build

Objectives

0 / 6 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. The best LoRA run's eval loss is within 5% of full fine-tuning while training under 1% of the parameters. Record it with `python tools/mlarena.py record BL-24 "<lora> vs <full>, <pct>% params"`.
  2. Both fine-tuned models beat base GPT-2 on your task score, and the contamination check finds no overlaps between test and train.
  3. Your LoRA's parameter count matches what the peft library reports for the same layers and rank, and the merged model matches to 1e-4.

Side quests

Fine-tune for a writing style instead of a command, and design a score for it that isn't just loss.
Train with DPO on pairs of good and bad answers you write, starting from your LoRA model.

Terms in plain words

Fine-tuning
Continuing to train a pretrained model on a small, specific dataset so it picks up a new behaviour.
LoRA
Low-rank adaptation. Freeze the original weights and train two thin matrices whose product is added to each chosen layer.
Rank
The inner size r of the two LoRA matrices. It caps how much the update can change, and sets how many parameters you train.
Loss masking
Computing the loss only on the answer tokens, so the model is not trained to repeat the prompt.
Contamination
Test examples, or near copies of them, leaking into the training data. It makes scores look better than they are.