Brief
You trained a GPT in BL-21 and know it works. This project asks how. Interpretability is the study of what is going on inside a trained network, and you'll use three of its basic tools on your own model, then check your tools against the standard library on GPT-2 small.
The logit lens is the simplest. Every block adds its output to the residual stream, and at the end the model applies a final LayerNorm and the unembedding matrix to get logits. Nothing stops you doing that early. For the residual after layer :
Watching the correct token climb the ranking layer by layer shows where the prediction forms.
A linear probe asks a different question: is some property, like "this character is inside quotation marks", written in the activations in a way a straight line can read? You train logistic regression on one layer's vectors. A good score alone proves little, since a probe can learn from almost anything, so you always compare it with a control baseline. PCA from BL-07 gives you a picture of the same thing.
Circuits are the third tool. An induction head is a well-studied one: on a sequence like A B … A, it attends from the second A to the B that followed the first A, and predicts B. It is how transformers copy patterns from their context. You'll find these heads in your model with a simple score, then confirm your code finds the same heads in GPT-2 small as TransformerLens. BL-30 goes further and changes what the model does.
DATA · your BL-21 model + GPT-2 small
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- Your own induction scores on GPT-2 small pick out the same top heads as the TransformerLens version. Record it with `python tools/mlarena.py record BL-29 "top heads match, probe <acc>%"`.
- At its best layer, your probe beats both the shuffled-label and random-model baselines by at least 10 points of accuracy.
- The logit lens chart covers every layer, and its last layer matches the model's real predictions exactly.
Side quests
Terms in plain words
- Logit lens
- Applying the model's final LayerNorm and output layer to the residual stream after an earlier layer, to see what it would predict at that depth.
- Linear probe
- A logistic regression trained on a layer's activations to predict some property. If it works, the property is readable from that layer.
- Control baseline
- The same probe on shuffled labels or a random untrained model. A probe only means something if it clearly beats these.
- Induction head
- An attention head that finds the previous occurrence of the current token and copies the token that came after it.
- Circuit
- A small set of heads and neurons that together carry out one behaviour you can name and test.