← All Base Form ML projects
BL-29NOT STARTEDBASE FORM · INTERPRETABILITYMLPYTHON

X-ray your GPT

Look inside your tiny GPT: decode every layer with the logit lens, probe what it knows, and find its induction heads.

EST.10 h
REWARD+100 XP
AFTERnostalgebraist

ML-ARENA BENCHMARKS

YOUR NUMBER VS THE INDUSTRY'S
METRICINDUSTRYMINE
probe accuracy; induction scores
GOAL · same induction heads found on GPT-2 small
TransformerLens on GPT-2 small—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

You trained a GPT in BL-21 and know it works. This project asks how. Interpretability is the study of what is going on inside a trained network, and you'll use three of its basic tools on your own model, then check your tools against the standard library on GPT-2 small.

The logit lens is the simplest. Every block adds its output to the residual stream, and at the end the model applies a final LayerNorm and the unembedding matrix WUW_U to get logits. Nothing stops you doing that early. For the residual hℓh_\ell after layer ℓ\ell:

logitsℓ=LNf(hℓ) WU\text{logits}_\ell = \text{LN}_f(h_\ell)\, W_U

Watching the correct token climb the ranking layer by layer shows where the prediction forms.

A linear probe asks a different question: is some property, like "this character is inside quotation marks", written in the activations in a way a straight line can read? You train logistic regression on one layer's vectors. A good score alone proves little, since a probe can learn from almost anything, so you always compare it with a control baseline. PCA from BL-07 gives you a picture of the same thing.

Circuits are the third tool. An induction head is a well-studied one: on a sequence like A B … A, it attends from the second A to the B that followed the first A, and predicts B. It is how transformers copy patterns from their context. You'll find these heads in your model with a simple score, then confirm your code finds the same heads in GPT-2 small as TransformerLens. BL-30 goes further and changes what the model does.

EXPECTEDFig. 1
KEYS (ATTENDED TO) QUERIES A B C D E F A B C D E F A B C D E F A B C D E F 1ST COPY 2ND COPY induction second B looks at the C after the first B same token the earlier copy itself masked future
An induction head's attention on six random tokens repeated twice: a stripe one step right of the 'same token' diagonal. Why it's here: This is what a high induction score looks like; use it to check your heatmaps in objectives 4 and 5.
HINT · OBJECTIVE 4
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 5
Stuck? Hints stay hidden until you ask.

DATA · your BL-21 model + GPT-2 small

Objectives

0 / 5 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. Your own induction scores on GPT-2 small pick out the same top heads as the TransformerLens version. Record it with `python tools/mlarena.py record BL-29 "top heads match, probe <acc>%"`.
  2. At its best layer, your probe beats both the shuffled-label and random-model baselines by at least 10 points of accuracy.
  3. The logit lens chart covers every layer, and its last layer matches the model's real predictions exactly.

Side quests

Train a tuned lens, a small learned linear map per layer, and compare it with the plain logit lens.
Save checkpoints during a fresh BL-21 run and find the step where induction heads appear. Look for a bump in the loss curve at the same time.

Terms in plain words

Logit lens
Applying the model's final LayerNorm and output layer to the residual stream after an earlier layer, to see what it would predict at that depth.
Linear probe
A logistic regression trained on a layer's activations to predict some property. If it works, the property is readable from that layer.
Control baseline
The same probe on shuffled labels or a random untrained model. A probe only means something if it clearly beats these.
Induction head
An attention head that finds the previous occurrence of the current token and copies the token that came after it.
Circuit
A small set of heads and neurons that together carry out one behaviour you can name and test.