Brief
Write an Othello engine to generate millions of random legal games, and train an 8-layer GPT on the move sequences. Then fit linear probes (small linear classifiers trained on the model's internal activations) that predict 'mine / theirs / empty' for each square at each layer, where 'mine' and 'theirs' are relative to the player about to move. Run causal interventions: edit the residual stream along the probe directions so that the model's internal board changes, and check whether its legal-move predictions follow the edited board.
Approach
The model only ever sees a list of move tokens. If it predicts legal moves well, it has to track the board somehow, and the question is whether that board is stored linearly. Li et al. found that black/white labels needed a non-linear probe; Nanda showed that relabelling squares as mine/theirs relative to the player to move makes a linear probe work. Your probe for layer is just
for each square , trained with cross-entropy on cached activations . Train all 64 squares as one linear layer.
An intervention uses the probe in reverse. To make square read as "mine", add to the residual stream at one layer and position, then let the rest of the forward pass run. Then compare the model's predicted legal moves with the legal moves of the board you meant to create. Define the predicted set before you start, for example "the top moves, where is the number of legal moves on the edited board", and keep it fixed.
Order: engine and its tests, then data, then training, then probes, then interventions. The model has about 25M parameters. A few million random games of about 60 moves each is a few hundred million tokens per epoch, roughly FLOPs, so one epoch takes well under an hour on a single consumer GPU. Budget a few GPU-hours for training and sweeps. Game generation runs on the CPU; parallelise it over cores before you start, because pure Python is the slow part.
Where people get stuck
- Passes break mine/theirs. When a player has no legal move, the turn passes and "mine" swaps without a move token. Track passes in the labels, or drop games that contain them and say so.
- Probing the wrong position. Decide whether the probe at position predicts the board after move or before it, and keep that convention in the intervention code.
- Training the probe on training games. Fit and score probes on held-out games, never on games the GPT trained on.
- One α for everything. The right step size differs by layer and square. Sweep it, and report success as a function of α rather than one lucky value.
- A loose success metric. A threshold that marks half the board as "predicted legal" will match almost any edit. Run the random-direction control.
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- Linear-probe error at most 1.7% at the best layer, and at least 80% of 500 single-square edits produce the correct legal-move set. Record it with `python tools/mlarena.py record PL-A03 "<probe err>, <edit success>"`.
- The random-direction control succeeds far less often than the probe-direction edits.
- Legal-move top-1 rate of at least 99% on held-out random games.
Side quests
Terms in plain words
- World model
- An internal representation of the state of the world, here the board, that the network builds even though it was never shown it.
- Linear probe
- A linear classifier trained on a layer's activations; if it works, the information is stored in a simple, readable way.
- Mine / theirs encoding
- Labelling squares by whose piece it is relative to the player to move, rather than black / white; this is what makes a linear probe work.
- Causal intervention
- Changing the model's internal activations on purpose and checking that its behaviour changes the way your theory predicts.
- Residual stream
- The per-token vector that runs through the transformer and that every layer reads from and writes to.
Why it rates ★★★☆☆
A full world-model probe-and-intervene pipeline is exactly what MI (mechanistic interpretability) teams ask about. It is still a reproduction.