Brief
Train a 1-layer transformer on (a+b) mod 113 with strong weight decay until it groks, meaning test accuracy jumps to near-perfect long after training accuracy reached 100%. Apply a DFT (discrete Fourier transform) to the embeddings to find the key frequencies: the network represents each number as sines and cosines at a few frequencies. Then compute restricted loss (the loss when you keep only those key frequencies) and excluded loss (the loss when you remove them) over the course of training. The twist: repeat for a second task (multiplication mod p, or a small permutation group) and test whether the universality story holds, that is, whether networks learn the same kind of circuit across related tasks.
VICTORY CONDITIONS
- ≤ 6 key frequencies explain ≥ 95% of logit variance; excluded loss rises before test loss falls; the second task's circuit is documented (grokking reproduced (test acc > 99% long after train acc reaches 100%); variance explained by the key frequencies; restricted/excluded-loss curves).
- A public repository with a README that shows your result and how to reproduce it.
Terms in plain words
- Grokking
- A network first memorises its training data and only much later, often suddenly, learns the rule that generalises.
- Modular arithmetic
- Arithmetic that wraps around, like a clock: (a+b) mod 113 is the remainder after dividing a+b by 113.
- Key frequencies
- The few sine and cosine frequencies the trained network actually uses to represent numbers.
- Restricted / excluded loss
- The loss after keeping only, or removing only, the key-frequency parts of the network's computation; it shows when the circuit forms.
- Circuit
- A specific, human-readable algorithm implemented by a small set of a network's weights and activations.
- Universality
- The hypothesis that different networks trained on similar tasks discover the same kinds of circuits.
Why it rates ★★★☆☆
Mechanistic reverse-engineering (working out the exact algorithm inside a trained network) is the standard entry ticket for interpretability teams, and the second task adds something of your own.