← All Cracked ML projects
PL-I11NOT STARTEDINTERMEDIATEMLPYTHONIMPACT ★★★☆☆

Grokking, reverse-engineered: the Fourier circuit and beyond

Watch a transformer suddenly generalise long after it has memorised the training set, then prove it does modular addition with trig identities.

EST.18 h
REWARD+300 XP
AFTERNanda et al. + 1

ML-ARENA BENCHMARKS

PROOF: FINDING
METRICINDUSTRYMINE
grokking reproduced (test acc > 99% long after train acc reaches 100%); variance explained by the key frequencies; restricted/excluded-loss curves
GOAL · ≤ 6 key frequencies explain ≥ 95% of logit variance; excluded loss rises before test loss falls; the second task's circuit is documented
Nanda et al. 2023 progress measures · reproduced locally—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Train a 1-layer transformer on (a+b) mod 113 with strong weight decay until it groks, meaning test accuracy jumps to near-perfect long after training accuracy reached 100%. Apply a DFT (discrete Fourier transform) to the embeddings to find the key frequencies: the network represents each number as sines and cosines at a few frequencies. Then compute restricted loss (the loss when you keep only those key frequencies) and excluded loss (the loss when you remove them) over the course of training. The twist: repeat for a second task (multiplication mod p, or a small permutation group) and test whether the universality story holds, that is, whether networks learn the same kind of circuit across related tasks.

VICTORY CONDITIONS

  1. ≤ 6 key frequencies explain ≥ 95% of logit variance; excluded loss rises before test loss falls; the second task's circuit is documented (grokking reproduced (test acc > 99% long after train acc reaches 100%); variance explained by the key frequencies; restricted/excluded-loss curves).
  2. A public repository with a README that shows your result and how to reproduce it.

Terms in plain words

Grokking
A network first memorises its training data and only much later, often suddenly, learns the rule that generalises.
Modular arithmetic
Arithmetic that wraps around, like a clock: (a+b) mod 113 is the remainder after dividing a+b by 113.
Key frequencies
The few sine and cosine frequencies the trained network actually uses to represent numbers.
Restricted / excluded loss
The loss after keeping only, or removing only, the key-frequency parts of the network's computation; it shows when the circuit forms.
Circuit
A specific, human-readable algorithm implemented by a small set of a network's weights and activations.
Universality
The hypothesis that different networks trained on similar tasks discover the same kinds of circuits.

Why it rates ★★★☆☆

Mechanistic reverse-engineering (working out the exact algorithm inside a trained network) is the standard entry ticket for interpretability teams, and the second task adds something of your own.