Brief
Write your own speedrun trainer without copying airbench. It needs: a whitening-initialised first conv (first-layer filters computed from the image patches' statistics so the layer's outputs start out decorrelated), BatchNorm tuning, label smoothing (train toward slightly softened targets instead of hard 0/1 labels), flip and translate augmentation, Nesterov SGD and then Muon written from scratch (an optimiser that orthogonalises each weight matrix's update before applying it), EMA/lookahead (keep a slow-moving average of the weights), test-time flips (average the predictions on an image and its mirror) and GPU-resident data loading (keep the whole dataset on the GPU so loading never waits on the CPU). Each trick gets an ablation over 25 seeds.
VICTORY CONDITIONS
- ≥ 94.0% mean accuracy in ≤ 1.5× airbench94_muon's time on the same GPU; an ablation table with 95% CIs (confidence intervals) (mean test accuracy over 25 runs and the wall-clock time to finish on your GPU).
- A public repository with a README that shows your result and how to reproduce it.
Terms in plain words
- Ablation
- Removing one component at a time and measuring how much the result drops, to see what each part contributes.
- Whitening init
- Setting the first conv layer's filters from the data so its outputs are decorrelated and equally scaled from step one.
- Muon
- An optimiser for weight matrices that replaces the momentum update with its nearest orthogonal matrix.
- Label smoothing
- Training on targets like 0.9 for the true class and a little for the others, which reduces over-confidence.
- EMA of weights
- A running average of the weights over training steps, which is often more accurate than the latest weights.
- Confidence interval (CI)
- A range that, at 95%, would contain the true mean in 95 out of 100 repeated experiments.
Why it rates ★★★☆☆
Speedrun engineering with honest ablations is exactly the empirical taste labs hire for.