← All Cracked ML projects
PL-A06NOT STARTEDADVANCEDMLPYTHONIMPACT ★★★☆☆

CIFAR-10 to 94% in seconds: rebuild airbench

Rebuild the world's fastest CIFAR-10 training run from the paper, then publish an ablation table (what each trick is worth when you take it away) for every trick.

EST.30 h
REWARD+400 XP
AFTERKeller Jordan + 2

ML-ARENA BENCHMARKS

PROOF: BENCH
METRICINDUSTRYMINE
mean test accuracy over 25 runs and the wall-clock time to finish on your GPU
GOAL · ≥ 94.0% mean accuracy in ≤ 1.5× airbench94_muon's time on the same GPU; an ablation table with 95% CIs (confidence intervals)
airbench94_muon: 94.01% in 2.59 s on an A100 (README); run it on your own GPU to get the reference time—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Write your own speedrun trainer without copying airbench. It needs: a whitening-initialised first conv (first-layer filters computed from the image patches' statistics so the layer's outputs start out decorrelated), BatchNorm tuning, label smoothing (train toward slightly softened targets instead of hard 0/1 labels), flip and translate augmentation, Nesterov SGD and then Muon written from scratch (an optimiser that orthogonalises each weight matrix's update before applying it), EMA/lookahead (keep a slow-moving average of the weights), test-time flips (average the predictions on an image and its mirror) and GPU-resident data loading (keep the whole dataset on the GPU so loading never waits on the CPU). Each trick gets an ablation over 25 seeds.

VICTORY CONDITIONS

  1. ≥ 94.0% mean accuracy in ≤ 1.5× airbench94_muon's time on the same GPU; an ablation table with 95% CIs (confidence intervals) (mean test accuracy over 25 runs and the wall-clock time to finish on your GPU).
  2. A public repository with a README that shows your result and how to reproduce it.

Terms in plain words

Ablation
Removing one component at a time and measuring how much the result drops, to see what each part contributes.
Whitening init
Setting the first conv layer's filters from the data so its outputs are decorrelated and equally scaled from step one.
Muon
An optimiser for weight matrices that replaces the momentum update with its nearest orthogonal matrix.
Label smoothing
Training on targets like 0.9 for the true class and a little for the others, which reduces over-confidence.
EMA of weights
A running average of the weights over training steps, which is often more accurate than the latest weights.
Confidence interval (CI)
A range that, at 95%, would contain the true mean in 95 out of 100 repeated experiments.

Why it rates ★★★☆☆

Speedrun engineering with honest ablations is exactly the empirical taste labs hire for.