← All Cracked ML projects
PL-A12NOT STARTEDADVANCEDMLPYTHONIMPACT ★★★☆☆

A tiny deep-learning compiler: trace, fuse, generate code

Turn your autograd engine into a compiler that fuses a transformer block into a handful of generated kernels.

EST.38 h
REWARD+400 XP
AFTERJAX + 2

ML-ARENA BENCHMARKS

PROOF: BENCH
METRICINDUSTRYMINE
forward+backward speedup over eager mode on a GPT-2 block and an MLP; kernel count; peak memory
GOAL · ≥ 70% of torch.compile's speedup over eager; kernel count ≤ 2× Inductor's; outputs match eager to 1e-5
torch.compile (Inductor backend) on the same block · measured locally—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Add a tracer to PL-I04 that captures the forward and backward graphs into an SSA IR (an intermediate representation where every value is assigned exactly once, which makes rewriting the graph easy). Write passes for DCE (dead-code elimination: drop results nobody uses), CSE (common-subexpression elimination: compute repeated expressions once), constant folding (precompute anything that depends only on constants), shape propagation (work out every tensor's shape ahead of time), elementwise and reduction fusion (merge chains of ops into one kernel so intermediate results never go to memory), and liveness-based memory planning (reuse a buffer as soon as the value in it is no longer needed). Then generate Triton (GPU) or vectorised NumPy (CPU) kernels, with a cache. Check correctness against eager mode.

VICTORY CONDITIONS

  1. ≥ 70% of torch.compile's speedup over eager; kernel count ≤ 2× Inductor's; outputs match eager to 1e-5 (forward+backward speedup over eager mode on a GPT-2 block and an MLP; kernel count; peak memory).
  2. A public repository with a README that shows your result and how to reproduce it.

Terms in plain words

Tracer
Code that runs the model once and records every operation into a graph instead of only computing numbers.
SSA IR
A program representation where each variable is assigned once, so it is easy to see where every value comes from.
Fusion
Combining several operations into one generated kernel to avoid writing intermediate results to memory.
DCE / CSE
Two standard clean-up passes: remove computations whose results are unused, and reuse identical computations.
Memory planning
Deciding ahead of time which buffers can share memory because their values are never alive at the same time.
Inductor
The default compiler backend behind torch.compile, which generates Triton kernels.

Why it rates ★★★☆☆

Few applicants can say they wrote a fusing compiler. Even at 70% of Inductor it is a strong systems signal.