Brief
Every neural network is mostly matrix multiplication. NumPy hands the work to a BLAS library that experts have tuned
for decades, so A @ B is absurdly fast. In this project you climb the same ladder they did, one rung at a time,
and measure every step.
You start with three nested Python loops. Then you change the loop order, rewrite the product as a sum of outer products so NumPy can do most of the work, and try blocking. You'll get a big speedup and still see why the library wins by a mile: it runs compiled C with SIMD instructions and many threads.
The maths you need: entry of is , and the same product can be written as , where is column of and is row of .
DATA · random matrices
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- At least 50× faster than your naive loop at 512 × 512, measured with `python tools/mlarena.py bench BM-02 your_file.py`.
- No np.matmul, np.dot, @, einsum or tensordot anywhere in your solution.
- Matches NumPy to a relative error under 1e-4, and the README has the chart and one paragraph per rung.
Side quests
Terms in plain words
- Matrix multiplication
- Each entry of C = A·B is the dot product of a row of A with a column of B.
- Outer product
- A column vector times a row vector gives a whole matrix; C is the sum of n of them.
- Broadcasting
- NumPy stretches arrays of different shapes so one expression works on whole blocks at once, without a Python loop.
- Cache
- Small, fast memory next to the CPU. Code that reuses nearby data stays in cache and runs much faster.