← All Base Form ML projects
BL-14NOT STARTEDBASE FORM · DEEP LEARNINGMLPYTHON

Convolution engine

Write convolution forward and backward in NumPy, make it fast with im2col, then classify pet photos.

EST.9 h
REWARD+100 XP
AFTERoriginal

ML-ARENA BENCHMARKS

YOUR NUMBER VS THE INDUSTRY'S
METRICINDUSTRYMINE
conv outputs; speedup
GOAL · within 1e-5; ≥ 20× faster than loops
torch.nn.functional.conv2d—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

A convolution slides a small filter across an image and takes a dot product at every position. The same weights are reused everywhere, so a 3 × 3 filter on a 3-channel image has only 27 weights plus a bias, whatever the image size. Stacks of these filters are what let a network pick out edges, then textures, then ears and noses. A network built from them is a convolutional neural network (CNN).

You write the forward and backward passes with plain loops first and check them against torch.nn.functional.conv2d. Deep-learning convolution doesn't flip the filter, so you are really computing a cross-correlation. With input height HH, kernel size kk, padding pp and stride ss, the output height is ⌊(H+2p−k)/s⌋+1\lfloor (H + 2p - k)/s \rfloor + 1.

Loops are correct but slow. The fix is a classic kernel trick called im2col. Copy every input patch into one column of a big matrix, reshape the filters into rows, and the whole convolution becomes one matrix multiplication that NumPy hands to a fast BLAS library. You trade memory for speed. To judge the result, count the work: a convolution does 2⋅NFCk2HoutWout2 \cdot N F C k^2 H_{out} W_{out} floating-point operations for NN images, FF filters and CC input channels.

Then you use your engine. The Oxford-IIIT Pet dataset has 7,349 photos of 37 cat and dog breeds, about 200 per breed, from Oxford's Visual Geometry Group. Load it with torchvision.datasets.OxfordIIITPet. Every breed is either a cat or a dog, so you can turn the 37 labels into 2 and start with the easier task.

VISUALFig. 1
input 4 × 4 3 × 3 patch im2col: patch → column filters as rows F × 9 (here F = 2) · patch matrix 9 × 4, one column per output position = output F × 4 map 1 map 2 reshape to F maps of 2 × 2
im2col on a 4 × 4 input with a 3 × 3 filter: each patch becomes one column, the filters become rows, and one matrix product gives every output at once. Why it's here: Objectives 3 and 4 depend on getting this layout right. Seeing which patch becomes which column makes the reshapes much easier.
HINT · OBJECTIVE 2
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 4
Stuck? Hints stay hidden until you ask.

DATA · Oxford-IIIT Pet

Objectives

0 / 6 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. Forward and backward outputs match torch.nn.functional.conv2d within 1e-5, and im2col runs at least 20× faster than your loops on the same input, recorded with `python tools/mlarena.py record BL-14 "<max error>; <speedup>x"`.
  2. On the official test split, the classifier beats always guessing dog. About two thirds of the photos are dogs.
  3. The README has a table of time and GFLOP/s for loops, im2col and PyTorch.

Side quests

Build the patch view with np.lib.stride_tricks.sliding_window_view and measure how much copying you avoid.
Train on all 37 breeds and show the confusion matrix. Which breeds get mixed up?
Plot the 16 filters of your first conv layer as small images after training.

Terms in plain words

Convolution
Sliding a small grid of weights over an image and taking a dot product at each position. Deep-learning libraries skip the kernel flip, so strictly it's cross-correlation.
Channel
One layer of numbers per pixel position. A colour photo has 3 channels, and each filter in a conv layer produces one output channel.
Stride and padding
Stride is how far the filter moves between positions. Padding adds a border of zeros so the filter can sit on edge pixels.
im2col
Copying every input patch into a column of one big matrix, so the whole convolution becomes a single matrix multiplication.
GFLOP/s
Billions of floating-point operations per second. It tells you how close your code gets to what the hardware can do.