Brief
Face detection asks one question over and over: is this small square of the image a face or not? Before deep learning, the best answers came from support vector machines. In this project you build one from scratch, feed it face and non-face patches, and use it to scan a photo.
For faces, use the Olivetti faces dataset, which ships with scikit-learn as sklearn.datasets.fetch_olivetti_faces.
It has 400 grayscale 64 × 64 images, ten each of 40 people. For non-faces, cut random 64 × 64 patches from photos
with no people in them. The two sample images in sklearn.datasets.load_sample_images are a start. Add a few of your
own landscape or object photos so the negatives aren't all from two pictures.
A linear SVM finds the line (in high dimensions, the hyperplane) with the widest margin between the classes. With labels , the soft-margin version minimises
The trick that makes SVMs powerful is the kernel. Written in its dual form, the classifier only uses dot products between points: . Swap the dot product for a kernel such as the RBF kernel and the boundary can curve, with no extra features computed. Most end up zero. The points with above zero are the support vectors.
DATA · Olivetti faces + random patches
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- Your SVM's test accuracy is within 1% of sklearn SVC with the same kernel, C and gamma, on a split where no person appears in both train and test. Record it with `python tools/mlarena.py record BL-08 "<your accuracy>"`.
- The PCA and any scaling are fitted on training images only.
- The README shows a grid of the test images your model got wrong and a sentence on what they have in common.
Side quests
Terms in plain words
- Support vector machine (SVM)
- A classifier that draws the boundary between two classes with the widest possible gap, or margin, on each side.
- Support vectors
- The training points that sit on or inside the margin. Only they decide where the boundary goes.
- Soft margin
- Allowing some points to break the margin for a penalty, set by C. Large C punishes mistakes hard, small C allows a wider, looser margin.
- Kernel
- A similarity function k(x, z) that acts like a dot product in a bigger feature space, so a linear method can draw curved boundaries.
- RBF kernel
- The Gaussian kernel exp(-γ‖x − z‖²). Nearby points score near 1 and far points near 0, and γ sets what counts as near.