← All Base Form ML projects
BL-07NOT STARTEDBASE FORM · CLASSICAL MLMLPYTHON

Map of anything

Turn high-dimensional data (songs, words, digits) into a 2D map you can explore, with PCA and t-SNE.

EST.6 h
REWARD+100 XP
AFTERvan der Maaten & Hinton

ML-ARENA BENCHMARKS

YOUR NUMBER VS THE INDUSTRY'S
METRICINDUSTRYMINE
PCA output; neighbourhood trustworthiness
GOAL · PCA within 1e-6; trustworthiness within 0.02
sklearn PCA · sklearn TSNE—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

A song described by ten audio features is a point in ten-dimensional space. You can't look at that. In this project you build a 2D map of it, so that similar songs sit near each other and you can see the structure by eye. The same code works for any table of numbers: songs, words, images of digits.

The data is the "Spotify Tracks Dataset" on Kaggle. It has over 100,000 tracks across more than a hundred genres, with audio features such as danceability, energy, loudness, acousticness, valence and tempo. Pick five or six genres that sound different so the map has something to show. Exact t-SNE compares every pair of points, so work with a sample of about 3,000 tracks.

PCA is the linear method. It finds the directions of greatest variance. These are the eigenvectors of the covariance matrix, and you get them from the SVD of the centred data, X=USV⊤X = U S V^\top. The top components are the first columns of VV, and the map is XV:,1:2X V_{:,1:2}. Component kk explains a share sk2/∑jsj2s_k^2 / \sum_j s_j^2 of the total variance.

t-SNE is non-linear. It turns distances into neighbour probabilities pijp_{ij} in the original space, with a Gaussian around each point. On the map it uses a heavier-tailed Student-t kernel, qij∝(1+∥yi−yj∥2)−1q_{ij} \propto (1 + \lVert y_i - y_j \rVert^2)^{-1}, and moves the points to make QQ match PP by minimising the KL divergence ∑i≠jpijlog⁡pijqij\sum_{i \neq j} p_{ij} \log \frac{p_{ij}}{q_{ij}}. Its gradient is

∂C∂yi=4∑j(pij−qij)(yi−yj)(1+∥yi−yj∥2)−1.\frac{\partial C}{\partial y_i} = 4 \sum_j (p_{ij} - q_{ij})(y_i - y_j)(1 + \lVert y_i - y_j \rVert^2)^{-1}.

VISUALFig. 1
PC1most variance PC2 DIRECTIONS = EIGENVECTORS OF THE COVARIANCE MATRIX
PCA finds the direction of greatest spread (PC1) and the next one at right angles (PC2). Why it's here: Before you compute an SVD, see what the components mean: the axes your 2D map is drawn on.
HINT · OBJECTIVE 3
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 5
Stuck? Hints stay hidden until you ask.

DATA · a public music-features dataset

Objectives

0 / 6 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. Your PCA coordinates match sklearn PCA to within 1e-6 (after aligning signs), and your t-SNE trustworthiness is within 0.02 of sklearn TSNE on the same 3,000 tracks. Record the trustworthiness with `python tools/mlarena.py record BL-07 "<your trustworthiness>"`.
  2. The KL divergence falls over the run, and the README shows the PCA map and the t-SNE map side by side, coloured by genre.
  3. The README says in a few sentences which genres separate, which overlap, and whether the two maps agree.

Side quests

Run k-means from BL-06 on the original features and colour the t-SNE map by cluster instead of genre. Do the clusters match the islands?
Point the same code at sklearn's load_digits (8 × 8 handwritten digits) and see whether the ten digits form ten islands.

Terms in plain words

Dimensionality reduction
Squeezing data with many columns into a few, while keeping as much of its structure as possible.
PCA
Principal component analysis. It finds the directions along which the data varies most and projects onto the top few.
t-SNE
A method that places points in 2D so that points which were near neighbours in the original space stay near each other.
Perplexity
The t-SNE setting for roughly how many neighbours each point pays attention to. Typical values are 5 to 50.
Trustworthiness
A score from 0 to 1 for how many of a point's neighbours on the map were also its neighbours in the original data.