← All Base Form ML projects
BL-17NOT STARTEDBASE FORM · DEEP LEARNINGMLPYTHON

Spoken-digit recogniser

Recognise spoken digits from short recordings by turning sound into spectrograms and classifying them.

EST.7 h
REWARD+100 XP
AFTERoriginal

ML-ARENA BENCHMARKS

YOUR NUMBER VS THE INDUSTRY'S
METRICINDUSTRYMINE
test accuracy
GOAL · ≥ 95%
librosa spectrograms · measured—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Sound is a list of numbers: air-pressure samples, 8,000 per second in this dataset. Raw samples are hard to classify, because the same word said twice gives very different waveforms. What stays similar is which frequencies are loud and when. A spectrogram shows exactly that, and once you have one, a spoken digit becomes a small image.

To make a spectrogram, cut the signal into short overlapping frames, multiply each by a window that fades its edges, and take the Fourier transform of each frame. This is the short-time Fourier transform (STFT). The squared magnitude of each frame's transform becomes one column. Human hearing is roughly logarithmic in pitch, so a mel spectrogram groups the frequencies into bands spaced evenly on the mel scale, m=2595log⁡10(1+f/700)m = 2595 \log_{10}(1 + f/700), and you take the log of each band's energy.

The Free Spoken Digit Dataset (FSDD) has 3,000 recordings of the digits 0 to 9 by 6 speakers, 50 of each digit per speaker, as 8 kHz WAV files. It's on GitHub under that name. Its README defines the test split: recordings numbered 0 to 4 are test, and 5 to 49 are training. Each file name holds the digit, the speaker and the recording number, as in 7_jackson_32.wav.

You'll classify the spectrograms two ways with what you've already built: as images with a small CNN, and as sequences of frames with an RNN. Then you'll ask a harder question. Does it work for a voice it has never heard?

VISUALFig. 1
WAVEFORM frame = 256 samples (32 ms), hop = 128, Hann window SPECTROGRAM: one column per frame, time → high f low f
The STFT: a Hann window slides along the waveform in overlapping steps, and each windowed frame becomes one column of the spectrogram. Why it's here: Objective 2 asks you to build this by hand. The picture shows how frame length, hop and window fit together.
HINT · OBJECTIVE 2
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 6
Stuck? Hints stay hidden until you ask.

DATA · Free Spoken Digit Dataset

Objectives

0 / 6 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. At least 95% accuracy on the official FSDD test split (recordings numbered 0 to 4), recorded with `python tools/mlarena.py record BL-17 "<accuracy>%"`.
  2. Your STFT power spectrogram matches librosa.stft with the same settings to a relative error under 1e-5.
  3. The README has the confusion matrix and the held-out-speaker accuracies, with one sentence on what the gap tells you.

Side quests

Add background noise and time or frequency masking during training, then test on noisy recordings.
Record your own voice with the sounddevice library and run the model on it live.
Turn the log-mel spectrogram into MFCCs with a discrete cosine transform and compare accuracy.

Terms in plain words

Sample rate
How many numbers per second a recording stores. This dataset uses 8,000, which keeps frequencies up to 4,000 Hz.
Spectrogram
A picture of sound with time across, frequency up, and brightness showing how loud each frequency is at each moment.
Short-time Fourier transform (STFT)
Cutting a signal into short overlapping frames and taking the Fourier transform of each one, which gives the columns of a spectrogram.
Window function
A curve such as the Hann window that fades each frame in and out, so the cut edges don't add false frequencies.
Mel scale
A pitch scale that matches human hearing, roughly linear below 1,000 Hz and logarithmic above it.