Brief
Sound is a list of numbers: air-pressure samples, 8,000 per second in this dataset. Raw samples are hard to classify, because the same word said twice gives very different waveforms. What stays similar is which frequencies are loud and when. A spectrogram shows exactly that, and once you have one, a spoken digit becomes a small image.
To make a spectrogram, cut the signal into short overlapping frames, multiply each by a window that fades its edges, and take the Fourier transform of each frame. This is the short-time Fourier transform (STFT). The squared magnitude of each frame's transform becomes one column. Human hearing is roughly logarithmic in pitch, so a mel spectrogram groups the frequencies into bands spaced evenly on the mel scale, , and you take the log of each band's energy.
The Free Spoken Digit Dataset (FSDD) has 3,000 recordings of the digits 0 to 9 by 6 speakers, 50 of each digit per
speaker, as 8 kHz WAV files. It's on GitHub under that name. Its README defines the test split: recordings numbered
0 to 4 are test, and 5 to 49 are training. Each file name holds the digit, the speaker and the recording number, as
in 7_jackson_32.wav.
You'll classify the spectrograms two ways with what you've already built: as images with a small CNN, and as sequences of frames with an RNN. Then you'll ask a harder question. Does it work for a voice it has never heard?
DATA · Free Spoken Digit Dataset
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- At least 95% accuracy on the official FSDD test split (recordings numbered 0 to 4), recorded with `python tools/mlarena.py record BL-17 "<accuracy>%"`.
- Your STFT power spectrogram matches librosa.stft with the same settings to a relative error under 1e-5.
- The README has the confusion matrix and the held-out-speaker accuracies, with one sentence on what the gap tells you.
Side quests
Terms in plain words
- Sample rate
- How many numbers per second a recording stores. This dataset uses 8,000, which keeps frequencies up to 4,000 Hz.
- Spectrogram
- A picture of sound with time across, frequency up, and brightness showing how loud each frequency is at each moment.
- Short-time Fourier transform (STFT)
- Cutting a signal into short overlapping frames and taking the Fourier transform of each one, which gives the columns of a spectrogram.
- Window function
- A curve such as the Hann window that fades each frame in and out, so the cut edges don't add false frequencies.
- Mel scale
- A pitch scale that matches human hearing, roughly linear below 1,000 Hz and logarithmic above it.