← All Cracked ML projects
PL-B04NOT STARTEDBEGINNERMLPYTHONIMPACT ★★☆☆☆

100 KB language identifier vs fastText

Get within a couple of accuracy points of fastText's 917 kB language-ID model with a naive Bayes model 9× smaller.

EST.6 h
REWARD+200 XP
AFTERfastText language identification + 1

ML-ARENA BENCHMARKS

PROOF: BENCH
METRICINDUSTRYMINE
macro accuracy (accuracy averaged per language, so each language counts equally) on the WiLI-2018 test paragraphs, for the languages that overlap with lid.176, at a given model size
GOAL · within 2 accuracy points of lid.176.ftz with a model ≤ 100 KB
fastText lid.176.ftz (917 kB, 176 languages) · accuracy measured locally on the same split—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Build a multinomial naive Bayes classifier over character 1–5-grams (every run of 1 to 5 characters in the text). Use feature hashing (map each n-gram into a fixed number of buckets with a hash function instead of storing a vocabulary), n-gram pruning (drop rare or uninformative n-grams) and int8-quantised log-probabilities (store each log-probability as an 8-bit integer instead of a 32-bit float). Together these let the whole model ship in ≤ 100 KB. Evaluate on the WiLI-2018 languages that fastText's lid.176 model also covers. The size budget forces real feature-engineering choices.

VICTORY CONDITIONS

  1. within 2 accuracy points of lid.176.ftz with a model ≤ 100 KB (macro accuracy (accuracy averaged per language, so each language counts equally) on the WiLI-2018 test paragraphs, for the languages that overlap with lid.176, at a given model size).
  2. A public repository with a README that shows your result and how to reproduce it.

Terms in plain words

Character n-gram
A run of n consecutive characters, such as 'th' or 'ing', used as a feature.
Feature hashing
Mapping each feature to a bucket number with a hash function, so you never store a vocabulary and memory stays fixed.
int8 quantisation
Storing numbers as 8-bit integers plus a scale factor, which is 4 times smaller than 32-bit floats.
Macro accuracy
Accuracy computed separately for each class and then averaged, so rare languages count as much as common ones.

Why it rates ★★☆☆☆

A tiny model that stays competitive under a size budget is a nice engineering story, though language identification is a mature problem.