← All Cracked ML projects
PL-A01NOT STARTEDADVANCEDMLPYTHONIMPACT ★★★☆☆

Contamination detective: membership inference on open LLMs

Did the model see the test set during training? Reproduce Min-K%++ on WikiMIA, then audit a real benchmark against a public pretraining corpus.

EST.22 h
REWARD+400 XP
AFTERShi et al. + 2

ML-ARENA BENCHMARKS

PROOF: FINDING
METRICINDUSTRYMINE
AUROC (how well a score separates training members from non-members; 0.5 is chance, 1.0 is perfect) on WikiMIA at text lengths 32/64/128, per method and model; overlap rate per benchmark split
GOAL · within 2 AUROC pts of the paper's Min-K%++ numbers for ≥ 2 Pythia sizes; a written contamination report for one benchmark
Min-K%++ paper (beats the runner-up by 6.2–10.5 AUROC pts on WikiMIA) · reproduced locally—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Implement five membership scores from per-token log-probs on Pythia models, whose training data (the Pile) is public. Each score guesses whether a given text was in the training data. Perplexity: training texts look less surprising to the model. zlib: perplexity compared with how compressible the text is, so simple, repetitive text is not mistaken for memorised text. Reference-model: compare against a second model that did not see the text. Min-K%: average the log-probs of only the k% least likely tokens. Min-K%++: the same, but each token's log-prob is first normalised by the mean and spread of the model's predicted distribution at that position. Reproduce the WikiMIA AUROCs. Then build an n-gram overlap index over a Pile shard, using either a suffix array (a sorted list of every suffix of the text, for fast exact substring search) or a Bloom filter (a compact bit array that answers 'definitely not present' or 'probably present'). Publish contamination rates for one popular benchmark, cross-checked against the membership scores.

Approach

Every score here comes from the same quantity: the log-prob log⁡p(xt∣x<t)\log p(x_t \mid x_{<t}) of each real token. Perplexity averages it. Min-K% averages only the lowest k%k\%, because members rarely contain tokens the model found shocking. Min-K%++ first asks how surprising the token is compared with what the model expected at that position:

st=log⁡p(xt∣x<t)−μtσt,μt=Ez∼p(⋅∣x<t)[log⁡p(z∣x<t)],σt2=Ez[(log⁡p(z∣x<t)−μt)2]s_t = \frac{\log p(x_t \mid x_{<t}) - \mu_t}{\sigma_t}, \qquad \mu_t = \mathbb{E}_{z \sim p(\cdot \mid x_{<t})}[\log p(z \mid x_{<t})], \qquad \sigma_t^2 = \mathbb{E}_{z}\big[(\log p(z \mid x_{<t}) - \mu_t)^2\big]

and then averages the lowest k%k\% of the sts_t. Both μt\mu_t and σt\sigma_t are sums over the full vocabulary, so you get them from the same logits at no extra model cost. Compute them in fp32 from log_softmax, not from fp16 probabilities.

Build in this order: one function that returns per-token (log⁡p,μ,σ)(\log p, \mu, \sigma); a cached table of those for every WikiMIA text; then each score is a few lines of NumPy on the cache. AUROC only depends on ranking, so you can check it against sklearn.metrics.roc_auc_score. The paper's Pythia results (Table 5) are for 2.8B, 6.9B and 12B, with Pythia-70M as the reference model. 2.8B and 6.9B run in fp16 on one 24 GB GPU; WikiMIA has only 1,650 texts, so each model takes minutes. 12B needs a rented 40 or 80 GB card for an hour or two.

The contamination half is a separate engineering problem. The full original Pile is hard to get as plain text today, so check what you can download first. Two public routes: monology/pile-uncopyrighted (text, with the copyrighted subsets removed, so a miss there is weaker evidence) or the tokenized EleutherAI/pile-standard-pythia-preshuffled (602 GB, Pythia's exact training order). Start with a shard of a few GB. A suffix array over nn bytes takes about 8n8n bytes of index with 64-bit offsets and answers any substring query by binary search; a Bloom filter is far smaller but only answers fixed-length n-grams and has a false-positive rate (1−e−hm/b)h(1 - e^{-hm/b})^h for mm items, bb bits and hh hashes.

VISUALFig. 1
ONE POSITION: LOG-PROBS OVER A 12-TOKEN TOY VOCABULARY 0 -3 -6 μ = -1.84 band: μ ± σ, σ = 1.00 seen token score = (log p − μ) / σ = (-4.84 − (-1.84)) / 1.00 = -3.01 A WHOLE TEXT: ONE SCORE PER TOKEN, KEEP THE LOWEST k = 20% Min-K%++ = mean of the 4 dark bars = -2.15. Members tend to score higher.
Toy numbers, not model output. Top: one position's log-probs, with the seen token normalised by the position's mean and spread. Bottom: the per-token scores of a whole text, of which only the lowest k% are averaged. Why it's here: The normalisation is the whole difference between Min-K% and Min-K%++. See it on one position before you vectorise it.

Where people get stuck

  • Off-by-one in log-prob gathering. Logits at position tt predict token t+1t+1. Shift the targets, drop the first token, and test on a two-token string by hand.
  • Tokenizer drift. WikiMIA texts are whole strings. Tokenize without truncation, and do not let a BOS token appear for one model and not another.
  • Picking k on the test set. k changes AUROC by several points. Choose it on a held-out half or report the full k curve.
  • Text normalisation in the index. Lowercasing, whitespace and Unicode forms decide whether a benchmark item "matches". Normalise the corpus and the queries identically and say how.
  • Reading overlap as proof. A 13-gram hit may be a common phrase or a template. Hand-check samples before you publish a rate.
HINT · OBJECTIVE 1
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 5
Stuck? Hints stay hidden until you ask.

Objectives

0 / 7 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. Min-K%++ AUROC within 2 points of the paper's WikiMIA numbers (Table 5, original setting) at lengths 32, 64 and 128 for at least two Pythia sizes. Record it with `python tools/mlarena.py record PL-A01 "<mean |gap| in pts>"`.
  2. A written contamination report for one benchmark, giving the overlap rate per split with the n-gram length and matching rules stated.
  3. The report includes a cross-check between overlap and membership scores, with a confidence interval, even if the answer is "no detectable relation".

Side quests

Run the same audit on the deduplicated Pythia models and data. Does deduplication change the membership signal on the items you flagged?
Use the public infini-gram API (Pile-train index) as a second opinion on your own index, and explain every disagreement.
Test a paraphrase attack: rewrite 50 flagged items with another model and see how far each score falls.

Terms in plain words

Membership inference
Guessing, from a model's outputs alone, whether a particular text was part of its training data.
Contamination
Benchmark questions or answers appearing in a model's training data, which makes its benchmark scores look better than they are.
Min-K%++
A membership score that looks at a text's least likely tokens, after normalising each token's log-prob by what the model expected at that position.
AUROC
Area under the ROC curve: the chance that a random member gets a higher score than a random non-member.
Suffix array
A sorted index of every suffix of a large text, which lets you check whether any string occurs in it very quickly.
Bloom filter
A small bit array with several hash functions that can tell you an item is definitely absent or probably present.

Why it rates ★★★☆☆

Contamination (benchmark test data leaking into training data) is an active, valued eval-integrity problem. An original audit is a real contribution at the level of a solid blog post.