← All Cracked Math projects
PM-B08NOT STARTEDBEGINNERMATHPYTHONIMPACT ★★☆☆☆

Bayesian ranking: sort Stack Exchange posts without being fooled by one vote

One upvote and zero downvotes is not 100%. Prove it with a time-split test.

EST.6 h
REWARD+200 XP
AFTEREvan Miller + 1

ML-ARENA BENCHMARKS

PROOF: BENCH
METRICINDUSTRYMINE
Kendall tau (a rank correlation where +1 means the two orders agree perfectly) between the early-vote ranking and the final ranking, on 3 sites
GOAL · The empirical-Bayes posterior beats the raw average by >= 0.10 tau and Wilson by >= 0.02 tau on all 3 sites.
Raw-average and Wilson-bound ranking (Evan Miller) · measured locally—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

Rank posts three ways. First, by raw average (the share of votes that are upvotes). Second, by the Wilson lower bound (a cautious estimate of the true upvote share that is lower when a post has few votes). Third, by a Beta-Binomial posterior quantile (a low percentile of the Bayesian estimate of each post's upvote rate). Fit the prior for the third method by empirical Bayes using the method of moments: estimate it from the vote histories in the Stack Exchange data dump by matching the prior's mean and variance to the data. Evaluate on a time split: rank using only each post's first 20% of votes, then score that ranking against the ranking from all votes.

VICTORY CONDITIONS

  1. The empirical-Bayes posterior beats the raw average by >= 0.10 tau and Wilson by >= 0.02 tau on all 3 sites. (Kendall tau (a rank correlation where +1 means the two orders agree perfectly) between the early-vote ranking and the final ranking, on 3 sites).
  2. A public repository with a README that shows your result and how to reproduce it.

Terms in plain words

Beta-Binomial model
A model where each post has an unknown upvote rate drawn from a Beta distribution, and its votes are Binomial given that rate.
Empirical Bayes
Choosing the prior by fitting it to the whole dataset, then using it to estimate each individual item.
Method of moments
Fitting a distribution by making its mean and variance equal to the mean and variance seen in the data.
Wilson lower bound
The bottom of the Wilson confidence interval for a proportion, used to rank items so that few votes count as less certain.
Kendall tau
A number from -1 to +1 that measures how often two rankings put pairs of items in the same order.

Why it rates ★★☆☆☆

A clean, well-measured applied-Bayes result, though the method itself is a decade old.