Brief
Train 3D-CNN / 2.5D ink-detection models (networks that look at a stack of scan slices through the papyrus surface and predict where there is ink) on the public fragment and segment data. Fragments are detached pieces with known ink labels; segments are surfaces virtually unrolled from intact scrolls. Use careful leakage-free spatial validation (PL-B06 skills): hold out whole regions, so overlapping or neighbouring patches of the same area never end up in both train and validation. Score a late submission on the 2023 Kaggle ink-detection competition. Then turn the work into an open-source tool, or into an ink prediction on a new segment submitted for a monthly Progress Prize (or as a First Letters candidate).
Approach
The model looks at a thin stack of CT slices through the papyrus surface and paints a 2D ink mask. The competition gave three training fragments with 65 slices each, and the hidden test set is two fragments of about the size of one training fragment. The score is for each test fragment, averaged:
With , a false ink pixel costs more than a missed one, because spurious strokes ruin letter shapes. Your threshold is therefore part of the model. Choose it on held-out regions, never on the pixels you trained on.
Build in this order: folds first, then a simple 2.5D baseline, then a 3D encoder. Ink signal is faint and labels come from infrared photos, so the honest validation score is noisy. Region hold-out is the only way to know whether a change helped. The archived private leaderboard (the copy in OpenAI's MLE-bench) has 1,250 entries. The winner scored 0.831, entry 125 scored 0.660 and the median was 0.418. Those are the numbers to place yourself against.
Compute is bounded by inference, not training. A Kaggle GPU notebook may run at most 9 hours with internet off, and offline pretraining is allowed if it is documented. Measure one fold's epoch time before you plan a sweep.
The prize work is on scroll data, not fragments. Data now lives in an open S3 bucket, mostly as OME-Zarr, with segments as tifxyz meshes. The community code is the ScrollPrize/villa monorepo, and the current public ink model is ink_canonical_2um on Hugging Face. Progress Prizes are judged monthly: a guaranteed $20,000 best submission of the month plus smaller awards. You must be registered on the Vesuvius Challenge Discord when you submit.
Where people get stuck
- Leaky validation. Patches with stride smaller than their size overlap. A random patch split then tests the model on pixels it trained on, and the score looks far better than it is.
- Threshold tuned on the wrong data. is sensitive to the threshold. Tuning it on training pixels inflates precision. Tune per fold on held-out regions and check the chosen thresholds agree.
- Mask edges. Pixels outside the papyrus mask are not part of the problem. Zero them in predictions, or tiles at the edge add false positives.
- Slice choice. Ink sits near the papyrus surface, and the useful slices differ between fragments. Plot mean intensity against slice index per fragment before fixing a window.
- Fragment success that doesn't transfer. Scroll segments differ from fragments in resolution and surface quality. A fragment gain is a hypothesis about scrolls until you test it on a segment.
Objectives
Read-only. Log in to update progress.
VICTORY CONDITIONS
- Late-submission F0.5 of at least 0.660 on the private leaderboard, the top 10% of the 1,250 archived entries, recorded with `python tools/mlarena.py record PL-W04 "<private F0.5>"`.
- An awarded Progress Prize, or a merged pull request in ScrollPrize/villa.
- A documented spatial validation protocol, with the gap between cross-validation and the private score reported.
Side quests
Terms in plain words
- Vesuvius Challenge
- An open effort to read Herculaneum scrolls, carbonised by Vesuvius in AD 79, from X-ray CT scans.
- 2.5D CNN
- A CNN that treats a few stacked scan slices as input channels, a cheaper middle ground between 2D and full 3D.
- Fragment vs segment
- A fragment is a loose piece with visible, labelled ink; a segment is a surface digitally peeled from an unopened scroll.
- Spatial validation
- Splitting data by location so the validation area is truly unseen, instead of random patches that overlap training patches.
- Progress Prize
- A monthly award from the Vesuvius Challenge for useful tools or discoveries.
- F0.5 score
- A mix of precision and recall that weights precision more, so false ink costs more than missed ink.
- Run-length encoding
- Storing a binary mask as pairs of start position and run length, which is how Kaggle expects the predictions.
Why it rates ★★★★★
A prize-winning contribution to a famous science effort is a standout story for any hiring manager.