Brief
Quantile-bin every feature into 255 bins, so each bin holds about the same number of rows. Build gradient/hessian histograms with np.bincount: for each bin, sum the first and second derivatives of the loss over the rows in it. Use the parent-minus-sibling histogram trick: build the histogram for the smaller child only and get the other child by subtracting it from the parent. Grow trees leaf-wise (always split the leaf with the biggest gain, rather than one whole level at a time) using L2-regularised second-order gain (the XGBoost-style split score built from gradient and hessian sums), min_child_weight (a minimum hessian sum per leaf) and NaN routing (learn which side of each split missing values go to). Train on 1M HIGGS rows and test on the official last-500k split.
VICTORY CONDITIONS
- AUC within 0.003 of LightGBM at ≤ 10× its wall-clock (test AUC and training wall-clock at equal hyper-parameters (num_leaves=255, 500 rounds, lr=0.1)).
- A public repository with a README that shows your result and how to reproduce it.
Terms in plain words
- GBDT
- Gradient-boosted decision trees: each new tree is fitted to correct the errors of all the trees before it.
- Histogram binning
- Replacing each feature value with a bin number so split search only has to scan 255 bins instead of every unique value.
- Second-order gain
- A split score that uses both the gradient and the hessian (curvature) of the loss, as XGBoost and LightGBM do.
- Leaf-wise growth
- Always splitting whichever leaf improves the loss most, which gives deeper, uneven trees.
- AUC
- The probability that the model ranks a random positive example above a random negative one.
Why it rates ★★☆☆☆
GBDTs (gradient-boosted decision trees) still win on tabular data and few people can build one. Matching LightGBM's AUC is credible.