Brief
Build PUCT MCTS with batched neural evaluation. MCTS (Monte Carlo tree search) plans by simulating many possible continuations; PUCT is the AlphaZero rule that uses the network's move probabilities and value estimates to choose which branches to explore. Add a small ResNet policy/value net (one head suggests moves, the other predicts who will win), self-play workers, a replay buffer (a store of recent self-play positions to train on) and an arena-gated training loop (a new network replaces the current one only if it wins a head-to-head match). Grade the agent with Pascal Pons's solver: the share of optimal moves on random mid-game positions, and games played as first player (a forced win under perfect play).
VICTORY CONDITIONS
- ≥ 95% optimal moves; ≥ 90% wins as first player vs perfect play; ≤ 24 GPU-hours of self-play (% optimal moves on 1,000 solver-labelled positions; win rate as first player vs the solver; GPU-hours used).
- A public repository with a README that shows your result and how to reproduce it.
Terms in plain words
- AlphaZero
- A method that learns a game purely by playing itself, combining tree search with a neural network that guides it.
- MCTS
- Monte Carlo tree search: build a tree of possible moves by repeatedly simulating games from the current position.
- PUCT
- The selection rule inside AlphaZero's search that balances the network's suggested moves against moves that are still unexplored.
- Policy / value net
- One network with two outputs: a probability for each move, and an estimate of who will win.
- Arena gating
- Only promoting a newly trained network if it beats the current best one in a match.
- Solved game
- A game where the perfect move in every position is known, so any agent's play can be checked exactly.
Why it rates ★★★☆☆
A complete AlphaZero loop graded by ground truth is far more convincing than reward curves from CartPole.