Brief
Write your own safetensors reader (safetensors is the file format the weights ship in), load GPT-2 124M/355M, and implement the forward pass in NumPy: causal multi-head attention (each token attends only to earlier tokens), GELU activations and pre-LN (LayerNorm applied before each sub-block rather than after). Add a KV cache (store each layer's keys and values for past tokens, so each new token costs one step instead of re-running the whole sequence), temperature/top-k/top-p sampling, and batched prompts with padding masks (so shorter prompts in a batch ignore their padding). Tokenisation reuses PL-B07.
VICTORY CONDITIONS
- max diff ≤ 1e-4; KV cache ≥ 10× faster than no cache at 512 tokens; decode ≥ 1.0× HF's CPU tok/s (max |logit diff| vs HF over 50 prompts; decode tokens/s at batch 1 (fp32, CPU); KV-cache speedup at 512 tokens).
- A public repository with a README that shows your result and how to reproduce it.
Terms in plain words
- picoGPT
- A famous very short NumPy implementation of the GPT-2 forward pass.
- safetensors
- A simple, safe file format for storing model weights as named raw arrays with a JSON header.
- KV cache
- Saving the attention keys and values of earlier tokens so generating the next token does not recompute them.
- Pre-LN
- Placing LayerNorm at the start of each attention and MLP block, which is how GPT-2 is built.
- Padding mask
- A mask that stops attention from looking at filler tokens added to make prompts the same length.
Why it rates ★★☆☆☆
Everyone has seen picoGPT, a well-known tiny NumPy GPT-2. The exact-match test and beating HF's speed make yours credible, not novel.