Topic map
Every topic, the syllabus node it belongs to, and the projects that use it. In Base Form, yellow marks the project that introduces a topic; the rest reuse it.
| TOPIC | SYLLABUS NODE | BASE FORM | CRACKED |
|---|---|---|---|
| Linear algebra | |||
vectors, dot products, norms, angles, projections la.vectors | Linear algebra fundamentals (in progress) | ||
matrix multiplication, linear maps, composition la.matmul | Linear algebra fundamentals (in progress) | ||
linear systems, Gaussian elimination, LU, pivoting la.systems | Linear algebra fundamentals (in progress) | ||
least squares, QR, Gram-Schmidt, Householder la.leastsq | Linear algebra fundamentals (in progress) | ||
determinants, inverses, rank, null space la.det-inverse | Linear algebra fundamentals (in progress) | ||
eigenvalues/eigenvectors, power iteration, diagonalisation la.eigen | Linear algebra fundamentals (in progress) | ||
SVD, PCA, low-rank approximation la.svd | Linear algebra fundamentals (in progress) | ||
sparse matrices, graph Laplacian, PageRank, spectral methods la.sparse-graphs | Linear algebra fundamentals (in progress) | ||
conditioning, numerical stability, floating point, precision (fp32/fp16/bf16/int8) la.conditioning | Linear algebra fundamentals (in progress) | ||
tensors, broadcasting, strides, memory layout la.tensors | Linear algebra fundamentals (in progress) | ||
iterative solvers: conjugate gradient, Lanczos, Krylov la.iterative | Linear algebra fundamentals (in progress) | ||
| Multivariable calculus & optimisation | |||
derivatives, finite differences, limits numerically calc.derivatives | Multivariable calculus (locked) | ||
partial derivatives, gradients, directional derivatives calc.gradients | Multivariable calculus (locked) | ||
chain rule, Jacobians, matrix calculus calc.chain-jacobian | Multivariable calculus (locked) | ||
Hessians, curvature, Taylor expansion, Newton's method calc.hessian | Multivariable calculus (locked) | ||
gradient descent, momentum, Adam, learning-rate schedules calc.gd | Multivariable calculus (locked) | ||
convexity, constrained optimisation, Lagrange multipliers, LP/QP calc.convex | Multivariable calculus (locked) | ||
automatic differentiation, forward and reverse mode calc.autodiff | Multivariable calculus (locked) | ||
numerical integration, quadrature calc.integration | Multivariable calculus (locked) | ||
ODE solvers (Euler, RK4, adaptive), adjoint method calc.odes | Multivariable calculus (locked) | ||
Fourier series, DFT/FFT, convolution theorem calc.fourier | Multivariable calculus (locked) | ||
| Probability, statistics & information | |||
probability rules, combinatorics, conditional probability prob.basics | Probability and statistics (locked) | ||
random variables, common distributions, PDFs/CDFs prob.distributions | Probability and statistics (locked) | ||
expectation, variance, LLN, CLT, concentration prob.expectation | Probability and statistics (locked) | ||
Bayes' rule, Bayesian inference, conjugate priors prob.bayes | Probability and statistics (locked) | ||
MLE, MAP, likelihood prob.mle | Probability and statistics (locked) | ||
RNGs, Monte Carlo, importance sampling, inverse-CDF sampling prob.sampling | Probability and statistics (locked) | ||
Markov chains, Metropolis-Hastings, Gibbs, HMC/NUTS prob.mcmc | Probability and statistics (locked) | ||
hypothesis testing, bootstrap, A/B testing, confidence intervals prob.testing | Probability and statistics (locked) | ||
entropy, cross-entropy, KL, mutual information, compression prob.info | Probability and statistics (locked) | ||
Gaussian processes, kernels prob.gp | Probability and statistics (locked) | ||
Kalman/particle filters, HMMs prob.state-space | Probability and statistics (locked) | ||
| Classical ML | |||
linear & polynomial regression, regularisation ml.linreg | Linear algebra fundamentals (in progress) | ||
logistic/softmax regression, cross-entropy loss ml.logreg | Multivariable calculus (locked) | ||
nearest neighbours, distance metrics, approximate NN / vector search ml.knn | Linear algebra fundamentals (in progress) | ||
decision trees, random forests, gradient boosting ml.trees | Probability and statistics (locked) | ||
SVMs, kernel methods ml.svm-kernels | Multivariable calculus (locked) | ||
k-means, GMM, EM ml.clustering | Probability and statistics (locked) | ||
PCA, t-SNE/UMAP, autoencoders as dim-red ml.dimred | Linear algebra fundamentals (in progress) | ||
naive Bayes, generative classifiers ml.naive-bayes | Probability and statistics (locked) | ||
train/val/test, cross-validation, metrics, calibration, leakage ml.eval | Probability and statistics (locked) | ||
feature engineering, tabular data, missing data ml.features | Probability and statistics (locked) | ||
| Deep learning core | |||
perceptron, MLP, activations dl.mlp | Backprop from scratch (locked) | ||
backpropagation, computational graphs dl.backprop | Backprop from scratch (locked) | ||
initialisation, batch/layer norm, residuals dl.init-norm | Backprop from scratch (locked) | ||
optimisers in practice, schedules, Muon-style methods dl.optim | Backprop from scratch (locked) | ||
dropout, weight decay, augmentation, overfitting dl.regularise | Backprop from scratch (locked) | ||
convolution, CNNs, vision dl.cnn | Backprop from scratch (locked) | ||
RNN, LSTM, GRU, sequence modelling dl.rnn | Backprop from scratch (locked) | ||
embeddings, word2vec, contrastive learning dl.embeddings | Backprop from scratch (locked) | ||
VAEs, GANs, diffusion, normalising flows dl.generative | Backprop from scratch (locked) | ||
bandits, Q-learning, policy gradients, MCTS/AlphaZero dl.rl | Backprop from scratch (locked) | ||
graph neural networks dl.gnn | Backprop from scratch (locked) | ||
audio, speech, time series dl.audio-ts | Backprop from scratch (locked) | ||
| Transformers & LLMs | |||
tokenisation, BPE tf.tokenize | Transformer from scratch (locked) | ||
attention, multi-head, positional encodings (RoPE) tf.attention | Transformer from scratch (locked) | ||
language modelling, n-gram → GPT, sampling/decoding tf.lm | Transformer from scratch (locked) | ||
training at scale: mixed precision, data/pipeline parallel, speedruns tf.training | Transformer from scratch (locked) | ||
KV cache, quantisation, batching, speculative decoding tf.inference | Transformer from scratch (locked) | ||
fine-tuning, LoRA, instruction tuning, RLHF/DPO tf.finetune | Transformer from scratch (locked) | ||
retrieval, RAG, rerankers tf.retrieval | Transformer from scratch (locked) | ||
LLM evaluation, benchmarks, contamination tf.eval | Transformer from scratch (locked) | ||
vision transformers, CLIP tf.multimodal | Transformer from scratch (locked) | ||
| Interpretability | |||
linear probes, world models interp.probes | Interpretability basics (locked) | ||
logit lens, tuned lens interp.lens | Interpretability basics (locked) | ||
circuits, attention patterns, induction heads, grokking interp.circuits | Interpretability basics (locked) | ||
sparse autoencoders, superposition, features interp.sae | Interpretability basics (locked) | ||
activation steering, ablations, causal interventions interp.steering | Interpretability basics (locked) | ||
| ML systems | |||
performance kernels, SIMD, GPU programming, roofline sys.kernels | Linear algebra fundamentals (in progress) | ||
memory layout, caching, arenas, file formats (GGUF/safetensors) sys.memory | Transformer from scratch (locked) | ||
threads, distributed training sys.parallel | Transformer from scratch (locked) | ||
graph capture, op fusion, JIT sys.compilers | Transformer from scratch (locked) | ||
model serving, latency, throughput sys.serving | Transformer from scratch (locked) | ||