Topic map

Every topic, the syllabus node it belongs to, and the projects that use it. In Base Form, yellow marks the project that introduces a topic; the rest reuse it.

TOPICSYLLABUS NODEBASE FORMCRACKED
Linear algebra
vectors, dot products, norms, angles, projections
la.vectors
Linear algebra fundamentals (in progress)
matrix multiplication, linear maps, composition
la.matmul
Linear algebra fundamentals (in progress)
linear systems, Gaussian elimination, LU, pivoting
la.systems
Linear algebra fundamentals (in progress)
least squares, QR, Gram-Schmidt, Householder
la.leastsq
Linear algebra fundamentals (in progress)
determinants, inverses, rank, null space
la.det-inverse
Linear algebra fundamentals (in progress)
eigenvalues/eigenvectors, power iteration, diagonalisation
la.eigen
Linear algebra fundamentals (in progress)
SVD, PCA, low-rank approximation
la.svd
Linear algebra fundamentals (in progress)
sparse matrices, graph Laplacian, PageRank, spectral methods
la.sparse-graphs
Linear algebra fundamentals (in progress)
conditioning, numerical stability, floating point, precision (fp32/fp16/bf16/int8)
la.conditioning
Linear algebra fundamentals (in progress)
tensors, broadcasting, strides, memory layout
la.tensors
Linear algebra fundamentals (in progress)
iterative solvers: conjugate gradient, Lanczos, Krylov
la.iterative
Linear algebra fundamentals (in progress)
Multivariable calculus & optimisation
derivatives, finite differences, limits numerically
calc.derivatives
Multivariable calculus (locked)
partial derivatives, gradients, directional derivatives
calc.gradients
Multivariable calculus (locked)
chain rule, Jacobians, matrix calculus
calc.chain-jacobian
Multivariable calculus (locked)
Hessians, curvature, Taylor expansion, Newton's method
calc.hessian
Multivariable calculus (locked)
gradient descent, momentum, Adam, learning-rate schedules
calc.gd
Multivariable calculus (locked)
convexity, constrained optimisation, Lagrange multipliers, LP/QP
calc.convex
Multivariable calculus (locked)
automatic differentiation, forward and reverse mode
calc.autodiff
Multivariable calculus (locked)
numerical integration, quadrature
calc.integration
Multivariable calculus (locked)
ODE solvers (Euler, RK4, adaptive), adjoint method
calc.odes
Multivariable calculus (locked)
Fourier series, DFT/FFT, convolution theorem
calc.fourier
Multivariable calculus (locked)
Probability, statistics & information
probability rules, combinatorics, conditional probability
prob.basics
Probability and statistics (locked)
random variables, common distributions, PDFs/CDFs
prob.distributions
Probability and statistics (locked)
expectation, variance, LLN, CLT, concentration
prob.expectation
Probability and statistics (locked)
Bayes' rule, Bayesian inference, conjugate priors
prob.bayes
Probability and statistics (locked)
MLE, MAP, likelihood
prob.mle
Probability and statistics (locked)
RNGs, Monte Carlo, importance sampling, inverse-CDF sampling
prob.sampling
Probability and statistics (locked)
Markov chains, Metropolis-Hastings, Gibbs, HMC/NUTS
prob.mcmc
Probability and statistics (locked)
hypothesis testing, bootstrap, A/B testing, confidence intervals
prob.testing
Probability and statistics (locked)
entropy, cross-entropy, KL, mutual information, compression
prob.info
Probability and statistics (locked)
Gaussian processes, kernels
prob.gp
Probability and statistics (locked)
Kalman/particle filters, HMMs
prob.state-space
Probability and statistics (locked)
Classical ML
linear & polynomial regression, regularisation
ml.linreg
Linear algebra fundamentals (in progress)
logistic/softmax regression, cross-entropy loss
ml.logreg
Multivariable calculus (locked)
nearest neighbours, distance metrics, approximate NN / vector search
ml.knn
Linear algebra fundamentals (in progress)
decision trees, random forests, gradient boosting
ml.trees
Probability and statistics (locked)
SVMs, kernel methods
ml.svm-kernels
Multivariable calculus (locked)
k-means, GMM, EM
ml.clustering
Probability and statistics (locked)
PCA, t-SNE/UMAP, autoencoders as dim-red
ml.dimred
Linear algebra fundamentals (in progress)
naive Bayes, generative classifiers
ml.naive-bayes
Probability and statistics (locked)
train/val/test, cross-validation, metrics, calibration, leakage
ml.eval
Probability and statistics (locked)
feature engineering, tabular data, missing data
ml.features
Probability and statistics (locked)
Deep learning core
perceptron, MLP, activations
dl.mlp
Backprop from scratch (locked)
backpropagation, computational graphs
dl.backprop
Backprop from scratch (locked)
initialisation, batch/layer norm, residuals
dl.init-norm
Backprop from scratch (locked)
optimisers in practice, schedules, Muon-style methods
dl.optim
Backprop from scratch (locked)
dropout, weight decay, augmentation, overfitting
dl.regularise
Backprop from scratch (locked)
convolution, CNNs, vision
dl.cnn
Backprop from scratch (locked)
RNN, LSTM, GRU, sequence modelling
dl.rnn
Backprop from scratch (locked)
embeddings, word2vec, contrastive learning
dl.embeddings
Backprop from scratch (locked)
VAEs, GANs, diffusion, normalising flows
dl.generative
Backprop from scratch (locked)
bandits, Q-learning, policy gradients, MCTS/AlphaZero
dl.rl
Backprop from scratch (locked)
graph neural networks
dl.gnn
Backprop from scratch (locked)
audio, speech, time series
dl.audio-ts
Backprop from scratch (locked)
Transformers & LLMs
tokenisation, BPE
tf.tokenize
Transformer from scratch (locked)
attention, multi-head, positional encodings (RoPE)
tf.attention
Transformer from scratch (locked)
language modelling, n-gram → GPT, sampling/decoding
tf.lm
Transformer from scratch (locked)
training at scale: mixed precision, data/pipeline parallel, speedruns
tf.training
Transformer from scratch (locked)
KV cache, quantisation, batching, speculative decoding
tf.inference
Transformer from scratch (locked)
fine-tuning, LoRA, instruction tuning, RLHF/DPO
tf.finetune
Transformer from scratch (locked)
retrieval, RAG, rerankers
tf.retrieval
Transformer from scratch (locked)
LLM evaluation, benchmarks, contamination
tf.eval
Transformer from scratch (locked)
vision transformers, CLIP
tf.multimodal
Transformer from scratch (locked)
Interpretability
linear probes, world models
interp.probes
Interpretability basics (locked)
logit lens, tuned lens
interp.lens
Interpretability basics (locked)
circuits, attention patterns, induction heads, grokking
interp.circuits
Interpretability basics (locked)
sparse autoencoders, superposition, features
interp.sae
Interpretability basics (locked)
activation steering, ablations, causal interventions
interp.steering
Interpretability basics (locked)
ML systems
performance kernels, SIMD, GPU programming, roofline
sys.kernels
Linear algebra fundamentals (in progress)
memory layout, caching, arenas, file formats (GGUF/safetensors)
sys.memory
Transformer from scratch (locked)
threads, distributed training
sys.parallel
Transformer from scratch (locked)
graph capture, op fusion, JIT
sys.compilers
Transformer from scratch (locked)
model serving, latency, throughput
sys.serving
Transformer from scratch (locked)