← All Base Form ML projects
BL-30NOT STARTEDBASE FORM · INTERPRETABILITYMLPYTHON

Feature finder and steering wheel

Train a sparse autoencoder on your GPT's activations, name the features it finds, and steer what the model writes.

EST.12 h
REWARD+100 XP
AFTERAnthropic

ML-ARENA BENCHMARKS

YOUR NUMBER VS THE INDUSTRY'S
METRICINDUSTRYMINE
reconstruction error and sparsity; steering success
GOAL · comparable error at equal sparsity; steering works on 8/10 prompts
SAELens defaults, same activations—
no results yet
YOUR RESULTS

Your recorded results appear here.

Brief

In BL-29 you read single heads and probed for properties you chose in advance. This project lets the model tell you what it represents. The difficulty is superposition: a residual stream of width dd seems to hold many more than dd concepts, packed as directions that overlap a little. Single neurons then mean several things at once, so looking at neurons one by one is confusing.

A sparse autoencoder untangles this. It maps an activation xx to a much wider vector of features ff, often 8 times dd, and rebuilds xx from them:

f=ReLU(We(x−bd)+be),x^=Wdf+bdf = \text{ReLU}(W_e(x - b_d) + b_e), \qquad \hat x = W_d f + b_d

trained on ∥x−x^∥2+λ∥f∥1\lVert x - \hat x \rVert^2 + \lambda \lVert f \rVert_1. The L1 term pushes most features to exactly zero, so each input is explained by a few active ones. Each column of WdW_d is one feature's direction in the residual stream. With luck, many of them mean one thing, such as "inside dialogue" or "a capital letter after a full stop".

Then you test whether a feature is real by changing the model with it. Steering adds a multiple of a feature's direction to the residual stream while the model writes. Removing it is an ablation. If adding the "dialogue" feature makes the model start writing quotes, and removing it makes them rarer, you've found a cause, not just a correlation. This is the last project built on your BL-21 model, and the hooks from BL-29 do most of the plumbing.

VISUALFig. 1
x WIDTH d encode: ReLU(W_e(x − b_d) + b_e) f WIDTH 8d · ONLY A FEW NON-ZERO decode: W_d f + b_d x̂ WIDTH d x̂ adds up the lit features' decoder directions, each scaled by its value, plus b_d.
A sparse autoencoder: an activation of width d becomes a much wider feature vector with only a few non-zero entries, then is rebuilt from those features' directions. Why it's here: Shows the shapes and the sparsity you are training for before you write the loss in objective 2.
HINT · OBJECTIVE 2
Stuck? Hints stay hidden until you ask.
HINT · OBJECTIVE 6
Stuck? Hints stay hidden until you ask.

DATA · your BL-21 model

Objectives

0 / 6 CLEARED

Read-only. Log in to update progress.

VICTORY CONDITIONS

  1. At the same L0, your SAE's FVU is within 10% of the SAELens run on the same activations, and steering works on at least 8 of 10 prompts. Record it with `python tools/mlarena.py record BL-30 "FVU <x> at L0 <k>, steer <n>/10"`.
  2. The splice test raises the model's loss much less than mean replacement does, and you report both numbers.
  3. The README lists 20 named features with their top contexts, and the steering rule was written before the 10 prompts were run.

Side quests

Train SAEs on two layers and match features across them by decoder cosine similarity.
Replace L1 with a TopK activation that keeps exactly k features, and compare the FVU against L0 curves.

Terms in plain words

Superposition
A network storing more features than it has dimensions, as directions that are nearly but not exactly at right angles.
Sparse autoencoder (SAE)
A wide one-layer autoencoder trained to rebuild activations from a few active features, with a penalty on how many fire.
L0
The average number of features that are non-zero for one input. Lower means sparser and usually easier to read.
Fraction of variance unexplained (FVU)
Reconstruction error divided by the activations' variance. 0 is perfect; 1 is no better than predicting the mean.
Activation steering
Adding a chosen direction to the residual stream while the model runs, to push its output toward a concept.