Interpretability

Articles

• Interpretability · Part 1

  Opening the Black Box: What Interpretability Can Explain

  What interpretability is for, its main methods, what counts as an explanation, and the toolkit, tested by asking two methods where GPT-2 keeps facts.

• Interpretability · Part 2

  The Residual Stream and the Logit Lens

  Decode a model's half-finished predictions layer by layer with the logit lens, train a tuned lens on a laptop, and see where the simple version breaks.

• Interpretability · Part 3

  Linear Probes: Reading What a Model's Activations Encode

  Train one-matrix classifiers on GPT-2 small and Qwen2.5-0.5B to read part of speech and truth from every layer, then check whether the probe did the work.

• Interpretability · Part 4

  Attention Heads and Induction Heads, Measured in GPT-2

  Find GPT-2 small's previous-token and induction heads, test how they compose, and measure what knocking them out does to in-context learning.

• Interpretability · Part 5

  Activation Patching and Causal Tracing, Measured in GPT-2

  Swap one activation between two prompts to find where GPT-2 tracks names and recalls facts, and measure how corruption and metric choices change the answer.

• Interpretability · Part 6

  Superposition: How Models Store More Features Than Neurons

  Reproduce Toy Models of Superposition on a laptop: phase diagrams, pentagons and sticky fractions, then measure how mixed GPT-2 small's MLP neurons are.

• Interpretability · Part 7

  Sparse Autoencoders: Pulling Features Out of a Model

  Train a sparse autoencoder on GPT-2 small on a laptop, compare it with public ones, then explore and score Gemma Scope features on Gemma 2 2B.

• Interpretability · Part 8

  Circuits and Attribution Graphs, Traced on a Laptop GPU

  From hand-found circuits to attribution patching, transcoders and attribution graphs, run with circuit-tracer on Gemma 2 2B and tested by intervention.

• Interpretability · Part 9

  Steering With Activations: The Refusal Direction, Measured

  Find one direction that controls refusal in an open chat model, remove it and add it, measure the side effects, and compare SAE steering with a mean difference.

• Interpretability · Part 10

  What Training Builds Inside a Model, Snapshot by Snapshot

  Word counts, then word pairs, then induction heads in 300 steps: 49 snapshots of the From Scratch model show what forms when, and what set the timing.

• Interpretability · Part 11

  Interpretability for Safety: Lie Detectors, Sycophancy and Audits

  What looking inside a model can and can't check about its honesty: lie and sycophancy probes, model diffing and real audits, measured on Gemma 2 2B.

• Interpretability · Part 12

  Limits and Open Problems in Interpretability

  The series in review: every part's headline result, a robustness audit of three of them, the field's sharpest critiques, and the problems still open.