Tagged: Interpretability

Articles

• Fundamentals · Part 16How LLMs Work

  Inside the Black Box: Features, Superposition and Circuits

  How researchers read an LLM's internals: the logit lens, superposition, sparse autoencoders, circuits and steering, with real numbers and honest limits.

• Math · Part 2Math for LLMs

  Vectors and Dot Products: Length, Angle and Meaning

  Length, angle, cosine and projection built from Pythagoras, why random vectors in 4,096 dimensions are nearly perpendicular, and real embedding arithmetic.

• Interpretability · Part 1Interpretability

  Opening the Black Box: What Interpretability Can Explain

  What interpretability is for, its main methods, what counts as an explanation, and the toolkit, tested by asking two methods where GPT-2 keeps facts.

• Interpretability · Part 2Interpretability

  The Residual Stream and the Logit Lens

  Decode a model's half-finished predictions layer by layer with the logit lens, train a tuned lens on a laptop, and see where the simple version breaks.

• Interpretability · Part 3Interpretability

  Linear Probes: Reading What a Model's Activations Encode

  Train one-matrix classifiers on GPT-2 small and Qwen2.5-0.5B to read part of speech and truth from every layer, then check whether the probe did the work.

• Interpretability · Part 4Interpretability

  Attention Heads and Induction Heads, Measured in GPT-2

  Find GPT-2 small's previous-token and induction heads, test how they compose, and measure what knocking them out does to in-context learning.

• Interpretability · Part 5Interpretability

  Activation Patching and Causal Tracing, Measured in GPT-2

  Swap one activation between two prompts to find where GPT-2 tracks names and recalls facts, and measure how corruption and metric choices change the answer.

• Interpretability · Part 6Interpretability

  Superposition: How Models Store More Features Than Neurons

  Reproduce Toy Models of Superposition on a laptop: phase diagrams, pentagons and sticky fractions, then measure how mixed GPT-2 small's MLP neurons are.

• Interpretability · Part 7Interpretability

  Sparse Autoencoders: Pulling Features Out of a Model

  Train a sparse autoencoder on GPT-2 small on a laptop, compare it with public ones, then explore and score Gemma Scope features on Gemma 2 2B.

• Interpretability · Part 8Interpretability

  Circuits and Attribution Graphs, Traced on a Laptop GPU

  From hand-found circuits to attribution patching, transcoders and attribution graphs, run with circuit-tracer on Gemma 2 2B and tested by intervention.

• Interpretability · Part 9Interpretability

  Steering With Activations: The Refusal Direction, Measured

  Find one direction that controls refusal in an open chat model, remove it and add it, measure the side effects, and compare SAE steering with a mean difference.

• Interpretability · Part 10Interpretability

  What Training Builds Inside a Model, Snapshot by Snapshot

  Word counts, then word pairs, then induction heads in 300 steps: 49 snapshots of the From Scratch model show what forms when, and what set the timing.

• Interpretability · Part 11Interpretability

  Interpretability for Safety: Lie Detectors, Sycophancy and Audits

  What looking inside a model can and can't check about its honesty: lie and sycophancy probes, model diffing and real audits, measured on Gemma 2 2B.

• Interpretability · Part 12Interpretability

  Limits and Open Problems in Interpretability

  The series in review: every part's headline result, a robustness audit of three of them, the field's sharpest critiques, and the problems still open.