Interpretability · Part 6Interpretability
Superposition: How Models Store More Features Than Neurons
Reproduce Toy Models of Superposition on a laptop: phase diagrams, pentagons and sticky fractions, then measure how mixed GPT-2 small's MLP neurons are.