Concept
Mechanistic Interpretability
The field of research that tries to understand what is literally happening inside AI models - tracing computations to find where and how specific knowledge, beliefs, and capabilities are stored and used.
Added May 18, 2026
Most AI research treats models as black boxes: you observe inputs and outputs, measure performance, and optimise accordingly. Mechanistic interpretability takes the opposite approach. It opens the black box and tries to understand the actual computation - which neurons activate for which concepts, how information flows between layers, where factual knowledge is stored, and how the model combines information to produce answers.
The field has produced a growing set of findings that are surprising and important. Circuits research, pioneered by teams at Anthropic, identified specific algorithms implemented by small groups of neurons and attention heads. Induction heads - pairs of attention heads that together implement a specific sequence repetition mechanism - were one of the first such circuits identified. Superposition is another key finding: individual neurons do not represent single clean concepts; they represent many overlapping concepts simultaneously, requiring the model to disentangle them through the structure of activation patterns.
Factual associations are stored in specific components. Research has shown that factual recall - retrieving the capital of France, or who wrote Hamlet - happens primarily in the multi-layer perceptron layers of the transformer, in particular weight matrices that act as key-value memories. Attention layers handle where information comes from in the context; MLP layers handle what factual information to retrieve.
The logit lens and activation patching are two of the key experimental methods. The logit lens reads out the model's implicit prediction at intermediate layers. Activation patching systematically replaces activations at specific positions and layers with those from different inputs to identify which components causally contribute to which aspects of the output.
Mechanistic interpretability has practical motivations beyond scientific curiosity. If you can identify where a specific undesirable behaviour is implemented in a model - where the lying is happening, where the harmful reasoning lives - you have a principled path to targeted interventions rather than brute-force training modifications.
Analogy
The difference between a cardiologist who reads an EKG printout (behavioural measurement) and one who physically examines the heart tissue under a microscope (mechanistic investigation). The EKG tells you what the heart is doing. The microscope tells you why, at the level of cells and structure. Mechanistic interpretability is neuroscience applied to artificial neural networks.
Real-world example
Anthropic's research team traced the 'indirect object identification' circuit in GPT-2 - the mechanism by which the model identifies the correct indirect object in sentences like 'John gave Mary the book; she thanked him.' They identified specific attention heads that copy the indirect object, specific MLP neurons that handle the subject-object distinction, and the specific sequence of operations that produces the correct answer. This level of mechanistic understanding had previously been considered unachievable.
Why it matters
Mechanistic interpretability matters because it provides the foundation for AI safety that does not rely solely on behavioural testing. If deceptive alignment or subtle goal misgeneralisation are real concerns for future AI systems, we need the ability to verify what those systems are actually computing - not just what outputs they produce. Mechanistic interpretability is the research programme developing those verification tools.
In the news
No recent coverage - check back later.
Related concepts