Singularity AI
Sparse autoencoders gave the field real traction. They did not give it the ability to certify a model as safe, and the distinction matters.
Mechanistic interpretability has had a genuinely good few years. The work on sparse autoencoders and feature decomposition produced something the field had been missing: a method for pulling human-legible structure out of activation space that survives contact with models people actually deploy.
It is worth being precise about what that does and does not establish.
What the results support
Features recovered by sparse decomposition are frequently interpretable, sometimes strikingly so. They can be intervened upon — steering behaviour by manipulating a feature direction works often enough to be useful. And the features found are not arbitrary: they recur across training runs and, to a degree, across architectures.
That is a real result. It suggests representations are more structured, and more accessible, than the "inscrutable matrices" framing implied.
What they do not support
Three gaps are load-bearing:
Coverage
Decomposition recovers some features. Reconstruction error is not zero, and the residual is not obviously benign — there is no argument that the uninterpreted remainder is safety-irrelevant. An explanation of part of a computation is not an explanation of the computation.
Faithfulness
A feature that correlates with a concept and one that causes behaviour related to that concept are different claims. Intervention experiments address this, but intervening on a direction and observing an output change does not establish that the direction played the role the label implies during normal operation.
Adversarial pressure
Interpretability results are typically established on natural inputs. A model under adversarial pressure is operating in a regime where the mapping from features to behaviour may simply differ. Very little work tests whether interpretations hold there — and safety cases care about precisely that regime.
Why this matters practically. If interpretability is offered as evidence in a safety case — "we inspected the model and found no deceptive circuitry" — then coverage and faithfulness are not academic caveats. They are the difference between evidence and reassurance.
The useful framing
Interpretability is currently a strong tool for generating hypotheses about model behaviour and a weak tool for certifying its absence. Those are both valuable; only one of them is what a regulator or a deployment decision actually needs, and the field's public communication does not always distinguish them.
The researchers doing this work are, in our reading, considerably more careful about these limits than the commentary around it. The overclaiming happens downstream.