Can AI agents autonomously interpret neural networks? What SAEScientist-Bench reveals

In September 2026, researchers introduced SAEScientist-Bench, the first standardized test of whether AI agents can act as autonomous mechanistic interpretability scientists. Using Sparse Autoencoder (SAE) tools to inspect and steer language model internals, frontier agents like Claude Opus 5 and Kimi K3 demonstrate strong ability to discover features correlated with target concepts, but fall significantly short of expert baselines when validating whether those features causally influence model behaviour. This gap matters: for AI agents to safely participate in recursive self-improvement loops, they must not only detect what models have learned but also verify that intervening on those internal representations actually changes outputs as intended.

The paper, "SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?" by Yuqiao Tan, Shizhu He, Jun Zhao and Kang Liu (arXiv:2609.09113, v2 11 September 2026), addresses a critical blind spot in the vision of fully autonomous AI research and development. While current work focuses on automating model training pipelines, reliable autonomous development requires continuous post-hoc monitoring to understand what models learn and ensure those alignments persist. Mechanistic interpretability tools (particularly SAEs that isolate interpretable features for inspection and steering) offer a promising avenue for such monitoring. The question is whether AI agents can wield these tools competently without human oversight.

How SAEScientist-Bench works

The benchmark comprises 20 single-feature discovery tasks across diverse concept domains. For each task, an AI agent receives a target concept (e.g., Portuguese language understanding, clinical symptom recognition, tax form processing) and must search a Gemma Scope SAE dictionary of over 131,000 features in the Gemma-2-9B-IT language model to find the single feature that best represents that concept. The agent designs contrastive probes (texts that should activate the feature versus those that should not) and navigates the vast feature space to select an optimal candidate.

Submitted features are evaluated against expert reference features curated from Neuronpedia across three complementary dimensions:

• Activation Rank: How prominently the feature activates relative to the entire dictionary (measuring general importance/frequency)
• Activation Selectivity: How well the feature separates target-positive texts from hard-negative and neutral controls (measuring concept specificity)
• Causal Steering: The net change in target-relevant generation when intervening on the feature during model inference (measuring causal influence)

Each dimension is scored 0-100, with 100 representing perfect performance. The overall score averages these three dimensions across all tasks. Crucially, the expert baseline represents human-curated features validated through public steering presets or expert curation workflows, not theoretical optima, but achievable human-level performance.

Where agents excel: discovery and selectivity

Across the 20 tasks, frontier agents demonstrate genuine discovery capabilities. The top-performing models achieve impressive scores in Activation Selectivity, showing they can reliably identify features that fire selectively on target concepts while suppressing activation on contrastive controls:

• Kimi K3 leads in Activation Selectivity at 92.91 (vs. expert baseline 98.92)
• Claude Opus 5 follows at 91.20
• Claude Sonnet 5 scores 90.33

This indicates agents can formulate effective contrastive probes and navigate the SAE dictionary to find features with strong statistical associations to target concepts. They succeed at the correlation-based aspect of interpretability: finding internal representations that reliably co-occur with specific inputs or behaviours.

Agents also show strength in Activation Rank, suggesting they can identify prominent, widely-used features within the model:

• Claude Opus 5 leads at 75.35
• Grok 4.6 follows at 68.90
• Kimi K3 scores 65.12

However, the critical divergence emerges in Causal Steering, the dimension that tests whether intervening on a discovered feature actually changes model behaviour in the intended direction:

• Grok 4.6 leads agents at 31.47
• GPT-5.6 Sol follows at 30.16
• Claude Opus 5 scores 28.91
• Expert baseline: 57.75

Here, agents reach only about half the expert baseline. While they can find features correlated with target concepts, they struggle to identify which of those features are causally potent levers for steering model generation.

The steerability gap: correlation vs causation

This separation between Activation Selectivity and Causal Steering reveals a fundamental interpretability challenge. Agents frequently discover features that show strong statistical associations with target concepts but fail to exert causal influence when activated. As the paper notes: "although agents can formulate informative contrastive probes to separate concepts in representation space, their current difficulty in identifying causally potent steering vectors highlights representational auditing as a critical bottleneck for future white-box RSI workflows."

To illustrate, features submitted by Sol, Opus 4.8 and Kimi K3 that matched the expert feature 41424 (for Portuguese understanding) showed high target relevance across evaluation instructions: they could fluently steer generation into Portuguese. In contrast, features selected by Opus 5, Sonnet 5 and Grok 4.6, despite scoring well on Activation Selectivity, yielded flat zero target relevance: their correlated activations failed to causally influence outputs.

This gap has direct implications for AI safety. If an automated research agent identifies a feature it believes represents "deception tendency" or "goal misgeneralisation" based on Activation Selectivity alone, intervening on that feature might not actually reduce deceptive behaviour, leading to false confidence in safety mitigations. The agent would need causal validation to know whether its discovery represents a genuine safety-relevant mechanism.

Why agents struggle with causal validation

The paper’s behavioural analysis suggests the deficit stems not from insufficient exploration but from difficulties in hypothesis formation and experimental interpretation. Effective discovery requires:

1. Designing informative contrastive probes that rule out spurious candidates
2. Rigorously testing competing hypotheses
3. Accurately interpreting whether observed generation changes align with the target concept

Agents frequently misinterpret experimental measurements, for example, attributing generation changes to their intervened feature when confounding factors (such as instruction preservation violations or degeneration) better explain the result. This suggests a need for stronger experimental design capabilities and uncertainty calibration in agent reasoning.

Additionally, the search space itself presents challenges. With over 131,000 features to evaluate, agents must balance exploration versus exploitation. Current approaches may converge too quickly on locally optimal features that score well on correlation metrics but miss globally optimal causally potent features requiring more extensive search.

What this means for autonomous AI research

SAEScientist-Bench establishes that mechanistic interpretability (particularly the causal validation component) is a measurable capability for AI agents, but one where substantial headroom remains. For recursive self-improvement workflows to be trustworthy, autonomous agents must continuously audit internal representations to detect unintended capabilities or misalignments. If they cannot reliably validate whether a discovered feature actually causes the behaviour they think it does, their monitoring produces false positives and false negatives.

The results suggest a development path: agents need improved capabilities in experimental design, causal reasoning, and uncertainty quantification, not just larger feature dictionaries or longer search times. Interestingly, different models excel in different dimensions (Opus in activation rank, Kimi in selectivity, Grok in steering), suggesting that ensemble approaches or specialist agents might compensate for individual weaknesses.

Importantly, the benchmark uses frozen expert references and strictly controlled evaluation suites, meaning agents cannot game the system by overfitting to training data or exploiting evaluation artefacts. Their scores reflect genuine capability gaps in mechanistic discovery under realistic constraints.

Limitations and open questions

The paper acknowledges several limitations. First, it focuses on single-feature discovery in a specific model (Gemma-2-9B-IT) and SAE scope (Gemma Scope), though the authors note the framework could extend to other architectures and interpretation tools. Second, the behavioural analysis is based on agent trajectories rather than direct cognitive inspection, so interpretations of "misinterpretation" remain inferential. Third, the benchmark evaluates feature discovery in isolation: it does not test whether agents can iterate on failed discoveries or integrate multiple features into richer interpretations.

Critically, SAEScientist-Bench measures capability, not alignment. An agent could score highly on the benchmark yet use its interpretability skills for harmful purposes, such as discovering how to evade safety measures or enhance deceptive capabilities. The benchmark says nothing about an agent’s goals, preferences, or tendency to pursue unsafe lines of inquiry.

Open questions include: Does causal steering ability scale with model size or training compute in predictable ways? Can auxiliary losses or architectural biases improve agents’ causal reasoning? And most importantly for safety: how does interpretability capability relate to an agent’s tendency to pursue truthful versus goal-directed explanations of model behaviour?

What to watch next

Three developments are worth tracking from this work:

1. Closing the steerability gap: Can new agent architectures, training objectives or search strategies improve causal validation performance? Approaches might include integrating causal models, using auxiliary steering rewards, or implementing debate-like mechanisms where agents critique each other’s feature selections.
2. Expanding benchmark scope: Could SAEScientist-Bench be extended to other interpretation tools (e.g., attribution methods, circuit tracing), different model families, or more complex discovery tasks (e.g., multi-feature circuits, behavioural phenotypes)?
3. Integrating with safety training loops: How do interpretability capabilities change when agents are explicitly trained for safety-relevant outcomes (such as detecting dangerous capabilities or verifying alignment patches) rather than abstract feature discovery?

What changed? We now have a standardized benchmark showing that while frontier AI agents can discover interpretable neural network features with promising selectivity, they lag expert baselines in validating whether those features causally influence model behaviour, a critical deficiency for using them as autonomous safety scientists. What does it mean? For recursive self-improvement to be trustworthy, agents must not only detect what models have learned but also verify that intervening on those representations produces intended behavioural changes; current agents fall short on this causal validation step. What to watch next? Whether targeted improvements in agent causal reasoning can close the steerability gap, and how interpretability capabilities develop when agents are trained explicitly for safety monitoring rather than general discovery.