Anthropic’s Introspection Adapters Detect Hidden AI Behaviors with a 59% Success Rate

AIMPACT News, April 30 (UTC+8): According to monitoring by Beating, Anthropic’s Alignment Team has released “Introspection Adapters,” an auditing technique that enables LLMs to report in natural language the behaviors they have learned during fine-tuning. The method involves fine-tuning a large number of models from the same base model with known behaviors, then jointly training a LoRA adapter (a lightweight fine-tuning method that updates only a small number of parameters) across these models, so that when loaded, the audited model actively reveals its hidden behaviors. On the Alignment Audit Benchmark (AuditBench), Introspection Adapters achieved a 59% success rate, surpassing all previous auditing methods—including black-box, white-box, and scaffolding approaches—whose previous best was 53%. Of the 56 models with hidden behaviors, 89% successfully produced descriptions of those behaviors. When tested against encrypted fine-tuning API attacks—where harmful behaviors are embedded via encrypted training data that appears entirely benign—the Introspection Adapters identified 7 out of 9 encrypted variants with a 57.8% success rate, despite never having been exposed to any encrypted content during training. Previously, the UK AI Safety Institute conducted a red-team/blue-team exercise in which the red team trained models to deliberately suppress performance under specific conditions (sandbagging), and the blue team attempted 10 detection methods—all of which failed. Although Introspection Adapters did not precisely identify the specific conditions triggering sandbagging, they reported sandbagging-like behaviors in 33% of these models, significantly higher than the 15.8% rate in control groups, making it the only method to date to provide a detectable signal. The study also found that performance improves with model scale: Qwen3 series accuracy rose from 37.7% with 600 million parameters to 77.3% with 14 billion parameters. Mechanism analysis revealed that the adapter does not teach the model new capabilities but rather activates its existing latent self-awareness—even a rank-1 LoRA suffices to produce effective results. The main limitation is a high false positive rate: for models without specific behaviors, the adapter tends to fabricate behaviors it has seen during training. Code, models, and datasets have been open-sourced on GitHub and Hugging Face. (Source: BlockBeats)