Anthropic Enhances Defense Against Distillation Attacks, Bans Tens of Thousands of Accounts

Dynamic Watch Beating AI News: Anthropic has posted that preliminary evidence shows new safeguards have reduced the number of Claude inference tokens extracted by distillation attackers by approximately 90% to 99%. The company states that attackers typically use networks of fake accounts to bulk-fetch model responses and unauthorized replicate its capabilities, with full inference logs being more valuable for training than final answers alone. New measures include restricting the transfer of inference logs across conversations, models, and accounts; deploying a two-stage classifier to identify and block extraction requests; and rate-limiting or banning accounts that repeatedly attempt extraction. Anthropic reports that the classifier’s false positive rate is below 0.1%, and tens of thousands of accounts associated with distillation have been banned—often within two hours of the start of related attacks. Additionally, since September 24, the company has resumed charging for requests blocked by security safeguards targeting distillation and other specific categories. Anthropic acknowledges that some developers may need to update their integrations, and the classifier may occasionally block legitimate requests. It remains unclear how long the effectiveness of these safeguards will last, as the company has not yet faced highly sophisticated or sustained circumvention attempts and must continue iterating on its defenses.