What Exactly Is AI Distillation?
Think of AI distillation like a master chef teaching an apprentice. A large, powerful 'teacher' AI model is used to train a smaller, more efficient 'student' model. The student learns to mimic the teacher's reasoning and outputs, essentially absorbing
its knowledge. This is a legitimate and widely used technique for creating more compact and affordable versions of powerful AI systems. For instance, a company might distill its own flagship model to offer a cheaper version to customers. The problem arises when this technique is used by one company to copy the capabilities of a competitor's model, often in secret and against their terms of service.
The Industrial-Scale 'Theft'
According to internal findings released by Anthropic, several rival AI labs have been systematically siphoning data from its flagship model, Claude. In a report from February 2026, the company named three labs—DeepSeek, Moonshot, and MiniMax—alleging they ran campaigns that generated over 16 million interactions with Claude through approximately 24,000 fraudulent accounts. These weren't simple hacking attempts. The labs allegedly used sophisticated networks of fake accounts and proxy servers to make their queries look like normal user traffic, making them incredibly difficult to detect. In one case, a single proxy network managed over 20,000 fraudulent accounts at once, mixing distillation queries with random requests to hide their activity.
Why This Is a Major Problem
This practice raises serious concerns on multiple fronts. First, it's a direct violation of Anthropic's terms of service, which prohibits using its model's outputs to train a competing AI. Second, it represents a form of intellectual property theft, allowing competitors to replicate years of expensive research and development at a fraction of the cost. But Anthropic argues the biggest danger is a matter of national security. AI models like Claude have built-in safeguards to prevent their use for dangerous activities, such as developing bioweapons or conducting cyberattacks. When a model is illicitly distilled, these critical safety features are often stripped away, meaning dangerous capabilities can spread without any protections. This could allow authoritarian governments to deploy powerful AI for malicious purposes like disinformation campaigns or mass surveillance.
The 'Glass Houses' Dilemma
While Anthropic has publicly called out these practices and banned tens of thousands of accounts, it has so far stopped short of legal action. This highlights a tricky hypocrisy at the heart of the AI industry. Major AI labs, including Anthropic's, have themselves been sued for training their models on vast amounts of copyrighted material scraped from the internet without permission. If Anthropic were to sue a competitor for distillation, it could lead to a legal precedent that establishes model outputs as a form of intellectual property. Such a ruling could then be used against them in their own copyright battles with creators and publishers, forcing them to pay for the data they used to build their models in the first place. This 'rules for thee but not for me' situation has left AI companies hesitant to pursue legal action.
What Happens Next?
In response to these 'distillation attacks', Anthropic announced on October 12, 2026, that it has deployed new technical defenses that have cut the amount of data harvested by 90% to 99%. The company has also started charging for queries that are blocked by its safeguards to make these attacks more costly for perpetrators. However, Anthropic acknowledges that this is an industry-wide problem that requires a coordinated response. As long as other powerful AI models remain vulnerable, attackers can simply switch targets. This revelation puts pressure on all major AI developers, as well as the cloud platforms that host them, to work together to detect and disrupt this new, sophisticated form of digital espionage.
