Anthropic says it has deployed new defenses that reduce the amount of training data attackers can extract from its Claude models through distillation by 90 to 99 percent. The company announced the measures as part of a broader effort to protect model integrity and its proprietary technology.
Distillation attacks involve querying a model repeatedly to collect its outputs, then using that data to train a competing system. The technique lets someone piggyback on another company's research without paying for it. Anthropic's new safeguards are meant to make that process far less effective.
How the attacks work
In a distillation attack, an adversary doesn't need access to the model's underlying code or weights. They just need the ability to send queries and record the responses. By running millions of prompts through a model, they can build a dataset that captures much of its behavior. That dataset then becomes training material for a rival model, which can mimic the original's capabilities at a fraction of the development cost.
For AI companies, this is both a security and a business problem. The outputs represent months of research and significant compute spending. If a competitor can harvest those outputs cheaply, the incentive to invest in frontier models shrinks. Anthropic's defenses aim to break that chain by making the harvested data less useful or harder to collect at scale.
What Anthropic changed
The company didn't detail the exact technical mechanisms, but it said the defenses are designed to detect and disrupt distillation attempts in real time. That could mean rate-limiting suspicious query patterns, adding noise to outputs, or watermarking responses so they can be traced. Anthropic claims the result is a 90 to 99 percent reduction in the data that attackers can successfully harvest.
The range is wide because effectiveness varies by attack method. A crude, high-volume scraping operation might be blocked almost entirely, while a more careful, low-and-slow approach could still get through. Anthropic says it will keep tuning the system as adversaries adapt.
This isn't the first time an AI company has talked about distillation. OpenAI and Google have both accused rivals of using their models' outputs to train competing systems. But public, quantified results are rare. Anthropic's numbers give a concrete benchmark for how well one set of defenses performs, at least by the company's own measurement.
Model theft sits at the intersection of security, intellectual property, and national competitiveness. Governments are starting to treat advanced AI models as strategic assets. If distillation can be curtailed, it strengthens the case for companies to keep investing in larger, more expensive models without fearing that their work will be copied overnight.
There's also a safety angle. If a malicious actor distills a model and strips out safety fine-tuning, they can create an unaligned version that's harder to monitor. Anthropic's defenses don't directly address that scenario, but by making distillation harder, they raise the cost for anyone trying it.
The announcement comes as Anthropic faces growing competition from open-weight models that are free to use and modify. Those models can be distilled without any API queries, so the new defenses don't apply to them. The company's approach is a bet that proprietary, API-based models will remain valuable enough that customers will pay for access, and that protecting them is worth the engineering effort.
Anthropic hasn't said whether it will publish a technical paper or open-source the defense tools. For now, the 90 to 99 percent figure is a claim that outside researchers can't easily verify. The company says it will continue to update the system as it learns more about how attackers operate.
Competitors are likely watching closely. If the defenses hold up, they could become a standard feature of commercial AI APIs. If they don't, the cat-and-mouse game between model providers and data harvesters will continue, with each side adjusting to the other's latest move.
