A new study from Lenz Research has found that artificial intelligence models disagree on 67% of fact-check claims, casting doubt on the reliability of automated verification systems. The findings, released without prior announcement, suggest that even advanced AI tools frequently contradict one another when tasked with determining truth. Researchers behind the study stress the need for diverse sources and human oversight in decision-making, especially in fast-moving areas like financial markets.
The scale of disagreement
The 67% figure is striking. It means that out of every three fact-check claims run through different AI models, two end up with conflicting verdicts. Lenz Research did not specify which models were tested or the nature of the claims, but the implication is clear: no single AI system can be trusted to sort fact from fiction on its own. The disagreement rate points to fundamental differences in how models are trained, the data they draw from, and their underlying logic.
In volatile markets, where rumors can move prices in seconds, the stakes are high. A trader relying on an AI fact-checker might act on a label that another model would reject. The study’s authors explicitly mention “volatile markets” as a context where diverse sources and human judgment become critical. Automated tools can process huge volumes, but they can’t resolve their own contradictions without a person in the loop.
What the study suggests
The recommendation from Lenz Research is straightforward: don’t lean on a single AI source. Instead, cross-check outputs, bring in human reviewers, and treat automated fact-checks as one input among many. This isn’t a call to abandon the technology—it’s a warning against overconfidence. The study doesn’t offer a fix for the disagreement problem, but it makes a strong case for keeping people involved.
The findings add to a growing pile of evidence that AI, for all its speed, still struggles with consistency. Lenz Research has not indicated whether it plans to follow up with a deeper analysis of why models disagree or how to align them. For now, the message is simple: when the machines can’t agree, the decision belongs to humans.
