CAISI has stopped public AI model evaluations under a new executive order. The company now conducts all assessments in classified channels. This shift has sparked immediate concerns about transparency, innovation, and U.S. competitiveness in the AI sector.
Executive Order Takes Immediate Effect
The administration's AI executive order forced CAISI to cease public reporting of its model evaluations. The company complied without delay, ending the practice of sharing test results with researchers and developers. Previously, CAISI's public evaluations provided independent performance data that the industry relied on for benchmarking. Now those assessments happen behind closed doors with no public disclosure.
Transparency Vanishes From AI Development
The move directly hinders transparency in AI development by cutting off public access to evaluation data. Independent researchers and smaller firms can no longer verify model capabilities or limitations through CAISI's work. This lack of open information may slow collaborative progress as teams lose a key reference point for their own development. The AI community previously used these public results to identify strengths and weaknesses across different models.
Innovation and Competitiveness Under Pressure
U.S.-based AI firms face two immediate risks from this shift. The change may stifle innovation by removing a shared knowledge resource that accelerated improvements across the industry. Teams previously built on others' evaluation findings to refine their own models. Additionally, competitive disparities could emerge as international firms operating without similar restrictions gain access to more open benchmarking data. Domestic companies now work without the same visibility into performance standards that once leveled the playing field.
Industry Adapts to New Reality
AI developers are already adjusting to the absence of CAISI's public reports. Some firms are increasing internal testing budgets while others seek alternative data sources. The sudden closure of this evaluation channel leaves companies scrambling to fill the information gap. There's no indication when or if public access might resume as the classified system becomes the new norm. The administration hasn't provided any timeline for revisiting the policy, leaving the industry to navigate blind.
Classified evaluations are now fully implemented at CAISI.
