More Than 100 AI Experts Sign a Letter Saying that OpenAI & Anthropic Need Independent AI Safety Evaluators

More than 100 artificial intelligence experts and safety evaluators are warning that outside oversight of companies such as Anthropic and OpenAI will be credible only if evaluators are given genuine independence, deep access to frontier systems and protection from retaliation when their findings are unfavorable.

The coalition, organized by the AI Evaluator Forum, published a public letter Friday laying out minimum conditions for third-party organizations embedded inside leading AI companies to assess increasingly powerful models and the risks surrounding their development.

The organization said that outside testing cannot be considered genuinely independent simply because a third party conducts it. Evaluators also need control over their methods and conclusions, adequate access to systems and protection from conflicts of interest.

"We're just trying to really demonstrate a shared common ground on basic principles and ensure that independent oversight can be a meaningful tool for managing AI risk broadly," Conrad Stosz, chair of the consortium, told CNBC. The letter's signatories include prominent AI researchers and experts affiliated with institutions including Stanford University, Johns Hopkins University and the nonprofit evaluator METR.

Their demands are significant because they would give outside evaluators access as "those available to senior internal company employees responsible for carrying out comparable risk assessments."

Under the proposed conditions, companies would allow evaluators to examine relevant systems, data, and tools, enter necessary physical spaces, and facilitate "candid and direct one-on-one communication with relevant staff." Evaluators would also retain editorial control over their findings and have the ability to communicate directly with company boards and other oversight bodies.

The push follows Anthropic CEO Dario Amodei's proposal to provide outside evaluators with "employee-like access" to frontier AI companies. OpenAI CEO Sam Altman, Elon Musk and Microsoft CEO Satya Nadella have publicly backed the concept, according to CNBC, though major questions remain over how it would work, including who would qualify as an evaluator and exactly how much access companies would provide.

Stosz said deeper access could allow independent experts to inspect sensitive internal data and unreleased AI systems, rather than limiting evaluations to models already available to customers.

The AI Evaluator Forum's existing AEF-1 standard already identifies five pillars for credible third-party testing: sufficient access and resources, minimized conflicts of interest, analytic autonomy, transparent methods and results, and protection of sensitive information.

The standard has begun appearing in independent assessments, including evaluations involving Anthropic, Google, Meta, and OpenAI.The new letter goes further by emphasizing protections against corporate interference or retaliation.

Evaluators, the signatories argue, "should not be owned or governed by frontier AI companies, should not have other significant commercial business with them, and should not accept any form of payment or other reward contingent on the evaluator's findings."

Companies should also limit "the scope of non-disclosure agreements" so evaluators can publish their conclusions, with narrowly defined redactions for issues such as intellectual property, customer information, privacy, security and public safety. They also want multiple independent organizations involved, reducing the possibility that oversight of increasingly consequential AI systems becomes concentrated in a single evaluator.