OpenAI And Anthropic Have Warned About AI Moving Too Fast. Now Its Employees Want The U.S. Government To Take Action

Employees from OpenAI and Anthropic are circulating a petition calling on the government to "deliberately pace" the development of AI to prevent it from moving too fast, according to a new report.

Bloomberg detailed that the letter expresses support for an international effort on the matter, warning there is a "real risk" that the technology advances faster than people can "understand or control."

"The US government should support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," reads a passage of the document.

Companies have made such calls over the past months as new, more advanced models continue to be released. Dario Amodei, co‑founder and CEO of Anthropic, published a policy essay last month confronting what he describes as a deepening mismatch between the breakneck pace of advanced artificial intelligence development and the slow pace of policymaking.

The central argument of the essay, titled "Policy on the AI Exponential," is that the technology's capability is scaling so rapidly that existing regulatory frameworks are too slow and too fragmented to meaningfully shape its safe deployment. "AI is progressing extremely fast — much faster than the policy process was built to handle," he wrote.

Amodei went on to emphasize that voluntary transparency measures, once sufficient when AI risks were theoretical, no longer provide meaningful protection. "Anthropic has long advocated for transparency requirements for frontier AI, because the risks weren't yet clear enough to regulate precisely. That is no longer sufficient."

He added that frontier AI models are approaching what he described as the "end of the exponential," a point where capability growth could rapidly outstrip human understanding and societal preparedness. "The most surprising thing has been the lack of public recognition of how close we are to the end of the exponential."

Elsewhere, respondents of a new study estimated that there is a 20 percent chance in the next five years that AI could do something "catastrophic."

Axios reported that MIT and the University of Queensland in Australia surveyed 272 experts on a variety of topics to gauge what the potential risk of AI was in the next five years. The study focused on 24 different risks.

"There are many AI risks," said Peter Slattery, a research scientist with MIT FutureTech and one of the study's co-authors. "One of the key things behind this work is trying to figure out who needs to do what differently, and in what sort of coordination."

The study defined catastrophic harm as having the "potential for more than 1 million deaths, more than $100 billion in financial losses, or comparable civilizational-scale intangible damages."

The top five concerns were:

• AI possessing dangerous capabilities (21.5 percent)
• Cyberattacks, weapon development or use, and mass harm (21 percent)
• Power centralization and unfair distribution of benefits (18 percent)
• Competitive dynamics (16.6 percent)
• False or misleading information (12.8 percent

"AI is uniquely well equipped to attack those aspects of society and cause harm," Slattery said. "Coding and hacking are some of the areas where we're seeing the fastest growth in AI capability."