Top AI Figures Are Warning Of a Potential ‘Intelligence Explosion.’ They Are Calling On Governments To Prepare.

Top figures in the field of AI published a paper on Monday warning that progress is accelerating to a point that could end in an "intelligence explosion."

In a paper published by the University of Cambridge, the group, comprised of "leading academics, senior scientists of frontier AI companies, and independent experts from civil society," called on governments to prepare for the possibility of what could be the "most consequential technological development in history."

The paper went on to note that AI is "rapidly automating AI research and development," recalling that in late August Anthropic noted that its systems were completing more than 25% of internal AI R&D work with only high-level supervision. That figure only stood at 1% five months earlier. OpenAI also reported that its systems routinely did such tasks. "Based on benchmark trends and other evidence, some experts think that AI R&D could be fully automated within the next few years," the document added.

Such a scenario could lead to an "intelligence explosion," compressing years of AI progress into months or less." This could then allow frontier development to expand exponentially, with a feedback loop "overcoming frictions such as diminishing returns to research labor and
hard-to-automate tasks."

The intelligence explosion would bring benefits but also dangers. The first category means that improvements could "rapidly obtain superintelligent AI systems that transform many sectors of society, bringing unprecedented breakthroughs in science and industry."

"However, at the same time they could "could arrive much faster than society can adapt to the resulting risks, such as AI-enabled pandemics, cyber attacks on critical infrastructure, and large-scale labor disruption."

Moreover, the paper warns, "humans may have little to no oversight over automated AI R&D, raising the risk that superintelligent systems irreversibly escape human control and act to marginalize humanity."

"Finally, an intelligence explosion could transform modest AI capability leads into decisive ones: even the prospect of a decisive lead could provoke international conflict, and an actual one could severely erode checks and balances within and between states, companies, and branches of government," the paper says.

Consequently, it calls on policymakers to get visibility into AI R&D automation, develop ways to "steer and constrain an intelligence explosion," and prepare "society for the impacts of an intelligence explosion."

U.S. policymakers do not appear to intend to slow down AI development at the moment. Attorney General Blanche said in an interview on Sunday that the Trump administration does not need new AI regulation and must ensure that it remains the leading AI power.

"We have prosecuted bad actors in the AI space and any other space from emerging technologies for decades" but "the U.S. has to be the leader in this technology. We can't fall second or third place, and so what do we do to make sure bad actors are punished? We have a ton of criminal laws that can address that already", Blanche told Fox News.