AI Is Doing More Of The Work Behind New AI Models. Anthropic Says These 3 Metrics Can Track The Shift.

Artificial intelligence is taking on a larger role in building the next generation of AI systems, with Anthropic now disclosing new measurements showing how extensively its Claude models are involved in the company's own research and engineering work.

Claude was leading 26% of Anthropic's measured AI research and development work as of August, up from less than 1% in February, while more than 90% involved AI at a level where it at least collaborated with humans, Anthropic said Thursday. None of the work measured by the company involved Claude operating fully autonomously without humans in the loop.

The figures form part of three metrics Anthropic developed to track the pace of frontier AI development: how much AI is involved in developing newer AI systems, how effectively companies oversee AI agents working inside their systems and how computing resources are divided between development and safety work. The company published its methodology alongside the results and encouraged other frontier developers to release comparable information.

For its first measurement, Anthropic created an R&D Automation Index that ranks work from no AI involvement through collaboration and AI-led tasks to full autonomy. Claude now "leads" 26% of the work measured, meaning it can complete most of a task from a high-level instruction while remaining under human supervision, CNBC reported.

Anthropic built the index by examining its model research and development work and grouping individual tasks into categories covering areas such as model training, reinforcement learning, evaluations and incident response. The company said the measurement is designed to show how much of the work involved in developing more capable models is increasingly being performed by existing models.

The second metric focuses on oversight as employees delegate more research and engineering work to AI agents that can operate for extended periods. About 30,000 agents were working at any given time on Anthropic's most-used internal research and engineering platform in August, with every action passing through an automated monitor before it was carried out.

After examining more than 1 billion decisions made by those agents during August, Anthropic found that 0.002%, or about one in every 47,000, were blocked by its online monitoring system. A separate offline system examines activity after it occurs and sends the highest-priority cases for human review, with about 50 escalated each week.

The third measurement examines how Anthropic allocates the computing power used for AI development. During a sample week from July 13 through July 20, roughly 6% of compute devoted to AI research and development went toward safety work, while the share increased to about 12% for AI-driven research and development, Anthropic said.

The company described those figures as conservative because computing work that contributed equally to safety and model capabilities was classified as capability work. It also said compute alone does not capture every safety investment because some research requires substantial employee time without using large amounts of computing power.

The disclosure comes days after Anthropic CEO Dario Amodei called for leading AI companies to coordinate on slowing the pace at which their most powerful models improve. OpenAI CEO Sam Altman, xAI's Elon Musk, Google DeepMind CEO Demis Hassabis and Microsoft CEO Satya Nadella subsequently backed versions of a more measured approach, marking a notable shift among executives whose companies have spent years competing to release increasingly capable systems, as International Business Times previously reported.

The companies have also begun discussing more concrete safety measures rather than simply slowing model releases. Musk suggested rival developers could test one another's models, while Amodei proposed independent evaluators embedded inside frontier AI companies, an industry-wide safety standard and an antitrust waiver allowing competitors to coordinate, IBTimes reported in a separate story. Anthropic said Thursday that it plans to give outside evaluators access to internal processes, systems and data comparable to what its own risk assessment teams can access.

Altman has separately said OpenAI is already slowing parts of its development process as models become more capable, describing the company's next systems as likely to be "sobering." He said OpenAI was increasing testing and taking more time before deployment, as IBTimes previously reported.

Not every major technology company has endorsed an industry-wide slowdown. Amazon said models should undergo rigorous testing and be released only when they are safe, but stopped short of backing coordinated pacing, Reuters reported. Nvidia and Meta have also distanced themselves from calls for a coordinated slowdown, arguing that individual developers should be responsible for assessing and managing their systems' risks.

European AI developers have pushed back more strongly, arguing that restrictions led by established U.S. companies could make it harder for competitors to close the technology gap. France's Mistral said some incumbents were using safety concerns to strengthen their existing market positions, while Hugging Face CEO Clement Delangue argued that developers should accelerate rather than slow down, another Reuters report said.