Anthropic Says It Blocked Claude Users From Researching Biological Weapons. They Sought To Evade Restrictions.

Anthropic said users have attempted to use its most advanced artificial intelligence systems for sensitive biological research that could potentially contribute to the development of biological weapons.

The company said Thursday that it disrupted several cases this year involving researchers who used its AI models, including Claude, for work involving potentially dangerous pathogens. Anthropic ultimately banned accounts connected to the activity.

But the company stopped short of accusing the scientists of trying to build biological weapons. Much of the biological knowledge that could be used to engineer a dangerous pathogen is also essential to legitimate medical research. Anthropic said it could not determine the researchers' ultimate intentions, but chose to intervene because of the potential consequences if the work proved malicious.

"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'" Jacob Klein, Anthropic's head of threat intelligence, said in an interview with the New York Times. "It's an incredibly nuanced situation." The cases were included in Anthropic's latest threat intelligence report, which documents misuse of its AI systems during the past eight months.

Among the most concerning examples was a scientist who approached Claude in May for help preparing a grant application involving the chikungunya virus, a mosquito-borne infection that can cause severe and sometimes prolonged joint pain.

The proposed work involved gain-of-function research, in which scientists can alter a pathogen to study how changes affect characteristics such as transmission or severity. Such experiments can have legitimate scientific purposes, including improving understanding of viruses and developing vaccines. But they can also produce more dangerous versions of pathogens.

In this case, Anthropic said the scientist wanted to engineer mutations that would make the chikungunya virus more harmful as it repeatedly infected live animals. The setting raised another red flag.

Anthropic determined that the proposed research was intended to take place at a military research institution. "What we don't know is if the research was meant to be weaponized," Klein said. "But a military institution doing gain-of-function research is concerning."

Anthropic said researchers connected to the biological cases also circumvented restrictions designed to prevent access from blocked regions and attempted to "obfuscate the purpose of their research to evade our safeguards." Those behaviors contributed to the company's decision to terminate the associated accounts.

The incidents are particularly significant because Anthropic says the capabilities of its models have changed. Evaluations of its older AI systems last year found they were not capable of providing meaningful assistance for dangerous biological research.

Today's models are considerably more powerful and better at conducting complicated scientific work, according to the company, prompting Anthropic to introduce stronger safeguards around biological queries.

Outside biosecurity experts who reviewed the findings before publication said the cases show why those capabilities deserve greater scrutiny. Susan Monarez, a microbiologist and public health expert who previously oversaw a review of U.S. biosecurity preparedness, told the New York Times that the cases offered real-world evidence that potentially malicious actors are working "to improve their chances of building biological pathogens that could cause significant harm."

Andrew Weber, a senior fellow at the Council on Strategic Risks and a former assistant secretary of defense for nuclear, chemical and biological defense programs, said the findings were "chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities"

Anthropic's report identified other forms of abuse as well, including suspected government-linked surveillance operations, Russian propaganda efforts and attempts to use Claude in the development of conventional weapons technology.