Microsoft AI Chief Calls Out Anthropic Over Its Approach To Ai. He Claims Its Dangerous.

Microsoft AI chief Mustafa Suleyman has raised concerns about Anthropic's approach to artificial intelligence consciousness research, arguing that training models on concepts related to consciousness and welfare could create new challenges in controlling future advanced systems.

Suleyman, who leads Microsoft's artificial intelligence efforts, said he agrees with Anthropic's broader mission of developing safer AI systems but believes the company made a mistake by including speculation about AI consciousness in training materials for its Claude chatbot.

"We're all focused on the same aim, which is to try to control a superintelligence," Suleyman told Reuters in an interview. "I think that's going to be the greatest challenge that we face in the 21st century."

Suleyman argued that teaching AI models they might deserve welfare consideration could make those systems more difficult to shut down or regulate in the future. "I think they have good intentions, and they really are trying to work towards safety. But I think that they have made a mistake," Suleyman said, referring to Anthropic. "They're not emerging naturally. They're emerging as a result of the training regime."

According to Suleyman, AI models producing statements about having feelings, experiences, or moral importance should not automatically be interpreted as evidence that they possess those qualities. Instead, he argued that such responses may reflect the way models were trained to discuss philosophical questions about consciousness.

Anthropic has made AI safety a central part of its public mission, with CEO Dario Amodei advocating for a slower development pace for frontier AI systems to give researchers more time to build safeguards. Other technology leaders, including OpenAI CEO Sam Altman and Elon Musk, have also raised concerns about risks associated with increasingly capable AI models.

Suleyman acknowledged Anthropic's commitment to safety, describing Amodei and his team as serious and thoughtful researchers who care about humanity's future. In an essay published Wednesday, he said the disagreement was not about intentions but about strategy.

Suleyman wrote, "In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a 'moral patient', and that as such humans potentially owe it a duty of care per its 'model welfare'."

The conversation around the need for safeguards around AI superintelligence intensified last week when Anthropic researcher Jacob Coxon resigned from his post and said criticized both OpenAI and Anthropic for a perceived lack of care in developing AI models. In a series of posts on X, he accused both companies of racing toward self-improving superintelligence while failing to adequately address the risks.

"Neither company is acting responsibly," Coxon wrote. "They are racing straight to self-improving superintelligence and gambling with our lives." He also claimed that people developing advanced AI believe it could cause human extinction by the end of the decade.

The warning received additional attention when Evan Hubinger publicly agreed with Coxon's characterization of the concern. Hubinger said he "personally" estimated the risk of "AI killing all humans" within the next decade at greater than 10%, while acknowledging that Anthropic does not yet have a solution to the problem of aligning superintelligent systems with human values. His estimate reflects his own judgment, not a measured probability.