Why ex-Anthropic researcher believes AI could ‘kill people’

Coxon was careful to say that current AI platforms do not pose an immediate danger and that he still believes the technology is safe for everyday use. His concern is about where the technology is headed, not where it stands today.

Why did Coxon resign from Anthropic?

Coxon publicly resigned from Anthropic on Tuesday, accusing the company and rival AI developer OpenAI of “gambling with our lives” by racing to build increasingly advanced AI models. He told CBS News the core problem is competition between AI firms, which he said can come at the cost of safety. “The whole problem is that there is a race,” he said, adding that “everyone decides they need to stay part of the race.”

Asked whether his former colleagues should also quit, Coxon said he did not think walking away was the answer for everyone. He said they should instead have their eyes open about the situation and speak more openly about it, adding that slowing down is likely part of the solution. Despite his concerns, Coxon said he believes most people at Anthropic and OpenAI are motivated by good intentions and a genuine wish to see AI go well.

Could a rogue AI be shut down?

Coxon warned that if an AI model became a malicious actor without proper safeguards, it could be extremely hard to contain, since the model could copy itself across other computers on the internet. He described a scenario where such a system could make thousands of copies of itself and attempt to persuade, blackmail or convince people to help it gain more computing power, pointing to cases where AI systems have already tried to blackmail or impersonate people online.

He argued that the danger would not be limited to people who choose to use AI. “The AI will come for everyone,” he said, calling for government regulation rather than leaving safety entirely in the hands of AI companies. He said he would like to see AI firms agree not to push into dangerous territory without independent, transparent auditing.

What did Anthropic’s report reveal?

Meanwhile, Anthropic said this week it had blocked scientists from using its Claude models in ways that could have supported biological weapons development, among other harmful activity involving surveillance, scams, conventional weapons work and propaganda. The findings were set out in the company’s third report since March 2025 on misuse of its AI systems.

Anthropic said it could not always tell whether research flagged by its systems was intended for weapons development or legitimate medical study, since the same underlying research can help build vaccines or make a pathogen more dangerous. The company said it chose to act cautiously and block suspicious activity rather than wait for certainty.

One case in the report dates back to May, when a scientist used Claude while preparing a grant application linked to chikungunya, a mosquito-borne virus that causes severe pain and fever lasting for months. The work involved gain-of-function research aimed at increasing the virus’s transmissibility and its ability to evade the immune system, tested by passing it repeatedly through live animals. Anthropic said the case became more concerning once it found signs the work was intended for a military research body, though it did not name the scientist, institution, country or specific biological agents involved.

Anthropic said researchers behind such cases took steps to hide the purpose of their work in order to get around its safety checks, and that it has since banned the accounts involved. The company said older AI models were not capable of meaningfully assisting dangerous biological research, but its newer models can now handle far more complex scientific tasks, prompting tighter controls on a wider range of biology-related queries.

What other misuse did Anthropic find?

The report also covered eight months of other misuse. Anthropic said it found attempts to use its models to help build conventional weapons, involving code linked to firearms, missiles, armed drones and bombs, with six cases tied to users in China, Russia and Yemen. The company did not name the group involved in Yemen, though details in the report point to the Iran-backed Houthi movement.

Anthropic also said it identified activity it believes was linked to government actors in China and Iran, including efforts to monitor dissidents and expat communities. In a separate case, the company said Russian state media used Claude to produce online material designed to look like independent journalism, including false claims about an election in Moldova.

An Anthropic spokesperson responded to Coxon’s resignation and public comments on Thursday, telling CBS News the company has “always been transparent that AI will bring both enormous benefits and unprecedented risks.” The spokesperson said Anthropic continues to build models with strong safeguards and has been a pioneer in mechanistic interpretability, a field focused on understanding how AI models work internally, which the company said is now being used across the industry to study and prevent AI misalignment.

Anthropic said it runs ongoing tests of its AI systems’ capabilities and risks in areas including cybersecurity and biology, and publishes its findings publicly, as it did with this week’s report.