New to this? Read it in simple words
- Jacob Coxon says he left Anthropic because leading AI labs are moving too fast. He worries about AI that can help design a better version of itself.
- Coxon believes such AI could create an extreme risk to human life before 2030. Evan Hubinger, who leads part of Anthropic’s alignment work, agreed with his main concern.
- Coxon’s exit shows a deep disagreement inside leading AI labs. Expert beliefs like his can still guide safety tests and emergency plans.
- These are personal beliefs, not forecasts that can be tested today. Many experts would choose a different chance or date.
- AI lab
- A company or research group that builds advanced AI systems.
- Self-improving AI
- AI that can help design or build a better version of itself.
- Alignment
- Work to make AI systems follow human goals and limits.
Researcher Jacob Coxon says he left Anthropic because leading AI labs are moving too quickly toward systems that may improve themselves. Other safety researchers share his concern. These are serious expert beliefs, but they are not measured forecasts of what will happen.
The warning came from inside leading labs
Coxon worked on model training at OpenAI and Anthropic for about three years in total, according to his public account. He says he resigned because the companies are racing toward much more powerful AI without enough care.
He is especially worried about self-improving AI. This means a system that can help design a better version of itself. If improvement becomes very fast, people may find it hard to understand or control the result.
Coxon believes such systems could create an extreme risk to human life before 2030. This is his personal view. It is not a date that researchers can test today, and many experts would choose a different chance or time.
A researcher left Anthropic over AI risk. His risk estimate is a personal belief, while public decisions need tests, evidence, and clear uncertainty.
Another Anthropic researcher shares the concern
Evan Hubinger leads part of Anthropic’s work on alignment. Alignment means trying to make an AI system follow human goals and limits. Axios reports that he publicly agreed with the main concern in Coxon’s message.
Hubinger gave his own chance of an AI disaster within ten years as higher than 10%. This number shows how one expert thinks about uncertainty. It is not the result of a repeatable experiment or a shared scientific forecast.
Expert beliefs still matter when evidence is limited and possible harm is large. They can guide safety tests and emergency plans. However, news reports should not turn one person’s number into a fact about the future.
Governance needs more than private fear
AI companies can publish clearer safety limits, outside test results, and rules for stopping a launch. Staff also need safe ways to raise a concern. An exit from a company can be an important signal, but it is not a full safety review.
Governments and researchers need useful measures for dangerous skills, control failures, and the speed of model improvement. They should also report uncertainty. Clear tests can support better decisions than a debate based only on confidence or fear.
Coxon’s resignation matters because it shows a deep disagreement inside frontier AI work. The right response is neither to ignore him nor to accept every claim. It is to ask for evidence, prepare for serious risks, and keep public choices open.
Sources
Every fact in this story comes from the sources below. Open them to check our work.
- 1Research · September 9, 2026AI researcher quits Anthropic, warning of race toward dangerous systems Axios
- 2
- 3
- 4
Risk numbers and dates are personal beliefs, not predictions proven by data.
A researcher left Anthropic over AI risk. Silicon AI News, September 9, 2026. https://siliconainews.com/stories/anthropic-researcher-resignation
A researcher quit Anthropic warning that labs are racing toward risky self-improving AI, and an Anthropic alignment lead publicly agreed. How we rate the mood
The mood shows how the news frames the event for the tech and AI world: as an opportunity (6 and above) or as a loss (below 5). It is not our verdict on who is right.