Jacob Coxon says leading labs are moving too fast toward much stronger AI. His concern deserves attention, but a personal risk estimate cannot tell us what will happen.
New to this? Read it in simple words
- Jacob Coxon says he left Anthropic because leading AI labs are moving too fast. He worries about AI that can help design a better version of itself.
- Coxon believes such AI could create an extreme risk to human life before 2030. Evan Hubinger, who leads part of Anthropic’s alignment work, agreed with his main concern.
- Coxon’s exit shows a deep disagreement inside leading AI labs. Expert beliefs like his can still guide safety tests and emergency plans.
- These are personal beliefs, not forecasts that can be tested today. Many experts would choose a different chance or date.
- AI lab
- A company or research group that builds advanced AI systems.
- Self-improving AI
- AI that can help design or build a better version of itself.
- Alignment
- Work to make AI systems follow human goals and limits.
The warning came from inside leading labs
Coxon worked on model training at OpenAI and Anthropic for about three years in total, according to his public account. He says he resigned because the companies are racing toward much more powerful AI without enough care.
He is especially worried about self-improving AI. This means a system that can help design a better version of itself. If improvement becomes very fast, people may find it hard to understand or control the result.
Coxon believes such systems could create an extreme risk to human life before 2030. This is his personal view. It is not a date that researchers can test today, and many experts would choose a different chance or time.
A researcher left Anthropic over AI risk. His risk estimate is a personal belief, while public decisions need tests, evidence, and clear uncertainty.
Another Anthropic researcher shares the concern
Evan Hubinger leads part of Anthropic’s work on alignment. Alignment means trying to make an AI system follow human goals and limits. Axios reports that he publicly agreed with the main concern in Coxon’s message.
Hubinger gave his own chance of an AI disaster within ten years as higher than 10%. This number shows how one expert thinks about uncertainty. It is not the result of a repeatable experiment or a shared scientific forecast.
Expert beliefs still matter when evidence is limited and possible harm is large. They can guide safety tests and emergency plans. However, news reports should not turn one person’s number into a fact about the future.
Governance needs more than private fear
AI companies can publish clearer safety limits, outside test results, and rules for stopping a launch. Staff also need safe ways to raise a concern. An exit from a company can be an important signal, but it is not a full safety review.
Governments and researchers need useful measures for dangerous skills, control failures, and the speed of model improvement. They should also report uncertainty. Clear tests can support better decisions than a debate based only on confidence or fear.
Coxon’s resignation matters because it shows a deep disagreement inside frontier AI work. The right response is neither to ignore him nor to accept every claim. It is to ask for evidence, prepare for serious risks, and keep public choices open.
Sources
Every fact in this story comes from the sources below. Open them to check our work.
- 1Research · September 9, 2026AI researcher quits Anthropic, warning of race toward dangerous systems Axios
- 2
- 3
- 4
We compared Axios, ABC News Australia, and El País. We treat risk numbers and dates as personal beliefs, not predictions proven by data. We explain the technical words and focus on what the disagreement means for public safety rules.