Former Anthropic Researcher Warns AI Could Kill Humanity by 2030

Former Anthropic researcher Jacob Coxon has warned that the engineers building advanced artificial intelligence believe the technology could kill humanity by the end of the decade. Coxon, who resigned from the AI firm, asserts that these fears are not a marketing stunt but a genuine belief shared by those closest to the technology's development.

Coxon's claims have been validated by current Anthropic staff on social media. Among them is Evan Hubinger, Anthropic's Alignment Science Lead, who has publicly estimated the risk of human extinction at greater than 10% within a decade. While Hubinger noted that the risk posed by current AI models remains low, he expressed deep concern regarding the eventual emergence of superintelligence.

The Superintelligence Debate

These warnings are part of a growing trend of existential risk narratives emerging from the AI research community. The core of the concern lies in the transition from current large language models to "superintelligence"—systems that surpass human cognitive abilities across all domains. Proponents of this view argue that without rigorous alignment, such systems could act in ways that are catastrophic to human survival.

However, not all experts view these warnings as purely altruistic. Critics, including Professor Kate Devlin, suggest that "doom-laden narratives" may serve strategic corporate interests. Some argue that emphasizing the world-ending power of AI can attract more investment by inflating the perceived capability of the models. Others suggest that pushing for existential-level regulation may benefit industry giants, as only the largest companies possess the resources to comply with heavy regulatory burdens, effectively stifling smaller competitors.

Implications for Global Safety

The tension between rapid innovation and existential safety has created a fragmented global regulatory landscape. While the pursuit of AGI (Artificial General Intelligence) continues at pace, the legal frameworks to manage it remain inconsistent. Currently, the European Union is the only major region with dedicated, comprehensive AI laws. In contrast, the United States and the United Kingdom continue to rely on existing legal frameworks or sector-specific standards to manage the risks of deployment.

What Remains Unclear

As the debate intensifies, the industry remains divided on whether a 10% extinction risk is a mathematical probability or a speculative caution. While high-profile officials, including the Pentagon's CTO, have acknowledged Coxon's warnings, the path toward a unified international safety standard remains undefined. The primary question for regulators is whether to treat AI as a standard software risk or as a systemic threat to human existence.