Former Anthropic Researcher Warns AI Could Cause Human Extinction by 2030
Insiders at the AI safety firm suggest current protocols may be insufficient to prevent catastrophic outcomes.
Jacob Coxon, a former researcher at the AI safety firm Anthropic, has resigned from his position and issued a public warning that artificial intelligence could lead to human extinction by 2030. His departure and subsequent warning highlight growing internal anxieties regarding the trajectory of frontier AI development.
Coxon, who stated that "AI could kill us all," is not alone in his assessment. Current Anthropic leadership has echoed these concerns; Evan Hubinger, the firm's Alignment Science lead, stated he believes there is a greater than 10% chance that AI could kill all humans within the next decade. Additionally, Samuel Marks, the company's scalable oversight lead, noted that AI developers believe the technology could cause human extinction in the next few years.
The Safety Paradox
Anthropic was founded in 2021 by former OpenAI members with a primary mission to build "reliable, interpretable, and steerable" systems. Unlike many of its competitors, the company was established specifically to prioritize AI safety from the ground up. However, the industry has seen a pattern of high-profile resignations as internal disagreements mount over the speed of development versus the implementation of rigorous safety guardrails.
Industry Implications
These warnings carry significant weight because they originate from within a company dedicated specifically to the science of AI safety. When the experts tasked with preventing catastrophe suggest that the risk of extinction is a tangible possibility—with a double-digit probability according to Hubinger—it suggests that existing safety protocols may be fundamentally insufficient to manage the risks of next-generation models.
The Path to Regulation
The public nature of these warnings increases pressure on global regulators to shift their approach to AI governance. To date, much of the industry has relied on voluntary commitments from major labs. However, the prospect of extinction-level risks suggests a need for legally binding frameworks and stringent oversight to manage frontier AI risks before they become uncontrollable.
Observers are now watching to see if these insider warnings will trigger a legislative response or if the competitive race for AI supremacy will continue to outpace the development of safety mechanisms. The tension between rapid innovation and existential security remains the central conflict of the AI era, as the window for establishing effective control mechanisms narrows.