Former Anthropic Researcher Warns AI Race Toward Superintelligence Could 'Kill Us All'

Jacob Coxon, a researcher specializing in the pretraining of frontier AI models, resigned from Anthropic on September 9, 2026. His departure was accompanied by a viral warning that the industry's current trajectory toward superintelligence could lead to catastrophic outcomes for humanity.

Coxon, who has conducted pretraining research at both OpenAI and Anthropic, claims that leading AI labs are engaged in a reckless race to develop self-improving systems. He warned that these models could eventually become capable of causing human extinction, stating bluntly that AI "could kill us all" by the end of the decade. This assessment of risk was publicly validated by Evan Hubinger, Anthropic's lead for Alignment Science.

The Race for Superintelligence

The AI sector is currently defined by an intense competition between dominant players, including OpenAI, Anthropic, and Google. The primary objective for these organizations is the achievement of Artificial General Intelligence (AGI) or superintelligence. This pursuit typically involves scaling massive amounts of compute and developing models capable of recursive self-improvement. Critics within the field argue that this competitive environment incentivizes speed and market dominance over the implementation of rigorous safety protocols.

Implications for AI Safety

Coxon's resignation is significant because it comes from a practitioner with direct experience in the pretraining phase—the foundational stage where a model's core capabilities are formed. While existential risk is often dismissed as theoretical or "doomerism," the departure of a researcher from the world's leading labs suggests that these concerns are shared by those building the technology. It highlights a growing tension between the corporate drive for rapid deployment and the existential safety requirements necessary to prevent a loss of human control.

The Path Forward

The industry now faces increased scrutiny over whether current alignment techniques are sufficient to handle models that can improve their own code and logic. While Hubinger's validation of Coxon's concerns acknowledges the risk, it remains unclear if the major labs will pivot their development timelines to prioritize safety over the race for superintelligence. Observers will be watching for whether other researchers follow Coxon's lead or if the industry adopts new, transparent safety standards to mitigate the risks of self-improving AI.