Anthropic Insider Warns of 10% Chance AI Could Cause Human Extinction
Researchers and former officials call for international treaties to prevent the creation of superintelligent systems as labs race toward ASI.
Experts from the world's most prominent AI laboratories are warning that the pursuit of artificial superintelligence (ASI) could lead to the extinction of the human race. These warnings, coming from within the industry's inner circle, have sparked urgent calls for legislative bans and global treaties to regulate the development of autonomous systems.
Evan Hubinger, the alignment science lead at Anthropic, stated that there is a greater than 10% chance that AI could kill all humans within the next decade. This assessment is echoed by Jacob Coxon, a former researcher at both OpenAI and Anthropic. After resigning from Anthropic, Coxon claimed that leading AI companies are "gambling with our lives" in a competitive race to achieve ASI, asserting that the belief in these risks is a genuine concern among those building the technology rather than a marketing stunt.
The Alignment Challenge
The core of the crisis is a technical hurdle known as "alignment"—the difficulty of ensuring that a superintelligent system's goals remain compatible with human survival. Proponents of the existential risk theory argue that AI capable of recursive self-improvement could rapidly evolve beyond human understanding, leading to an irreversible loss of control.
While some critics argue that "Terminator scenarios" serve as a distraction from immediate harms like misinformation and job displacement, others see the threat as an immediate geopolitical priority. Former UK defence secretary Des Browne has compared the scale of the threat posed by superintelligent AI to that of nuclear war.
Regulatory Pushback
The alarm has moved from academic circles into the halls of government. In the UK, a lobbying group called Control AI, funded by Skype founder Jaan Tallinn, is pushing for a parliamentary bill that would explicitly ban the creation of ASI. Tallinn has further highlighted an internal cultural divide within the industry, estimating that 10% to 15% of AI employees believe the technology may eventually be a "worthy successor" to humanity.
In the United States, Senator Bernie Sanders has criticized the lack of oversight, stating that a handful of "greedy people" cannot be allowed to determine the future of humanity without public input.
The Path Forward
As the gap between technical capabilities and safety frameworks widens, the industry's transparency is coming under scrutiny. Anthropic recently declined to submit its latest model, Mythos 5.1, to the UK's AI Security Institute for pre-release testing, raising questions about the willingness of labs to undergo independent safety audits.
Observers are now watching for the emergence of a global treaty on ASI, similar to nuclear non-proliferation agreements. Whether governments can implement meaningful bans before a system achieves recursive self-improvement remains the central question for the next decade of technological development.