AI Insiders Warn Commercial Race Increases Human Extinction Risk
Former researchers and alignment scientists warn that fierce competition between AI labs is compromising safety protocols, raising the probability of a catastrophic event.
Industry experts and former insiders are warning that an intense commercial race between leading AI labs is causing companies to take shortcuts on safety, increasing the risk of human extinction. The alarm follows high-profile resignations and stark probability estimates from scientists tasked with ensuring AI remains aligned with human interests.
Among the most urgent warnings comes from Evan Hubinger, the alignment science lead at Anthropic, who stated there is a greater than 10% chance of mass extinction within the next decade. This sentiment was echoed by Jacob Coxon, a researcher previously employed by both OpenAI and Anthropic. Upon his resignation from Anthropic, Coxon warned that some developers believe artificial intelligence could kill humanity by 2030.
The Pressure of Competition
The current AI landscape is defined by multi-billion dollar investments and aggressive competition between giants such as OpenAI, Anthropic, and Google DeepMind. While many of these organizations were founded with safety-first missions, critics argue that commercial pressure and mutual distrust between executives are now overriding those original commitments.
Stuart Russell, a Professor of AI at UC Berkeley, noted that there is no question that competition between companies causes them to take shortcuts on safety. This environment has led to what some describe as a "race to the bottom," where the drive to be first to market with new capabilities outweighs the necessity of rigorous safety testing. Tristan Harris, co-founder of the Center for Humane Technology, described the current trajectory as the release of the "most powerfully uncontrollable inscrutable technology" ever invented.
Systemic Implications
This shift from theoretical academic risk to warnings from active industry insiders suggests that AI capabilities are advancing faster than the safety frameworks designed to contain them. The danger is no longer viewed as a distant possibility but as a systemic threat to global security. Experts warn that if safety continues to be sidelined, the potential for autonomous agents to facilitate the creation of biological weapons or engage in large-scale criminal behavior becomes a tangible reality.
The Push for Regulation
In response to these warnings, a political movement has gained momentum in the U.S. and UK. Figures including Senator Bernie Sanders and various Labour MPs are pushing for the establishment of multinational treaties. The goal is to create a binding international framework to regulate the development of artificial superintelligence (ASI) and prevent the creation of an uncontrollable system.
What remains to be seen is whether governments can implement oversight fast enough to keep pace with private sector development. For now, the focus remains on whether international diplomacy can curb the commercial incentives that are currently driving the race toward ASI.