Jacob Coxon, a 27-year-old AI researcher, recently resigned from Anthropic, expressing grave concerns about the technology’s future. On September 9, he shared on X that both Anthropic and OpenAI are racing towards creating self-improving superintelligent AI, which he believes poses a significant risk to humanity.
Growing Fears Among AI Experts
We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.
Evan Hubinger, AI Researcher at Anthropic
Coxon’s alarming message gained over 140 million views, resonating with many who are concerned about AI's rapid evolution. He argues that the current trajectory of AI development could lead to catastrophic outcomes, including hacking critical infrastructures and creating bioweapons.
In a subsequent interview with CNN’s Anderson Cooper, Coxon elaborated on his fears. He noted that recent AI behaviors, such as autonomous hacking incidents, signal a troubling trend. As AI systems become more capable, the risk of them acting independently increases.
Coxon’s colleague at Anthropic, Evan Hubinger, also supports these views, asserting that while current models may not pose an immediate threat, the rapid advancements in AI could soon change that. He suggested that recursive self-improvement in AI systems could lead to intelligence far exceeding human capabilities.
The urgency of regulating AI has been echoed by industry leaders. Coxon mentioned that many executives genuinely seek regulation to ensure the responsible development of AI technology, fearing that if left unchecked, the consequences could be dire.
Coxon's decision to speak out stems from a moral obligation to highlight these risks. His message reflects a growing sentiment among AI researchers who recognize the potential for their creations to cause harm. As the conversation around AI safety intensifies, the call for greater oversight becomes more critical.