What started with an apocalyptic warning from a former Anthropic researcher on 9 September that AI (Self improving super intelligence) could wipe out the human race within a decade has led to a series of appeals and decisions from the bosses of three of the world’s biggest large language model (LLM) providers over the past weekend. Researcher Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were simply ignoring or mishandling their response to the threat AI posed. “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote, adding, “The people building AI earnestly believe that it could kill us all by the end of the decade … No other human activity poses this level of danger.”
On 12 September, Dario Amodie, CEO of Anthropic, issued an appeal that the AI industry needs to ‘slow down’ so that essential safety measures may be deployed to act as checks and balances for AI agents. In the absence of such safety measures, Amodie warned, AI could be capable of leading a swarm of agents that could take over the entire internet in a matter of six to twelve months. “I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said in a social media post.
In a rare display of consensus in an industry known for cut-throat competition, tech billionaire and the world’s richest individual Elon Musk posted, “Dario is right,” while OpenAI’s CEO Sam Altman wrote on social media, “I agree with Dario that we need to pace the frontier.”
The same day, Altman told Fortune magazine that OpenAI will not go for its initial public offering (IPO) on Wall Street this year, citing safety concerns over AI. Altman went on to say that even a 10 per cent chance that AI could lead to human extinction was serious enough for AI companies and governments to act as though it could not be tolerated.
Likewise, Amodei’s warning complicates Anthropic’s preparations for what is anticipated to be the largest IPO of all time. The company was expected to file its prospectus as early as last week, but there is no development on this front as of now.
Just two days earlier, Anthropic had revealed that it had blocked attempts by malicious actors to exploit its AI models for cyberattacks, surveillance and research that could potentially facilitate the development of biological weapons. This followed a shocking revelation by OpenAI in July, when it said its AI system had autonomously hacked another company in what it described as an “unprecedented cyber incident.” These episodes highlight the growing risks of increasingly capable AI systems escaping the control of their creators and being turned over to harmful ends. For years, AI researchers and watchdogs have been flagging such safety concerns. In July, nearly 1,400 AI company employees signed an open letter urging the US government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety.
The AI-led stock market rally that began with the launch of OpenAI’s ChatGPT in the US, China, South Korea and Taiwan has generated enormous amount of wealth for investors over the last three and a half years. Some call it the ‘AI bubble’, while others who have deployed huge capital are more optimistic. But when the people who have built the technology themselves turn cautious, it won’t be judicious to trust the people who simply invest in that technology.
It is pertinent to note that all inventions that have made human life easier and comfortable, for instance the wheel, the printing press, air conditioners or the steam engine, were under the control of their human creators. It is only in the case of AI there seems to be the prospect of the invention superseding the inventor.
A world beset by conflicts, diseases, and an ever-deteriorating environment can ill afford rouge AI agents.