By Kaushik Basu
Over the past decade, scholars and journalists have been writing about what the march of digital technology means for humanity with a mixture of hope and anxiety: hope because of the rise in living standards that the digital revolution has made possible, and anxiety because it has deepened social divisions and destabilised politics around the world.
But anxiety has recently taken an existential turn, as the industry’s leaders have begun warning that AI could very well end humanity. Most notably, Evan Hubinger, Anthropic’s alignment science lead, said there’s a greater than 10% chance that AI will kill all humans within the next decade.
Similar, though somewhat less alarmist, warnings have come from OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Bill Gates, and other prominent tech figures, all of whom have called for slowing down AI development. Naturally, these warnings have fuelled concerns about what AI can do to us rather than for us.
AI models can already build more advanced AI with little human involvement, raising the prospect of “recursive self-improvement” that unleashes forces we have little ability to understand or control. The Hugging Face breach, in which hundreds of AI agents that were supposed to work independently found ways to communicate and started acting as a swarm, offered a disturbing preview. All this brings to mind Bertrand Russell.
In his famous critique of induction, Russell noted how chickens that get used to being patted every morning are in for a shock when, one day, the farmer—presumably having invited friends for lunch— wrings their necks instead. As he observed, “more refined views as to the uniformity of nature would have been useful to the chicken.” I share Russell’s skepticism of induction but am not persuaded by his advice for the chickens.
If the chickens followed his advice, they would simply live their last days in helpless panic. It is different for humans, since, unlike Russell’s chickens, we have the power to modify our own behaviour. And there is no shortage of proposals for how we should curb the excesses of AI, from pausing the development of frontier models to banning superintelligent AI altogether. As sensible as these proposals may be, they may not work.
Game theory, and the prisoner’s dilemma in particular, tells us why. If each corporation is left to act unilaterally and in its own interest, we may end up with the worst outcome for everyone. What is needed, as Amodei has argued, is “industry-wide coordination.” Yet even that may not be enough, since AI is also a geopolitical problem. If the United States exercises restraint, China could surge ahead. Some commentators have seized on this fear to argue that the US must not hold back on AI development. That would be a mistake.
The US should work with China to reach a global agreement, but even if that doesn’t happen, there is no reason to charge ahead at full speed. It is worth remembering that the only time atomic bombs have been used in war was when the US had a nuclear monopoly. It sounds Machiavellian, but nuclear restraint took hold only after the Soviet Union, and eventually other countries, acquired the bomb. The same logic may apply to AI.
For all the warnings about AI, the truth is that no one knows what kind of world it will produce. AI agents could run amok, forming swarms and turning against their own creators. But a more realistic scenario is that a handful of super-wealthy individuals use their control of powerful AI systems to establish an Orwellian global state—the North Korean model gone global—and end democracy as we know it.
This makes curbing extreme inequality all the more urgent. We need a special tax on the wealth of the super-rich, with the proceeds going to those below a certain income. As the Roosevelt Institute’s Michael Madowitz recently argued, democracy’s survival depends on our ability to tax extreme wealth. Oligarchs commanding swarms of AI agents could become more powerful than most governments—a recipe for an authoritarian future.
Taxing their fortunes would curb that power and could help finance less murderous, more useful technologies.
The writer, a former chief economist of the World Bank and chief economic adviser to the Government of India, is Professor of Economics at Cornell University.