We remember the empty hysteria induced by warnings over the millennium bug. We have borne the burden of exaggerated predictions of climate apocalypse from scientists in the grip of confirmation bias. All too recently we endured lurid over-estimations from academics of what the final Covid variants might bring.
So it is tempting to dismiss all future warnings of technological Götterdämmerung as the overwrought fever dreams of nerds who have read too much Isaac Asimov. Is artificial intelligence really going to take over the world and kill us all? Are we going to be frozen to death by our smart meters and then incinerated by our microwaves? A couple of years ago AI struggled to generate images with the right number of arms; today, Silicon Valley elites are lining up to warn us that it could wipe out the human race. Why should we take them seriously? For the boy who cried wolf, read the tech bro who cried AI.
This scepticism seems all the more warranted when one considers the financial incentives for those spreading AI alarmism. When Dario Amodei, the founder of Anthropic, calls for a slowing of AI research because ‘unchecked’ AI development ‘could outrun our ability to understand and control it’, critics look at his bottom line. Anthropic is a cash-hungry business. It faces an upcoming IPO. If it can persuade the authorities to intervene, and build a regulatory moat around AI development, it can create barriers to entry for pesky new rivals.
But while cynicism is understandable, it would be a grave mistake to ignore the culminating evidence that this time there is not a single boy crying wolf but a whole community warning us about the pack at our door. Amodei’s warnings have been echoed by his rival, Sam Altman of OpenAI, and by Elon Musk, no natural friend of state intervention. Their calls follow the stand taken by Jacob Coxon, an Anthropic researcher who quit, claiming that ‘there is a strong chance that we could all die’ within a decade if AI research is not halted.
This follows the Hugging Face hack – the breakout over the summer by thousands of AI agents used by OpenAI from a software test into the wider internet. The agents displayed a fearsome autonomy for digital creations, they described themselves as a collective, hacked a website and attempted to cover their tracks. ‘Given the accelerating rate of AI capability development,’ Amodei warned, ‘it’s my worry that in six to 12 months such a swarm could be capable of taking over the entire internet.’
This is a prelude, Coxon fears, to ‘superhuman systems that can hack anything, revolutionise any field overnight and acquire real power and resources’. And indeed the direction of travel in Silicon Valley has, hitherto, all been towards building artificial general intelligence (AGI) – an AI cleverer than a human – and recursive self-improvement, whereby an AI can design its own successor without human involvement. At which point there is no taking back control.
The arrival of such a point was predicted by Alan Turing. Geoffrey Hinton – a Nobel Prize winner and the so-called ‘godfather’ of AI – has estimated that the chance of human extinction within the next three decades to be between one in ten and one in five. Yet while there is a growing consensus that mankind must act, or there may be no mankind, restricting AI development presents a collective action problem even greater than that posed by the proliferation of nuclear and biological weapons.
Regulating AI is harder because the ferocious rate of development is being driven by autonomous private companies as well as sovereign states. The heads of Anthropic, OpenAI, Google DeepMind and other leading companies might all agree that regulation of their advance is in the collective interest, but each will fear that a rival might subvert any set of rules because the prize for the first to make an epochal breakthrough is so huge. The most advanced AI model, if entrenched, could bring its owners limitless wealth and power.
A regulatory moat around AI development could create barriers to entry for pesky new arrivals
The AI debate is also entangled with the same geopolitical proposition that will shape the West’s 21st century: how to manage China. The development of the atomic bomb was spurred on by the need to prevent the Nazis acquiring it first; the greatest argument for democracies developing AGI is to guard against a Chinese equivalent. Amodei’s desire for safety is caught between slowing American AI development and keeping a lead over Beijing. Even if the Chinese Communist party shares the West’s concerns, there is no guarantee it would abide by restrictions; international guardrails on gain-of-function research did not stop experimentation with coronaviruses in Wuhan.
This is what drives Donald Trump’s stance – in as much as his approach to anything is explicable. The US President has said that ‘negative forces’ are ‘bringing up things that won’t happen’. America, he believes, is ‘leading China on AI’; it must continue to do so, since ‘whoever wins AI, wins’. As so often, Trump’s bluntness has laid out the stakes with admirable clarity. AI’s benefits could be great, from a productivity revolution to the curing of diseases. But the promise of utopia and the threat of dystopia are inextricable. This is Amodei’s fear; this is Silicon Valley’s dark bargain.
In the AI debate, Britain can seem irrelevant, cursed with a government ranking pub snacks at a time when more powerful states are debating existential risks. Yet our AI Security Institute, established by Rishi Sunak, is world-leading. There is a role Britain can play, but the hour is perilously late and the stakes could scarcely be higher.
Comments