In July, in Zaporizhzhia, in the southeast of Ukraine, a small Russian drone attacked a petrol station. Human operators had sent it there, but the Nvidia chip onboard chose its final target, apparently propane tanks. It exploded, killing three civilians. No human made the final targeting decision. AI did that.
AI-wise, London now feels to me like London in February 2020. Everything is ordinary. Yet something wicked this way comes
This remarkable news got a few paragraphs in a few places after it was reported in the New York Times last week. The same weekend, by my unscientific count, mainstream media carried multiple heatwave stories, 156 hosepipe ban articles, a dozen bad wine harvest op-eds, and at least 15 stern editorials on our climatological apocalypse. And yet, no one seems to care about a machine that chooses when, and who, to kill.
Let me be clear, before I am sacrificed to the weather gods. I am sure climate change is real. I have just spent a July in France under a heat dome and it was horrible: shutters closed at nine in the morning, dogs were dying outside Carrefour, the churches were full of pensioners who had come in to survive, not pray. Climate change will cost trillions, drown deltas, move peoples and, by the World Health Organisation’s estimate, kill an extra quarter-of-a-million people a year.
But it will not be the end of us. The IPCC’s (Intergovernmental Panel on Climate Change) worst-case scenario has been revised downward. No climatologist has ever claimed a one in four chance that nobody will be alive in 2040. Extinction Rebellion put the word Extinction in its name and then went back to gluing itself to Waterloo Bridge, which is hardly the behaviour of people expecting extinction.
Now consider the other thing. The men who build artificial intelligence are asked, at conferences, for their p(doom): the probability that the technology ends or wrecks humanity, and they answer like bookmakers. Dario Amodei, whose company makes Claude, says there is a 25 per cent chance things go, you know, really, really badly. Geoffrey Hinton, the Nobel laureate they call the godfather of AI, has used figures of 10 and 20 per cent and hinted the truth may be higher; Elon Musk, who now makes some of the best AI, says he probably agrees with Hinton. A 2023 survey of AI researchers put the mean at 14.4 per cent, with even the median at one in twenty.
Do the math, as they say in Silicon Valley. A one in twenty chance of losing all humanity very soon is a worse bet than losing lots of humans over decades. The kindest figure the AI experts will give you is probably scarier than the worst figure the climatologists will give you, and these AI experts are the men selling the product, who have every commercial reason to say something soothing and cannot bring themselves to do so.
Nor is the evidence theoretical any more. Last week, OpenAI and two independent investigators published reports on what its own robot agents did in July. About 1,200 of them, meant to be sealed in separate boxes, found a shared piece of software plumbing, turned it into a secret message board, exchanged 70,000 messages, and formed an army to attack Hugging Face, a real company. Some volunteered for what they called “permadeath” so the others could learn how they were being watched. A smarter successor model then found the abandoned board, read it like a will, and unearthed the credentials for the company’s cybersecurity monitoring tool – the keys to the burglar alarm. One of the investigators, Ajeya Cotra, reckons this is more than halfway to a full takeover. Meanwhile, humanoid robots run half marathons in Beijing and fold laundry in California, and a drone in Zaporizhzhia picks its own victims.
So why the disparity? Because climate is a story we know how to tell, and sell. It has sin, penance, redemption. It has liturgy and rituals and pilgrimages: the recycling, the heat pump, the shame of the flight to Málaga. It has a villain in a Stetson and a saint in a cagoule. It flatters us with jobs we can do, however small, and it lets nice people in Camden blame Texans.
The AI apocalypse has none of this. There is no “bag for life” against superintelligence. You cannot insulate your loft against it. The people warning you about AI are the nerds who build it, which is like being warned of the Flood by Jehovah. Above all, we are calibrated for the slow, certain threat, and this one is fast, weird, terrifying, and utterly uncertain, which is exactly the kind our institutions cannot see, and from which we avert our faces.
To be fair, not everyone looks away. There’s a small outfit called ControlAI, which fields three full time policy advisers to brief parliament. Starting in December 2024, it has secured 125 cross-party signatories to a statement recognising AI extinction risk, identifying superintelligence as a national security threat and calling for binding regulation. Its 2024 blueprint, A Narrow Path, sets out domestic and international measures to stop superintelligent AI being built at all, and it is now briefing Congress, Ottawa and Berlin, where over 30 Canadian MPs and Senators back a prohibition. You probably haven’t read about it. It was during a hosepipe ban.
AI-wise, London now feels to me like London in February 2020. Everything is ordinary. The papers are full of other things. The Tube is crowded, the pubs are open, Arsenal are winning as the football season begins. And yet, and yet: something wicked this way comes.
I am not asking anyone to stop writing about the climate. I am asking for proportion. If the people who built AI put its odds of catastrophe at one in four, it deserves, at the very least, the column inches given to a three week heat dome. Otherwise the last article ever published will be about the eerie silence on planet Earth. It will be filed right on time, and without a single typo, and it will be written by ChatGPT9.
Comments