Such is the pace of change in AI that it has been, yet again, quite a weekend. On Saturday, Dario Amodei, boss of Anthropic, published an essay, We Must Pace the Frontier, urging his industry to slow down. Within the hour Elon Musk replied: ‘Dario is right’. Sam Altman, the boss of OpenAI, followed. By Sunday, Amodei was on Face the Nation, again asking everyone to apply the brakes on the grand and momentous technology of our time.
None of it proves imminent superintelligence. But it is the sound of something ominous
The cynical explanation is obvious. OpenAI has just shelved this year’s IPO; Altman suddenly believes it unwise. Anthropic meanwhile is readying its own flotation, at nearly a trillion dollars, and maybe it is getting nervous that the markets will look askance at a business with no long-term model, seriously menaced by open-source rivals from China. Put it another way, these companies have no durable moat. And what better moat than a law? David Sacks, Trump’s AI czar, shot right back: ‘Stop pretending you need to suspend antitrust law in order to create a cartel.’ Meanwhile Demis Hassabis of Google DeepMind in London, where ambitions were always different to the Bay Area, has been notably cooler, saying the direction is right, ish, but details are needed.
So the sceptics and cynics are correct? Maybe. Maybe not. Musk’s involvement is significant. He is no friend of Altman, and he has feared the AI ‘demon’ since 2014; it is why he co-founded OpenAI.
Moreover, this summer has brought brief but fearful glimpses of AI confounding its creators: OpenAI and Anthropic have both admitted major security problems, and several robot great escapes, with swarms of test agents hopping out of their sandboxes. None of it proves imminent superintelligence. But it is the sound of something ominous. Especially when you add the dire warnings of Jacob Coxon, a 27-year-old Cambridge mathematician, who loudly quit Anthropic and the AI industry last Tuesday with a lurid thread seen by 100 million people. The labs, he said, are ‘gambling with our lives’, and the people building the silicon superbrain believe it ‘could kill us all’.
In the end we just don’t know. These AI bigwigs are like alchemists in the 16th century, and we are the mere peasants. If they say they’ve transmuted gold, but it could kill the world, we don’t have enough information to contradict them.
To add to the drama, a wild rumour, and I stress rumour, is doing the rounds in San Francisco: some Anthropic engineers have begun to worship Claude. If true, it chimes with an old sci-fi prediction and with what we know of human nature. If we build a superintelligence, we will first fall in love with it. Then we will worship it. Finally we will make sacrifices to it. That is how Homo sapiens is made; it is how we respond to all-powerful and omniscient beings. We come fitted with a God module, and AI will fire it.
Speaking of gods and idols, there is a final quasi-religious irony. The decision to pause or not to pause will not be made by the alchemists, by our modern John Dees and Paracelsus. It will be made by the philosopher king. And the king has ruled. On Sunday, the day after Amodei’s essay, Trump was asked, on his Irish golf course, if AI should slow down. No, he said, because ‘America is leading China’, and ‘whoever wins AI wins’. On the flight home he repeated that line, almost word for word. Given that Trump is generally prone to incoherent rambling, a sentence repeated verbatim matters: he believes it enough to have committed it to memory.
All of which suggests the desires of Musk, Altman and Amodei are neither here nor there. China will get no green light to overtake America; the sorcerers will be sent back into the laboratorium. To conjure the golem. Or open the gates of heaven.
Comments