It’s tempting, even fashionable, to pooh-pooh the hyperbole from our tech overlords. The release in 2022 of ChatGPT, the first mass-market conversational AI system, unleashed a volley of supercilious put-downs. The chatbot was not intelligent. It was merely hallucinating, manipulating statistics, regurgitating phrases from the internet: it was a ‘stochastic parrot’.
Well, over the next three years, ChatGPT became unputdownable. It learned to handle photographs and videos, extract wisdom from dense textbooks, sound like Scarlett Johansson, write everything from code to songs to emails and offer tips on fixing washing machines. Not bad for a parrot.
Claude Mythos heralds a future in which AI-empowered hoodlums empty your bank account
Since 7 April, when the Californian lab Anthropic announced the latest of its own AI supermodel, the same sceptical reflex has bubbled up again. The model in question, Claude Mythos Preview, specialises in finding vulnerabilities in software, heralding a future in which AI-empowered hoodlums steal data from your smartphone or empty your bank account. In the United States, Mythos has elicited a reaction bordering on panic. US Treasury Secretary Scott Bessent has directed the big banks to fortify their cyber defences, and Anthropic has declared its creation to be too dangerous for general release. Instead, the company is sharing Mythos with a restricted list of software giants. The hope is that the model will identify the chinks in their systems before an AI-powered bad guy brings them down.
This ‘Mythos moment’, as people have begun to call it, will cost companies billions: my advice is to buy shares in the consulting outfits that will help patch holes in their code. Yet in some ways Mythos is less shocking than ChatGPT’s arrival, when ordinary netizens came face to face with general machine intelligence for the first time. Mythos is more dangerous, certainly. But it has a precedent.
Ten years ago, the London AI lab DeepMind, a subsidiary of Google, unveiled its AlphaGo system, which defeated the South Korean Go champion, Lee Sedol. Until that showdown, Go experts had assumed that centuries of human exploration had discovered all possible stratagems the game allowed for; after Lee’s defeat, it was clear that a machine had fathomed unsuspected depths.
Mythos marks a repeat of the same story, but for cybersecurity. The pillars of the internet – the browsers, operating systems and database protocols – have been scrutinised millions of times by human experts. But now a machine has discovered faults in the code’s armour that programmers did not suspect.
And yet the fashionable scepticism lives on. The doubters observe that Donald Trump’s acolytes may be panicking, but that these are the people who brought us the Iran war – their judgment is suspect. Likewise, Anthropic is rationing access to Mythos, but this may be simply a ruse to inflate investor appetite ahead of a planned stockmarket listing; the lab is peddling ‘disaster porn as marketing’, in the words of one analyst.
Yet a third complaint is that Anthropic is hyping Mythos to gain leverage over the Trump administration, which recently branded Anthropic as a supply chain risk after a spat over the use of AI systems in autonomous weapons and mass surveillance. But whatever the merit of these speculations, it always seemed obvious that the Mythos threat was genuine. After all, Anthropic was promising to share the model with its biggest rivals. If the powers of Mythos turned out to be illusory, the lab would quickly become a laughing stock.
On 13 April the UK’s AI Security Institute, an underappreciated legacy of Rishi Sunak’s government, swept away the sceptics’ suspicions. In a sober analysis of Mythos, it concluded that the system ‘represents a step up over previous frontier models in a landscape where cyber performance was already rapidly improving’. In just two years, the institute noted, AI cyber capabilities have changed out of all recognition. Before, models could barely complete ‘beginner-level cyber tasks’. Today, Mythos can ‘execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously’. On expert-level challenges, particularly, Mythos outdoes all its rivals, penetrating its target’s defences 73 per cent of the time.
So much for the claims of hype. What’s more, the alarming truth is that rivals will soon have similar models. Already OpenAI, the maker of ChatGPT, says it has a Mythos competitor; and over the weekend, the head of a third AI lab assured me that he would develop an equivalent within a few months. A whole new class of powerful AI systems is coming at us. Buckle up.
Faced with this unsettling prospect, what is to be done? Anthropic’s strategy is surely not the right answer: the five-year-old startup has taken it upon itself to announce a shortlist of companies that can use Mythos Preview, condemning the massed ranks of cyber peons to life outside its fortified castle. One understands the logic here: if Mythos circulates widely, competitors, including Chinese ones, will rapidly ‘distil’ the model – reverse-engineering its capabilities and hastening the moment when bad guys can put them to bad ends. But it is not politically tenable for a private company to decide the fate of every entity in cyberspace. Nor will it be comforting if, as is rumoured, the Trump administration decides to requisition Mythos and arrogate that power unto itself.
Since 2012, when AI models first learned to identify cat photos, pioneers of the technology have wrestled with this conundrum. They recognised that AI would be too dangerous to be left unregulated, yet too important to be allowed to fall under the sway of a narrow, unrepresentative authority. Demis Hassabis, the CEO of DeepMind, dreamed that superintelligence could be birthed by a united multinational team of geniuses, overseen by philosophers and global statesmen who would guide the technology’s deployment on behalf of all the world.
OpenAI’s Sam Altman harboured even more dizzying visions. ‘We’re planning a way to allow wide swaths of the world to elect representatives to a new governance board,’ he proclaimed, having read Founding Father James Madison’s notes on the Constitutional Convention of 1787. Anything less than an internationally inclusive and participatory system would lack legitimacy, Altman believed. ‘Because if I weren’t in on this I’d be, like: why do these fuckers get to decide what happens to me?’
Seeking to make up for the lack of global governance bodies, both Hassabis and Altman experimented with novel oversight mechanisms within their own companies – attempts to blend public-minded safety vetting with the for-profit machinery needed to raise capital. For Hassabis, this meant beginning within the capitalist structure of DeepMind’s parent company, Google, then pushing for a public-interest safety board that would have the last say over AI deployment. For Altman, it meant setting up OpenAI as a public-spirited nonprofit, then grafting on a for-profit company to allow him to raise capital.
Both experiments fizzled. Over a period of three years, Hassabis retained teams of lawyers and advisers to advance his case with Google. But when push came to shove, Google’s corporate board would not allow an outside committee of do-gooders to hold sway over its technology. Likewise at OpenAI: when the nonprofit board took its public-interest mandate seriously and tried to fire Altman for alleged duplicity, the erstwhile student of James Madison defenestrated the coup leaders and sidelined the nonprofit. Both outcomes proved that companies are just companies. Despite what Anthropic is suggesting with its controlled release of Mythos, a company cannot be the vehicle for enlightened global governance.
In the wake of ChatGPT’s release, the AI safety community learned a second lesson. In 2023 and 2024, several advanced democracies began to set up machinery for national AI regulation: Sunak’s UK AI Security Institute was the best example, but there were others in the United States, Japan, Canada, Singapore and the European Union. The logical evolution of these safety bodies was that they would come to resemble pharmaceutical regulators, vetting AI models before their release to the public and vetoing the dangerous ones. In November 2024, just as Trump was re-elected, an international network of these safety overseers, now including representatives from South Korea and Kenya, met for the first time.
Proposed ways of handling the threat of cyber chaos fail to respect the lessons of artificial intelligence
But at the start of last year, the inadequacy of this effort was exposed. A Chinese lab called DeepSeek unveiled a powerful AI system, shattering the delusion that China could be left out of AI governance. Despite an American-led effort to deprive China of the most powerful computer chips, the DeepSeek model was almost as close to the Promethean goal of Artificial General Intelligence, or AGI, as its western rivals. Other Chinese tech firms soon rolled out similar models. Hassabis spelled out the consequences of this setback. An international safety collaboration that excluded China would be pointless. ‘If even only one or two of these projects design harmful AGIs then it could be seriously existential for humanity.’
Which brings us back to Mythos. The proposed ways of handling the threat of cyber chaos – Anthropic’s restricted-release policy, or a variant in which the US government dictates who gets Mythos – fail to respect the lessons taught by the brief history of artificial intelligence. A lasting, legitimate safety regime cannot be spearheaded by companies, and it cannot ignore China. It will have to involve governments and it will have to be global.
Given the rocky relations between China and the main democracies, negotiating global AI governance is a daunting challenge. But the Cold War teaches that it is not impossible. In October 1956, just a fortnight before Soviet tanks rolled into Hungary and Moscow threatened the West with nuclear strikes over the Suez Crisis, western and Soviet diplomats joined non-aligned countries in agreeing the blueprint for the International Atomic Energy Agency. The following decade, despite the near catastrophe of the Cuban Missile Crisis, negotiators came up with the Nuclear Non-Proliferation Treaty.
The lesson from the Cold War is that adversarial cooperation is possible. The Mythos threat demands that today’s leaders recall this history.
Sebastian Mallaby’s The Infinity Machine: Demis Hassabis, DeepMind and the Quest for Superintelligence is out now.
Comments