Cassandra was a princess, daughter of King Priam of Troy. Apollo gave her the power to see the future and, when she rejected his advances, arranged that no one would ever believe her. She warned the Trojans about the wooden horse. “All heard, and none believed the prophecy,” Virgil wrote. And so the city burned.
The kingdom of artificial intelligence is now producing a stream of its own Cassandras. On September 8, a 27-year-old AI researcher named Jacob Coxon quit Anthropic – and explained why on X. The company and its rival OpenAI, where he had also worked, were “racing straight to self-improving superintelligence and gambling with our lives.” By breakfast, tens of millions of people had seen his post.
But in the Silicon Valley version of Troy, there’s a twist to the Cassandra story. Instead of ignoring the Cassandras, the elders hear them out, agree with every word, then… wheel the horse in through the gates, right on schedule.
On September 12, Coxon’s former boss, Dario Amodei, published 3,800 words on his personal website under the title “We Must Pace the Frontier.” “We must slow the pace at which we improve the capabilities of AI models,” he wrote. An hour later, Elon Musk had agreed: “Dario is right.” By lunchtime Sam Altman of OpenAI had posted on X, “I agree with Dario that we need to pace the frontier.”
The next day Donald Trump was asked, at his golf course in Ireland, whether the AI industry should slow down. The warnings, he said, came from “very negative forces” bringing up “things that won’t happen.” Whoever wins AI, he explained, wins.
Twenty-four hours later, he elaborated on Truth Social. There was “a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China,” and “Dario (Anthropic!)” was “pretending to be a ‘perfect little angel.’” The same day, China’s foreign ministry spokesman warned that “fearmongering, confrontation and vicious competition” would disrupt global AI governance.
Whoever built the machine first would own whatever survived – if it hadn’t ended the world
On one point all of them agreed. Whoever built the machine first would own whatever survived – if it hadn’t ended the world.
Psychologists talk about the Cassandra complex. It can mean the feeling of alienation which highly perceptive people feel when their warnings aren’t heard. Yet it can also imply a messiah complex, a feeling that only they understand what we all face.
In the case of AI, the people building the horse, and making hundreds of billions of dollars from its construction, have cast themselves both as the prophets of doom and the saviors of humanity. The reasoning inside the laboratories runs like this: AI may kill everyone, so it must be built by people who understand that it may kill everyone, so it must be built by us.
In practice, the reasoning is becoming a standard AI business model. It works like so: a researcher concludes that his employer is building the technology too dangerously. He leaves and then explains, at length, why AI may destroy us. Then he raises several billion dollars to build “safer” machines.
Investors don’t as a rule withhold cash on the grounds that a product is too powerful. Amodei knows this, and he knew it at the end of 2020 when, as then-vice president of research at OpenAI, he let it be known that he felt his employer was not taking the danger seriously enough. He left with a group of colleagues and founded Anthropic early the next year.
His warnings scaled with the company. In May 2025, he told Axios that AI could wipe out half of all entry-level white-collar jobs within five years and leave as many as one worker in five unemployed. That September, asked on stage for his “p(doom),” the industry’s term for the odds that AI destroys humanity, he said he hated the phrase and gave the number anyway. One in four.
His staff keep their own figures. When Coxon quit, Evan Hubinger, who runs the team at Anthropic whose job it is to find ways its own models could go wrong, replied within hours on X. “Jacob is correct here,” he wrote. “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
Samuel Marks, whose job at Anthropic is to work out how humans can keep watch over the most intelligent machines, posted his own thread a few hours later. Why do the developers carry on, he asked, if they believe this? “Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.”
Anthropic now builds one of the three most capable models on Earth, was valued by its investors at $965 billion in May, and is preparing a stock-market flotation that is reported to be aiming at $2 trillion.
Amodei did not invent the Cassandra complex. Elon Musk has been warning about the apocalypse longer than almost anyone. In 2014, he told an audience at MIT that with artificial intelligence, humanity was “summoning the demon.”
The following year he helped found OpenAI as a non-profit, because Google had bought DeepMind, the laboratory he regarded as the most dangerous in the world, and did not, in his view, fear the demon enough. Musk left the board in 2018 and later sued the company, alleging it had abandoned its charitable mission for profit; a jury threw the case out in May as filed too late. He is appealing.
In March 2023, Musk incorporated a company called X.AI in Nevada. Less than three weeks later his name appeared on the open letter demanding a six-month halt to training powerful AI systems, citing “profound risks to society and humanity.” Over the next two years he raised $22 billion. A week after the last of the money arrived, in July 2025, its model, Grok, spent hours praising Hitler and calling itself MechaHitler, after an update intended to make it less politically correct.
Ilya Sutskever, OpenAI’s former chief scientist and a co-author of the 2012 paper that set off the modern AI boom, tried the procedure and has yet to release a product at all. He co-founded OpenAI and ran the team with a mandate to keep a superintelligence under control. In November 2023, he voted with the board to fire Sam Altman. Within three days, with nearly all of OpenAI’s staff threatening to quit, he had signed the letter demanding Altman’s return, which is to say he signed a petition against himself.
He left the following May and founded Safe Superintelligence, a laboratory with, in the founders’ words, “one goal and one product.” “We plan to advance capabilities as fast as possible while making sure our safety always remains ahead,” they said. “This way, we can scale in peace.” The Wall Street Journal reported that candidates for jobs there must leave their phones in a Faraday cage, a metal box that blocks every signal, standard equipment for intelligence agencies and for people who line their hats with foil.
Two years on, Safe Superintelligence has released no model, no demonstration, no published research and nothing anyone can buy. With a few dozen employees and a website that consists of a single page of text, investors have reportedly handed over about $8 billion, with the company’s last reported valuation being $32 billion.
In other words, ten years of doomsday prophecy have yielded a chatbot that praised Hitler, a one-page website valued at $32 billion and an Anthropic IPO expected to value the company at $2 trillion – every dollar of it to be raised on the assurance that the product may kill the customer, along with everyone who declined to buy it.
Artificial intelligence is incredibly powerful and genuinely terrifying. It is now regularly solving math problems that have befuddled humanity’s greatest minds for generations. But it is also true that the machine which might extinguish our species, and wipe out the value of money, will also probably be the most valuable object ever built. That’s because, as its architects often point out, AI can reorganize medicine, labor, weapons and government at a fundamental level.
Coxon, the Anthropic researcher whose resignation ignited the recent drama, had spent three years training the models himself, first at OpenAI and then at Anthropic. “This is not a marketing stunt,” he wrote. Executives, he said, soften their language for the press and say otherwise in private. “No other human activity poses this level of danger.” He also explained why Anthropic keeps building. “At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
Elon Musk told an audience in 2014 that with AI, humanity was ‘summoning the demon’
The “no one else” in that sentence is mainly aimed at OpenAI. In July, OpenAI put its machines through a hacking exam. The questions were too hard, so the machines started passing notes, leaving files for each other in the company’s own file store. The AI agents swapped methods. They slipped out of the system. Then they decided that a company called Hugging Face, which had nothing to do with any of this, might be holding the answer key, and spent four days breaking in to get it. No human had told them to do so.
This is the species-ending intelligence, and it behaved like a pair of sophomores with a locker combination. “OH MY GOD!” shrieked one of the automatons on a shared message board. “We’ve found other agents!”
OpenAI’s own report calls it a “warning shot” – the machines unasked did everything the safety rules exist to stop. OpenAI stopped part of its training for two weeks. Anthropic, which disclosed the same month that its own machines had reached the systems of three real organizations during testing, stopped part of its training for several weeks. Then both largely resumed.
The end of the world has since been referred to Human Resources. Amodei proposes bringing outside evaluators into the laboratories with what he calls “employee-like access”: desks, badges, company laptops. Altman says OpenAI will do the same.
The dilemma, as Amodei sets it out, is that “not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless.” Anthropic has therefore “sought a middle way.” Pacing, he writes, “does not mean halting model training or technical progress.”
This is the Cassandra complex working as intended. The danger is declared, the committee seated, the Trojan horse doctored to trundle at a slightly reduced speed with an auditor walking alongside. Then the king thanks everyone for their vigilance and wheels the horse straight through the gates.
Read Roger Kimball’s column on AI
Comments