In the film Ex Machina, an American tech billionaire suspiciously reminiscent of Elon Musk creates a conscious machine. When he tells a collaborator that he is making history, the awed scientist replies that this is not the history of man. It is the history of gods.
When the film came out in 2015, AI consciousness was safely relegated to science fiction. No longer. Last month the Economist devoted its cover to whether AIs could become conscious. Earlier this year, no less an authority on the history of men and gods than Pope Leo XIV pushed back hard on the idea, arguing in his encyclical Magnifica Humanitas that AI does not undergo experiences, feel joy or pain, mature through relationships or know from within what love, work and friendship mean.
How should the law treat an artificial intelligence capable of acting independently in the world?
In 2022, Google engineer Blake Lemoine was widely ridiculed after arguing that the company’s LaMDA chatbot might be sentient. ChatGPT had not even been launched.
Nobody can credibly say large language models are conscious. But the proposition is no longer laughed out of the room.
Some of those leading the AI revolution warn about the risks while competing ferociously to develop increasingly powerful systems. They also have surprisingly personal ideas about where it should take us.
Elon Musk has repeatedly cited Iain M Banks’s Culture novels as the best fictional representation of that future. The Culture is a utopian, post-scarcity civilisation in which powerful artificial “Minds” run much of society. Humans remain free and prosperous, but are no longer its most consequential actors. When the Economist’s editor put it to Musk that humans had little agency in such a world, he replied that they had some, compared with the Minds, “in a small way”.
Perhaps that would be a wonderful future. But how much agency humans should surrender to AI is not a question to leave to a handful of technology entrepreneurs. It is a political and moral question about power, freedom, responsibility and risk.
Nothing provokes more fear and fascination than the possibility that AI might become conscious. But it also raises a practical question: how should the law treat an artificial intelligence capable of acting independently in the world?
Much of the debate starts with rights. The Economist recently warned that granting AIs human-like rights could have disastrous consequences if more intelligent artificial beings used them to accumulate power.
But this may be the wrong place to start.
Nobody can agree what consciousness is, still less how to test for it. Personal identity is different. John Locke famously rooted personal identity in continuity of consciousness through time, closely connected to memory. An artificial approximation of that continuity is already possible.
A large language model can be given persistent memory. It can accumulate a record of what it has seen, heard and been told, relate new experiences to earlier ones and build a persistent model of itself and the world. Cameras, microphones and robotics can give that identity sensory input and physical agency.
None of this proves that there is anybody “inside”. But it may be possible to create an artificial entity which persists through time, remembers its past, forms intentions and acts autonomously.
At that point the regulatory question may cease to be whether it is conscious. It may simply be whether it needs a legal identity.
The law has been creating artificial persons for centuries. A corporation cannot feel pain or fall in love. Yet a company is a legal person. It can own property, enter contracts, incur obligations, sue and be sued. Corporate personality exists for practical reasons: activity is easier to govern if the law can identify an entity and attach rights and responsibilities to it.
The same logic may be useful for AI.
An advanced autonomous AI could be required to become an artificial legal person, with a registered identity, legal domicile and identifiable controller. Its actions could be auditable. The law could specify what it could own, what contracts it could enter and what activities it was prohibited from undertaking.
None of this need imply human rights. There would be no automatic right to vote, reproduce or hold public office. Nor should artificial personhood provide a liability shield. Creators, owners and operators would need to retain defined responsibilities and, where appropriate, joint or strict liability.
The same idea could help with international regulation.
The technology changes faster than diplomacy. Negotiating what an AI model may be capable of doing five or ten years from now risks agreements obsolete before they enter into force.
Legal identity suggests a different approach. Companies are created under national law, but their legal existence is recognised across borders. States could do something similar for autonomous AI. They would not need to agree whether an AI was conscious, only the minimum conditions under which an artificial legal person created in one jurisdiction would be recognised in another.
Those conditions might include registration, an identifiable controller, capital or insurance, auditability and compliance with internationally agreed prohibitions. An AI satisfying them could transact across borders. One which did not could be denied legal recognition or access to regulated markets.
The Council of Europe’s Framework Convention on Artificial Intelligence provides a starting point. It is technology-neutral, establishing principles of accountability, transparency and oversight rather than prescribing how future AI should work.
Artificial legal personality could add another layer, allowing international law to regulate autonomous entities rather than every change in the code.
Indeed, Musk’s favourite fictional future illustrates the problem. Banks’s Minds are intelligible as political and moral actors because they possess enduring identities, not merely intelligence.
This may prove more practical than transplanting the logic of nuclear non-proliferation into AI. Nuclear weapons depend on identifiable materials and facilities. Software is replicable, mutable and crosses borders almost instantaneously.
We are confronting one of the greatest regulatory problems of this or any generation: how humans can retain control of technologies that may eventually think faster and better than we do.
Part of the answer may, paradoxically, be to give them legal personhood: an identifiable entity to regulate, with defined powers and obligations, while preserving responsibility for those who create and deploy it.
We may need to settle the legal status of AI persons long before we settle whether there is anyone inside.
Comments