Everywhere we go these days, we hear people talking about the inevitable arrival of artificial general intelligence (AGI), usually understood as the moment when machines can perform every cognitive task as well as or better than humans. Some people await this moment with excitement, others with dread. Either way, the assumption is that sooner or later machines will cross some threshold and surpass us across the board. I have come to summarize my response to this perception in seven letters: AGI is BS.
Part of the problem is that AGI has no agreed scientific definition. The term has increasingly become shorthand for the idea that intelligence exists on a single ladder, with today’s machines somewhere below us and tomorrow’s destined to climb past us. But intelligence does not work that way.
Don’t get me wrong: we are absolutely entering an age of extraordinarily powerful machine intelligence. Our AI systems will solve problems we cannot today solve, accelerate scientific discovery, transform industries and help us understand complex systems in ways we can today barely imagine. They will become superintelligent and even creative in important domains. But the idea that machines will become superior to us at everything that matters rests on a narrow understanding of intelligence and an impoverished appreciation of ourselves.
Arguments about exactly when AGI will arrive can distract us from the challenge already in front of us
Human intelligence is the embodied product of nearly four billion years of evolution, built through interactions among our brains, bodies, senses, environments, cultures, relationships and experiences. We know far more than we can put into words. We recognize faces, sense danger, fall in love, read rooms, create transcendent art, comfort frightened children, improvise with strangers, develop novel ideas, imagine new worlds and make countless judgments, big and small, without being able to explain precisely how we do any of these things.
Our most advanced AI systems, by contrast, have been trained largely on our recorded words, images, sounds, computer code, scientific data and other real and synthetic digitized manifestations of human activity. These records are astonishingly rich but they do not remotely represent the full complexity of our humanity.
The animal kingdom should make us humble about our ability to even fully define what intelligence is. A dog inhabits a world organized partly by smells we can barely perceive. Migratory birds navigate across continents using sensory capacities we do not possess. Dolphins experience an acoustic environment profoundly different from ours. An octopus processes information through a nervous system distributed in ways radically unlike our own.
Intelligence is a widely varied collection of capabilities shaped by bodies, environments, histories and needs. This helps explain why AI progress is so jagged. AI systems can perform astonishing feats of mathematics, coding, pattern recognition and scientific analysis while not being able to solve many different problems in these domains and failing at many tasks we find straightforward. Those gaps will shift as technology improves, but new gaps will appear. We will keep discovering unique human capacities where we clearly outperform our machines, as well as recognizing others we had barely noticed before because they seemed too ordinary to name.
Future machines will increasingly have bodies, sensors, persistent memories and richer interactions with the physical world, but this does not mean they will simply recreate nearly four billion years of biological evolution and then pass us on some single scale. They will develop different combinations of capabilities, some vastly exceeding ours and others profoundly unlike our own. None of this should make us complacent, not least because machines do not need to be better than humans at everything to become immensely powerful.
Evolution offers a useful analogy. Human beings did not become the dominant species on Earth because we outperformed every other animal at every task. We cannot outrun a cheetah, outlift a gorilla, outsmell a dog, outswim a dolphin or outnavigate a migratory bird. What made us unusually powerful was a particular combination of capabilities including language, abstraction, social coordination, cumulative culture, toolmaking and our extraordinary ability to pass down knowledge through the generations.
We should think about AI’s evolutionary path similarly. Imagine machines that remain profoundly inferior to humans in many dimensions of lived experience but become vastly better at many aspects of analyzing enormous datasets, writing software, modeling complex systems, optimizing processes, uncovering scientific patterns, designing molecules, coordinating networks and helping build subsequent generations of machines. They would not necessarily need consciousness, love, wisdom, childhoods, bodies and biologies like ours, or a sense of mortality, to become transformatively consequential. They would simply need to become extraordinarily capable at enough consequential things.
This is why arguments about exactly when AGI will arrive can distract us from the challenge already in front of us. We should worry less about crossing a mythical AGI threshold and more about what happens as machines become increasingly powerful at activities that shape our economies, societies, security, health and politics.
Intelligence, capability and power are not the same thing. A system does not need to possess some generalized human-equivalent intelligence to exercise enormous power. Speed, scale, autonomy, replication, connectivity and access to critical systems can turn even relatively narrow capacities into collectively transformative capabilities. A system able to act millions of times at machine speed, or increasingly to operate laboratories, execute financial transactions, write and deploy software, control robots or help design subsequent generations of machines can transform the world without ever crossing an imaginary AGI line.
The good news for us is that our societies have successfully navigated revolutionary transformations before, even if rarely at anything approaching the speed we are experiencing today. Nearly all humans lived as hunter-gatherers only 12,000 years ago. Agriculture eventually allowed increasing numbers of our ancestors to specialize in activities other than foraging for food and to build civilizations. Industrialization dramatically reduced the share of human labor required for farming while creating entirely new and deeply meaningful categories of work. None of our ancestors at the dawn of agriculture or even of the steam engine could have imagined semiconductor fabs, space stations, digitally networked civilizations or intelligent machines, but these, and much more, have become our new normal.
Because all major transitions come with dislocation and pain as well as opportunity, we need to be putting tremendous energy into managing our current shift as wisely as possible. We cannot and should not try to preserve every task humans currently perform, but must seek to build a world in which machines do more of what they do best while fostering greater opportunities for humans to discover and do all that we do best. Just like our ancestors found new ways of exploring their human capacities when past technologies made it less collectively necessary to hunt and gather, farm, and perform manual labor, so must we.
This transition will not only change what humans do but also how wealth and power are distributed. Machines do not need to reach AGI to displace large numbers of workers, alter the relative value of capital and labor, or dramatically increase the power of the people, companies and governments controlling the most capable systems. One of the greatest risks of the AI revolution may be not that machines become more powerful than humans, but that some humans become vastly more powerful than others by using those machines. Questions of ownership, access, opportunity and the distribution of benefits must therefore be central to how we navigate this transition.
To increase our odds of getting things right, we desperately need better governance at all levels
The potential gains of getting this right are enormous. As I have argued in my book Superconvergence, the merging of artificial intelligence with genetics, biotechnology and other powerful technologies will help us radically improve health and healthcare, make agriculture more productive and sustainable, accelerate the development of new materials and energy systems and process information with efficiencies closer to our highly efficient brains than today’s energy-hungry data centers.
But the same systems that accelerate drug discovery can accelerate the design of deadly pathogens. Systems that personalize education and optimize supply chains can scale dangerous and targeted manipulation and harmful mass surveillance. Concentrated technological power can deepen inequality and undermine democratic accountability. The more capable our machines become, the more we will need to ensure that our most cherished values guide their application.
We are living through a Goldilocks moment in which we cannot afford to either overstate or understate the challenges we face. Fantasies of inevitable AGI and machine takeovers can encourage a fatalistic, disempowering and ultimately dangerous belief that humans have already lost control of a future that has not yet been determined. Dismissing AI because today’s systems remain uneven in important ways is equally dangerous. We have more agency than the fatalists imagine and less time than the complacent accelerationists assume.
The arms-race dynamics relentlessly driving technological development forward make all of this more pressing. Even responsible companies and countries face powerful incentives to move faster when they fear competitors will do the same. The possibility of reaching a point where AI and robotic systems can increasingly improve themselves and act in the world with decreasing real-time human input raises the stakes still further. This is precisely why relying on the wisdom or restraint of individual actors will never be enough.
To increase our odds of getting things right, we desperately need better governance, at all levels, that can grow alongside our technological capability. Among many other things, these efforts must require meaningful safety testing before the most powerful systems are released; establish clear lines of responsibility and liability for when AI systems cause harm; create transparency standards that allow governments and researchers to understand emerging risks; and ensure that companies cannot simply externalize potentially catastrophic dangers onto the rest of society.
We need to develop common international standards for challenges that cannot be contained within national borders, from AI-enabled biological threats and autonomous weapons to cyberattacks and the destabilizing effects of increasingly capable systems. No responsible company or country can fully solve these collective-action problems alone when restraint by one actor can advantage a less responsible competitor. To avoid massive social upheaval, we must also ensure that the benefits of these technologies are shared rather than captured by a small elite, so that people feel less that the AI revolution is happening to them and more that it is happening with and for them.
We also need to invest in ourselves. Education in an AI age cannot simply be preparation for performing tasks machines cannot currently perform, because we have no exact vision of what functions machines will be able to perform in the future. We should therefore cultivate judgment, curiosity, empathy, creativity, critical thinking, courage, community, moral imagination, technological literacy and the capacity to continuously explore, learn and find meaning. These qualities will matter even more as machines become more capable. Our goal should be to become better at being optimally human in our rapidly changing world while building technologies that extend, support and inspire the best of our humanity. We will need to be wiser humans to most beneficially guide the development of smarter machines.
The future belongs neither to humans nor to machines but to both working in concert
Intelligence can help us determine how to achieve an objective, but intelligence alone cannot tell us which objectives are worth pursuing. Wisdom requires making judgments in the face of uncertainty, balancing competing values, recognizing limits, thinking across domains and generations and deciding not merely what we can do but what we should. It demands constant reinvention to adapt to changing circumstances.
The mythical AGI threshold is not coming because there is no single threshold to cross. Humans are not becoming obsolete. But as our machines grow increasingly powerful, we will need to make profound choices about what we value, how we want power and opportunity to be distributed, and what kind of future we seek to build. The future belongs neither to humans nor to machines, but to humans and machines working in concert, guided by the values we embed in that relationship.
Our machines will become smarter than they are now. Whether our lives become better will depend on whether we can become wiser and embed that wisdom in the choices, technologies, institutions and governance systems that will shape our future.
Comments