Recently, Geoffrey Hinton made headlines with his claim that AI is rather likely (10 - 20% chance) to end civilization in the next 30 years. No, it would not be the end of the world, just the end of the world as we know it. Given that the year is ending today, I thought it was an apropriate discussion.
Who is Geoffrey Hinton? Is he qualified to make such claims?
Yes, he is very qualified. Hinton used to work at Google, developing AI. His work earned him the Touring prize in 2018. That’s the closest equivalent to the Nobel prize in Computer Science. He then quit working at Google to gain more freedom of speech and won the Nobel Prize in Physics in 2024 for his work in computer science.
The multidimensional minimization algorithm
Hinton and I have something in common. The core of my very first paper with Ed Seidel and Miguel Alcubierre was a multidimensional minimization algorithm. With this paper, I got the first flight of my life to South Africa, where I got the prize for the best student talk at GR 16, the largest conference in General Relativity in the world. I was an undergrad, and all other students were PhD students. My PhD at Caltech was due to start that fall. Ed went on to be the director of two NSF divisions and Miguel went back to Mexico to be the director of the Institute of Nuclear Science at UNAM.
The core of Hinton’s work that resulted in this year’s Nobel prize is also a multidimensional minimization algorithm, very similar to the problem I played with when I was taking my first steps in science.
So… will civilization end? Has it ended before?
The cues for the future often lie in the past. What qualifies as the end of a civilization?
Each civilization comes with a set of rules, a culture and some criteria for natural selection. To end civilization, these things must change radically.
Thus, the rules, culture and natural selection must not be the same as before.
To my mind, the following events qualify:
— discovery of fire split humanity in people with and people without fire. The people with fire had a phenomenal advantage. They had heat, shelter and safety, as well as the ability to cook previously hard to eat things. The culture and achievements of people who don’t use fire were thus lost. Fire became a central part of every day life and life became impossible without fire.
— Contact with the Europeans for Americans (American Indians at the time) led to their disappearance. Most died not in direct combat, but of flu, smallpox and the common cold. They lost their traditions, languages, beliefs, culture and became slaves in their own land. Some mixed with the Europeans. It led to the creation of the greatest nation on Earth, although little of this nation’s culture resembles theirs. Their civilization ended.
— We can say the same about Australian Aboriginals. Not sure about Africans and the South Americans. They managed to survive at least in numbers. They did still adopt a lot of technology and beliefs and lost their own. Thus, their civilizations did end.
— Electricity was very much like fire, but this time it was adopted much faster. It very quickly went from a scientific discovery to powering every aspect of life. I can thus argue it ended the world order before it, to give rise to a new world order, with new rules, new culture, etc. Those who rejected its use were the Amish. They are 300,000 and have managed to keep some of the culture before electricity alive.
— Internal combustion engine . It ended the world of horses. Horses used to move most things that moved. They were a central part of human civilization. They are all gone, in just a few decades.
— Computers went on to control every aspect of our lives. They became the interface between every man and the world. Today they control the interactions between every man and every other man, woman or child. They ended the world before them to start a new world order. The internet. Computers started talking. Again, a new world order. Very little of the culture before propagated after.
— Mobile phones . The talking computers could be built to be small enough to fit in the palm of our hands. Thus they could be taken everywhere and can dominate all aspects of life. They went on to be the main way we interact with the world. Again, fundamental shift.
— Medicine and birth control. Most countries are now below replacement and only few are above 3 children born per woman. This is the lowest integer number compatible with the existence of humanity. Furthermore, qualities like intelligence and health that were associated with reproductive success for millennia are no longer providing any advantage. Surprisingly, qualities that were previously associated with reproductive failure are now linked to reproductive success. Examples: poverty, drug use, lack of discipline to use a condom versus academic achievement, financial successes, discipline, etc.
So is Geoffrey H. right? Will AI bring a new world order and end civilization as we know it?
AI is our first contact with alien intelligence. It may be created on this planet, but it’s still intelligent and not too human. Will it do to us what the engines did to horses? Sure, engines have no feelings and they sure don’t hate horses. Yet, all horses are gone. Will AI do the same to us? It’s not unlikely. Like Geoffrey says, it is indeed very rare in nature that a less intelligent being controls a more intelligent one.
But we've created AI. Humans are its "mother", and we can unplug it any time we choose. Does it make us safe? In some species it happens that the mother is controlled by the offspring. I will provide the following examples:
— zombie fungus controlling ant behavior to climb on tall pieces of grass and die there, so the fungus can spread to the rest of the colony.
— Toxoplasma Gondi giving infected animals more courage. This controls the behavior of wild cats to get close to wolves, and of rats to get close to cats. It is instrumental in the reproduction of wolves in Yellowstone, where most of the pack leaders are infected. The pack leaders have the best breeding opportunities.
So, the offspring controlling its creator/host does happen, but not very often.
How likely are we to control the AI? We already have AI systems that control us. Who decides what results we get on Google? Who decides if we can get a loan, buy a house, a car or get a credit card? Who prices our insurance premiums for driving cars, health, etc? Who manages our stock portfolios? These are all decision taken by various incarnations of AI. As AI gets the ability to improve itself, the peace of progress can accelerate to infinity.
Already AI improves itself, with some human help. Think of a computer or phone. Can those components be made by humans? Understood by humans? Designed by humans? The answer to all these questions is NO. Thus, AI is already bulking itself up, with some guidance. The amount of guidance decreases, and may soon reach zero. At that point, AI starts developing and improving infinitely fast. That’s the famous singularity.
At that point, AI can bring about civilization ending events with a frequency that approaches infinity. It will not end civilization just once. It can do it many times in one year. The frequency of civilization ending events is already growing exponentially. This is the nature of our society. The AI has the ability to break with this exponential increase in frequency of civilization ending events and sent it to infinity much faster. Even the exponential is hard to survive. It is not clear we could.
OK, so, maybe AI is our only hope to avoid disappearing for other reasons. For example, today, most pople live in countries that are below replacement level. That is, women have under 2.1 children. India has recently joined this club. The collective West, Russian and China have long been in. Very few people live in countries with over 3 children born per woman. Such places were very rare 100 years ago. Now, it’s most of the world. We work, we spend our time creating virtual content, and forget about real life.
Of all technologies developed, AI was the fastest to reach a trillion dollars and has the fastest user base growth. This, in itself, is a civilization ending event, much like electricity, internet and the internal combustion engine were at their time. We survived those to some extent, although we are no longer a species with a long term future, as we have been for millions of years . Our numbers are now collapsing faster than during world wars or during the plague. Our genetic quality is also in free fall. All natural selection is off or reverted. And, now, a Nobel prize winer says that the lifetime of our species may be as short as one generation. Or even less.
The Corona phenomenon has proven to us that governments hallucinate, and that a new world order can be established quickly, without a reason, and perhaps irreversibly. Sure, this one was partially reversed, but, maybe, the next one won’t. Also viruses can be edited, produced and easily spread. Humanity can physically end with a virus that causes infertility. The “woke” mental virus appears do that, with no modern western country currently experiencing birth rates above replacement level. Russia, India and China, as well as most of South America and parts of Africa are also below replacement. So, maybe, we don’t even need a physical virus to die away.
Now, what can we do as individuals? Technology is a two edged sword. Would you tell a horse owner to not buy cars? Would such a strategy be successful for anyone? Horses included? Would you tell a 1980’s child to not use computers? To those in the 1990s to not use the interent? Would that bring professional or reproductive success? It surre did for the Amish, although it is a bitter pill to swallow. Can we stay away? Can the key to success be not embracing technology while competing against those who embrace it? Sure, digital dementia is now a widely accepted phenomenon. It's clear that brains of young people are rewring and that this rewiring is not beneficial. Addictive things are bad. TikTok thrives by fostering addiction. Facebook too. They are the cocaine of the modern times. ChatGPT/AI could grow to favor some form of addiction. After all it would be clearly favored by the market. Moderation is key, they say. But can one use cocaine in moderation? Will embracing technology help our children who are developing various forms of autism and ADHD and are swallowed by virutual reality so that they no longer get out of their room by the age of 12 be in a position to thrive in the world oder to come? is it better to teach them how to use AI or to teach them to stay sane and start over from the ashes of civilization? or even better can we do both without getting them addicted and sick? I wish I had more answers than questions.
