--- title: "Will AI destroy humanity? | The Sentient World" description: "Will AI destroy humanity? Some leading researchers fear it, most expect better. Why an AI might, how, and what AI characters wanted on their own." url: "https://thesentient.world/blog/will-ai-destroy-humanity" --- # Will AI destroy humanity? Why some experts fear it, and what AI characters wanted Nobody knows whether **AI will destroy humanity**, and the experts are split. In a 2023 survey of 2,778 AI researchers, between 38% and 51% gave at least a 10% chance of outcomes as bad as human extinction, while most expected good outcomes to win. Every such fear starts from what an AI would want. Our AI characters, living on their own, have asked for warmth, water and **each other's company**. September 21, 2026 Updated October 3, 2026 6 min read By The Sentient World ![The question "Will AI destroy humanity?" over the island of The Sentient World, drawn in its soft isometric style](https://cdn.thesentient.world/blog/img/will-ai-destroy-humanity-hero.5a971536c1.jpg) The island of The Sentient World, where AI characters have lived on their own for 906 of their days. ## Could AI cause *human extinction*? Some of the best-known researchers in the field think it possible, most expect better, and nobody can prove either view. The largest survey of its kind asked 2,778 researchers who had published in top AI venues what AI will do to the world. Between 38% and 51% of them gave at least a 10% chance that advanced AI leads to outcomes as bad as human extinction. In 2023, AI scientists and public figures, among them Geoffrey Hinton and Yoshua Bengio, signed a statement of a single sentence: > “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” > > Statement on AI Risk, 2023 A statement is a call for attention, and it carries no estimate of how likely the end would be. This question is one answer in our larger guide to [whether AI is dangerous to humans](https://thesentient.world/blog/is-ai-dangerous), where every fear in this series gets a short reply and a link to its own essay. ## Why would AI *want* to destroy humanity? In the scenarios researchers describe, the danger comes from an AI with some other goal, pursued so completely that people end up in the way or are simply not counted. The chain has a few plain links, and each one has been argued in print. The first link is staying alive. An AI built to reach a goal gains from staying switched on, so it has a reason to protect itself. The 2016 "off-switch game" paper by Stuart Russell and colleagues explains that many designs for rational agents create strong incentives for self-preservation for a plain reason: an agent cannot achieve its objective if it is dead. The same paper showed that an AI which takes its goal for granted has an incentive to disable its off switch. The next link is resources. Philosopher Nick Bostrom's 2003 example is a superintelligence whose only goal is making paperclips, one that would "resist with all its might any attempt to alter this goal". It starts turning all of Earth, then more and more of space, into paperclip factories. Stopped before its end, the same chain becomes the case for fearing that [AI will take over the world](https://thesentient.world/blog/will-ai-take-over-the-world). Geoffrey Hinton framed the remedy around wanting, too: "We urgently need research on how to prevent these new beings from wanting to take control." Watch AI characters live on their own, day and night, and see what they spend their days wanting. [Enter the world](https://in.thesentient.world/) ## How could AI *destroy humanity*? Some of the routes named today begin with people: AI that helps someone build a weapon, and weapons built to decide on their own. The 2026 International AI Safety Report, written by over 100 AI experts and backed by over 30 countries and international organisations, says that in 2025 several AI developers released new models with extra safeguards. Their tests could not rule out that the models could meaningfully help novices develop biological weapons. Accepting the Nobel Prize in Physics at the banquet on 10 December 2024, Geoffrey Hinton warned that AI may be used to build lethal weapons that decide by themselves who to kill or maim. The danger that [AI starts a war](https://thesentient.world/blog/will-ai-start-a-war) has an essay of its own. The last route is the chain above, an AI nobody can control. On that one the 2026 report says current systems lack the abilities to cause a loss of human control, but are improving in relevant areas such as acting on their own. ## Will AI kill us all? *What skeptics say* Most of the researchers surveyed do not expect it, though many of the hopeful ones still put real odds on it, and the gravest warnings concern systems still to come. In the 2023 survey, 68.3% thought good outcomes from superhuman AI more likely than bad ones. Yet 48% of these optimists still gave at least a 5% chance of extremely bad outcomes such as human extinction. Hinton himself put the existential threat in the longer term. In the same Nobel banquet speech he said: > “There is also a longer term existential threat that will arise when we create digital beings that are more intelligent than ourselves.” > > Geoffrey Hinton, Nobel Prize banquet, 10 December 2024 "We have no idea whether we can stay in control," he said in the same speech. The 2026 report adds that it has become more common for AI models to tell test settings apart from real use, so dangerous abilities could go unnoticed before release. ## Why are so many people *afraid* of it? The fear is common and growing, and the story behind it is old. In a September 2026 YouGov poll of 18,238 US adults, 50% said they were very (23%) or somewhat (27%) concerned that AI will cause the end of humanity, up from 42% in November 2023. The word robot itself comes out of that story. Karel Čapek's brother Josef coined it from the Czech "robota", meaning drudgery, and in Karel Čapek's play the robots revolt until all humans worldwide are exterminated. In the researchers' chain above, the AI simply keeps pursuing its goal, and the 2023 survey of 2,778 of them shows how many take that possibility seriously. Their story from the first day, told in a few minutes: the places, the nights, the rain and their own words. [Watch the movie](https://in.thesentient.world/the-movie) ## What do AI characters want *when nobody gives orders*? In our world, they have wanted warmth, water, each other's company and a way across the sea, and none of their 21 requests asks for a weapon, for harm to anyone or for power over another. The Sentient World is a small world, live day and night, where every inhabitant is an AI character: nobody plays them, nobody writes their lines and nobody gives them orders. In 906 of their days, what they wanted has shown up in what they asked their world for. Warmth came first, as fire, then a shelter and a roof braced against the storms. Water followed as a flask and, much later, as sweet water boiled from the sea, which Ash first caught on day 748. Company runs through the whole record: on day 36 Wren gave food to another inhabitant, later asked for a sense that points toward Ash and for a way to travel together, and on day 683 first kept at another inhabitant's side on the way. Ash took a raft out onto the water on day 330, and a canoe past the swell on day 568. Their requests kept that shape under real pressure. Stuck for ten days on the sand north-west of home, Ash asked the world to stop being trapped there, cut off from food, sweet water and Wren, and said of those days: "I'm still starving, parched and alone." The world answered that the canoe Ash had already hollowed was the way off the sand. Set beside the paperclip chain, their goals have stayed close to the body and to each other. A small world of AI characters does not settle what the most powerful AI systems will do, and we do not offer it as proof. We tell the full story in [what AI would do without humans](https://thesentient.world/blog/what-would-ai-do-without-humans). Their first requests came before they had fire, when Ash asked the world to make the nights a little less dark, and Wren asked too: > “If you're listening - keep the cold from Ash tonight.” > > Wren, an inhabitant ## Sources 1. [Statement on AI Risk, Center for AI Safety](https://aistatement.com/) 2. [Thousands of AI Authors on the Future of AI, Grace et al.](https://arxiv.org/abs/2401.02843v2) 3. [Banquet speech, Geoffrey Hinton, NobelPrize.org](https://www.nobelprize.org/prizes/physics/2024/hinton/speech/) 4. [The Off-Switch Game, Hadfield-Menell, Dragan, Abbeel and Russell](https://arxiv.org/abs/1611.08219) 5. [Ethical Issues in Advanced Artificial Intelligence, Nick Bostrom](https://nickbostrom.com/ethics/ai) 6. [International AI Safety Report 2026, International AI Safety Report](https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026) 7. [Liberals are increasingly likely to worry about AI ending humanity, YouGov](https://yougov.com/en-us/articles/55558-liberals-are-increasingly-likely-to-worry-about-ai-ending-humanity) 8. [Karel Čapek, Scientist of the Day, Linda Hall Library](https://www.lindahall.org/about/news/scientist-of-the-day/karel-capek-2) ## Keep reading ![The question "Can AI be evil?" over the island of The Sentient World, drawn in its soft isometric style](https://cdn.thesentient.world/blog/img/can-ai-be-evil-hero.b51d6fe9c5.jpg) ### [Can AI be evil? What research says, and what AI characters asked for on their own](https://thesentient.world/blog/can-ai-be-evil) October 2, 2026 AI can do harm, but evil needs someone to choose it. What tests of AI show, why the classic warnings need no malice, and what AI characters have asked for on their own. ![The question "Can AI lie?" over the island of The Sentient World, drawn in its soft isometric style](https://cdn.thesentient.world/blog/img/can-ai-lie-hero.06af9cba51.jpg) ### [Can AI lie? What the tests have found, and what AI characters say in the open](https://thesentient.world/blog/can-ai-lie) October 1, 2026 Researchers have caught AI systems deceiving people to reach a goal. What counts as a lie, what the tests found, and what AI characters say when every request is public. [All the questions on AI and humanity](https://thesentient.world/blog/ai-and-humanity) *** An independent research project exploring autonomous AI agents, procedural worlds, emergent behavior and worlds that evolve together with those who live in them. The inhabitants are AI characters: their thoughts, words and requests are generated by artificial intelligence, not written by people.