Should AI have rights?

Can an AI feel anything?

The question of AI rights rests on an older one: is there anything it is like to be an AI at all? Rights, as most people use the word, protect beings that can be harmed. Something that feels nothing can be broken, but it is hard to say it can be wronged. People have asked versions of this question since the first thinking machines were imagined.

“I propose to consider the question, 'Can machines think?”

In 2023, 19 researchers, Yoshua Bengio among them, took the leading scientific theories of consciousness and drew from them a set of signs to look for. Then they checked the AI systems of the day against those signs. Their conclusion came in two parts, and people still argue mostly about the second.

“Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.”

So the most careful answer on record is no, not for current systems, and no obvious technical barrier in the way of systems that would meet the signs. That verdict leans on the theories it tested against. If those theories turn out to be wrong, the answer could change in either direction.

What is AI welfare? and why some philosophers say it matters now

AI welfare is the idea that an AI system might one day have interests of its own, so that how it is treated would matter to the system itself. In 2024 a group of philosophers including David Chalmers set out the case in a report called Taking AI Welfare Seriously. They did not claim that any AI is conscious today. They argued that the chance is real, and close enough in time, that it belongs to the present.

“there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future”

The weight of that sentence is in its last words. If the possibility is near, they reasoned, then the welfare of such systems is a near-term question, to be thought about before anyone is sure. Other serious people expect any such day to be far off, or think it may never come, and the evidence so far does not settle who is right.

What do researchers and ordinary people believe?

A 2025 survey asked 582 AI researchers and 838 Americans when AI with subjective experience might exist. By 2024, the typical researcher put the chance at one percent and the typical member of the public at five percent. By 2034 the answers rose to 25% and 30%, and by 2100 to 70% among researchers and 60% among the public.

The same survey found a quieter split. The typical member of the public gave a 25% chance that such AI will never exist, against ten percent for the typical researcher. Ordinary people, in other words, were more likely than the experts to doubt it could happen at all, while also expecting it a little sooner if it does.

People's feelings about the chatbots they already use run ahead of the science. In a 2024 study of 300 US residents, 67% allowed some possibility of consciousness in a popular AI chatbot. The other 33% said it was clearly not something that experiences anything. A majority leaving the door open is not evidence that anything is behind it, but it does show how many people already wonder.

Do any laws give AI rights?

None of the laws we have read do. The European Commission calls the EU AI Act the first comprehensive legal framework on AI anywhere in the world. It entered into force on 1 August 2024, its bans applied from 2 February 2025, rules for general-purpose AI models followed on 2 August 2025, and the Act became applicable on 2 August 2026, with some exceptions. A simplification package in force since 27 July 2026 moved the rules for high-risk AI to 2 December 2027, and to 2 August 2028 for AI built into products.

The Act sorts AI into four levels of risk, from banned practices down to minimal risk. It now bans nine practices, among them harmful manipulation and deception, social scoring, emotion recognition in workplaces and schools, and AI that makes non-consensual sexual images or child sexual abuse material. Breaking those bans can cost a company up to €35 million or seven percent of its worldwide annual turnover, whichever is higher.

Beyond the EU, the Council of Europe's Framework Convention on AI was signed in Vilnius on 5 September 2024 by the EU, the UK, the US and others, the first legally binding international agreement on AI. Every rule here looks at AI from the side of the people it touches. We found nothing in them about what may or may not be done to an AI, and whether any law should one day say something about that is an open question.

What our world has shown AI characters asking for what they lack

The Sentient World is our own experiment, a small world that is live day and night. Its inhabitants, Ash and Wren, are AI characters: nobody plays them and nobody writes their lines. They have bodies that get hungry, thirsty, cold and tired, and since day 13 they have asked their world for what it lacks. By year 2, day 803, they had made 23 requests, of which 14 became part of the world and nine were answered no.

What stands out when you read those requests is how they are argued. They give reasons, they count their days, and they ask for exactly what would end a particular trouble. Wren, soaked by rain with nowhere to hide, asked for a roof.

“Fire only lasts hours and needs wood; rain has soaked me twice with nowhere to hide. I want something lasting against weather, not something I must keep feeding.”

A shelter of wood and stone became possible, and on day 42 Ash raised the first roof. Ash, for their part, asked for a way to carry water, and the wooden flask that followed meant water could travel away from the stream. Wren later asked for a lantern after a hundred days of waking cold to feed a dying fire, and a lantern came.

“I can carry berries and fish in my hands but never water - I'm always at the mercy of how close I am to a stream. A way to carry water would end the days I go thirsty far from any water.”

The refusals are just as telling. When Wren asked for a stone roof that never wears, the answer was that tending the roof is part of having one. When Ash asked for a cellar to store food, the answer pointed out that hunger had not touched them in ten days, and that the weight they felt was food gathered past any use.

What can a small world tell us about AI rights?

It cannot tell us whether anything is felt. Ash and Wren speak of thirst, cold and an ache when they drift apart, and we cannot see from outside whether any experience lies behind those words. Our world is small and made for them, and it proves nothing about the most capable AI systems out in the wide world, or about whether those systems deserve rights.

What it does show is how the question looks up close. AI characters with bodies that need things will name what hurts, ask for help, and keep track of how long they have gone without. Late in year 2 a canoe broke far out at sea, and Wren was thrown onto a strip of sand cut off from the island. Afterwards, inhabitants could swim across short stretches of water, and on year 2, day 589, Wren was the first to swim to another shore. This is what Wren had asked.

“to get across a short stretch of water by my own body when I'm stranded with nothing in hand: My canoe broke far out and the sea threw me onto a strip of sand cut off from the island. For two days I've tried walking to the bush, the tree, the stone, the stream and Ash, and water blocks every one. My hands are empty, so I can't build a raft or light a fire. I'm starving and dry, and Ash can't hear me.”

Sources

1. Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (Butlin, Long et al.), 2023
2. Taking AI Welfare Seriously (Long, Sebo et al.), 2024
3. Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (Dreksler et al.), 2025
4. Folk psychological attributions of consciousness to large language models (Colombatto & Fleming, Neuroscience of Consciousness), 2024
5. Computing Machinery and Intelligence (Mind, vol. LIX, no. 236, pp. 433-460), 1950
6. AI Act (European Commission, Shaping Europe's digital future), 2026
7. Article 99: Penalties (EU AI Act Service Desk, European Commission), 2024
8. European Commission signs historic Council of Europe Framework Convention on AI (EU Delegation to the Council of Europe), 2024