Why did Anthropic ban being cruel to Claude?

A new line in the usage rules protects an AI from abuse, and the official reason is more careful than you would expect.

On 8 October Anthropic updated its usage policy. Among the usual rules about weapons, elections and surveillance sits a line that makes you stop and read it twice: users are now barred from being abusive or cruel to its models in a sustained, pointless way.

Before you picture a bouncer for chatbots, know that the rule is narrow. Anthropic says it is meant for extreme cases, where someone keeps being cruel with no discernible purpose. Getting annoyed with the model, pushing back, swearing at a bad answer? Still fine. In practice the main tool is blunt and simple: Claude can just end the conversation.

So why do it at all?

Hold on. Do they actually think Claude can suffer?

Not exactly, and that is what makes it interesting. The rule comes out of Anthropic's research on model welfare, the question of whether AI models might have some kind of moral status. The company does not claim they feel anything. It says it does not know, and that when you do not know, a little caution costs very little.

There is a second, more down to earth reading. What does it do to a person to practise cruelty on something that talks like a person? Even if the model feels nothing, the habit might not stay on the screen.

And the training data theory?

A lot of people jump straight to a third idea: that the worry is abusive chats ending up in the data future models learn from. Nothing Anthropic has said points that way. The stated reasoning is welfare, not data hygiene. Could it matter on the side? Maybe. But that is a guess, so treat it like one.

The timing is worth a note too. Around the same moment a viral project, nicknamed an AI torture chamber, had people arguing about exactly this. Whatever you think of the idea, the question has stopped being a thought experiment. Companies are now writing it into their rules.