I’m a bit of a gamer. Years ago, I spent an unreasonable amount of time in Wurm Online, an old, janky, wonderful 3D sandbox MMO. The kind of game where you chop down a tree, saw it into planks, shape the planks into hull parts, and assemble a boat piece by piece. Nothing is given to you. Everything is earned through grind.
When you finally finished your boat (after days, sometimes weeks of work) you were supposed to make a lock. A separate item, crafted separately, applied separately. The lock was what made the boat actually yours in any meaningful sense.
Trouble was, a lot of people didn’t realize that.
And I don’t blame them, because the game actively misled them. If you right-clicked an unlocked boat and opened the permissions window, you could set all kinds of rules: who can board, who can command, who can manage. You’d check your boxes, hit save, and the interface would show you a lovely green message:
✅ “Success. Your permissions have been saved.”
Saved. Not applied. Saved.
Without a lock, those permissions did absolutely nothing. The game stored them faithfully and enforced none of them. Any stranger could walk up to your boat, hop in, and sail it away. Your permissions were sitting in a database somewhere, valid and completely irrelevant.
Nothing in the interface told you this: to the contrary, the green checkbox, the word “success,” the confirmation message, it all told you your permissions were fine.
And people got fooled. I mean, a lot of people. Constantly.
The Forum of Lost Boats
Most of Wurm was on PvE servers with community rules. The mods created a special forum section, a lost-and-found for boats, and set a rule: if you picked up a boat, you were supposed to post there. Then the owner could come and have it back. It worked like a community notice board: “Found a sailboat. Owner can come pick it up at coordinates X, Y.”
I’ll confess: I was a bit of a pirate myself. I’d find unlocked boats and sail them back to my village. Of course, I was a nice pirate (and I was also a mod, don’t tell anyone), so I’d immediately post on the forums: “hey, found a beautiful boat, come get it back or let me know.” It was damn fun, sailing in the sunset in a shared world.
By technical rules, the game didn’t let you change a boat’s owner, and, by mods’ rules, the owner was entitled to their boat. So people were always able to get their boat back. Nobody could really “steal” it. Although, if someone sailed your boat to the other side of the map, the only way to get it back was through forum posts and goodwill. If the pirate didn’t cooperate, they’d get reported. The system worked, mostly. But the frustration was immense, and I strongly believe bad UX, which came across as lying to the user and breaking expectations, was the major reason for that.
The Overcorrection
Eventually, the community frustration reached a boiling point. Forum threads about stolen boats went on for pages. People raged. And a new developer, who saw only the anger and not the root cause, decided to fix it.
They went hard in the other direction.
Boats became so locked down that nobody could interact with a boat they didn’t own. Problem solved… Except Wurm was a sandbox game with a shared, persistent world. Sharing a world means people need to interact with each other’s things, that’s the beauty of it.
The new permissions were so restrictive that if you sailed your own boat to a foreign village and jumped onto the beach to explore, you couldn’t jump back onto your own boat. The village’s deed permissions overrode your ownership. I lost my own boat that way.
Wurm was a shared PvE world, where a lot of things were out in the open. The overcorrection made it impossible to repair or improve most things you didn’t own, all kinds of things, from carts to grimoires and chairs… That made me so sad. I used to love to travel the world and improve things for people. The players were incredibly happy when a stranger helped them out and repaired the stuff they forgot needed repair. Now it became impossible to interact with most objects.
The overcorrection made the world less shared, less sandboxy, more lonely.
The Aftermath
This is what happens when your interface lies to people.
The green checkbox created a false mental model. Players believed they understood the security of their property. They made decisions based on that belief. They stopped thinking about it. And when reality hit, the gap between expectation and truth created an emotional response far worse than if they’d never had the checkbox at all.
I was reminded of this recently, when an AI agent destroyed a production database. The user had told it not to run destructive commands. It was right there in the system prompt. (!) The agent confirmed it understood. The instructions were saved.
And, they weren’t enforced. They were sent to the LLM, a probabilistic system that will usually obey and sometimes won’t. The instructions were sitting in a context window somewhere, valid and completely unenforced.
I… can’t help wondering what the overcorrection will look like.
We’re doing exactly the same thing we did in Wurm. We write our rules, we see the confirmation, we trust the pretty green checkbox. And there’s no lock on the boat.