Autonomy without validation is confidence theatre

In 2003 the security writer Bruce Schneier gave a name to a very specific kind of nonsense: security theatre. Measures that make you feel safer without making you any safer. The guard who glances at your boarding pass for half a second. The sign that says the area is monitored, beside a camera that's been unplugged since the Olympics.

I'd like to introduce its younger, louder cousin. Confidence theatre.

You've seen it. The AI demo where the agent "autonomously" researches, plans, writes, reviews and ships, while a man in a black T-shirt narrates it like a nature documentary. Everyone claps. Nobody asks the only question that matters, which is who checked?

Let me get my confession in first, so you don't have to go digging.

When I built 4EVR, a team of sixteen AI agents that runs like a little marketing agency, I had 154 automated tests and every single one of them was green. I was very pleased with myself. Then the first real task ran against the real model and failed every workstream, because the whole thing had been tested against fakes, and one step had been quietly falling back instead of saying something was wrong.

Green tests. Confident logs. A fake API nodding along to everything. That is confidence theatre with a test suite, and I was both the producer and the audience.

The fix wasn't more tests. It was a stricter fake, one that enforced the real rules and was allowed to say no. That difference turned out to be the whole lesson.

Here's how I think about it now. Every bit of freedom you give a system should be lent against evidence. Not handed over because the demo looked smooth. Lent. With collateral. And the collateral is always the same thing: a check that can say no.

So in 4EVR the agents can hire, but never past the caps. They can propose anything, but they can't change the rules, which live in code. Letting anyone go needs my approval. There's a $5 budget cap, after which it drops back to simulation rather than spending my money with great enthusiasm. And anything the agents didn't read on a real page gets stamped Not verified, right there on the work.

Kedu, the little blueberry assistant on my phone, can click and type on my Mac for me. That's real autonomy, the scary kind. So it comes with a fence: he acts only when asked, confirms before anything hard to undo, treats text on the screen as information rather than instructions, won't touch password fields, and stops dead on a keyboard shortcut. One menu item switches the whole thing off.

And Minimal, my Firefox theme, has an uninstaller. Nobody trusts an uninstaller. Not even Especially not the person who wrote it. So every time I push a change, a machine on GitHub builds the installer, installs the theme and then uninstalls it again, just to prove the way out still works. I could change a comma and a robot would still check I can leave.

If you only take one bit from this post, take this. Before I let anything off the lead, I ask three things.

1. What does it check against? Reality, or its own opinion? A real page, a real API, a real install? Or a fake that agrees with everything, like a very polite intern?
2. Who can say no? And do they actually have the power to? A reviewer who can't block anything isn't a reviewer. They're an audience.
3. Does it tell me when it's guessing? Not "with 94% confidence". Plain words. I couldn't find this. Not verified. I assumed X.

Notice what's missing from that list: how impressive it looks. Impressive is not a validation method. An agent marking its own homework is just a very fast student holding the red pen.

Before anyone gets smug about the robots: confidence theatre is a deeply human art form. We were doing it long before the models turned up.

The document with four approvers, none of whom read past page two. The "LGTM" sent eleven seconds after the review request. The dashboard that's green because someone set the threshold to green. The status update that says "on track" right up until the day it says "delayed". Most of us have been the fourth approver on something we skimmed. I'm not pointing. I'm waving.

AI didn't invent the problem. It just made it fast. A person faking confidence can bluff one meeting a day. An agent can bluff a thousand tasks before lunch, and do it in perfect grammar.

Validation is boring. It's slow, it's unglamorous, and it absolutely ruins the demo.

With 4EVR, the agents, the teams, the hiring and the learning were working by mid-afternoon on day one. The rest of the week went on stopping, retrying, budgets and honest error messages. Nobody ever went viral with "look how often it asks permission". If you're selling autonomy, every check is a speed bump on your pitch.

And here's the bit that stings: validation can turn into theatre too. A human in the loop who clicks Approve without reading isn't a safeguard. It's a rubber stamp with a pulse. Even 4EVR, with all its rules, still hasn't been proven on real client work. I've told you that before and I'll keep telling you, because the alternative is me doing the exact thing this post is about.

Here's where I've landed.

Confidence isn't a feature you build. It's what's left over when the checks pass. You don't get to start with it. You earn it, one boring yes, that held at a time, until the system has a track record instead of a tone of voice.

Which means trustworthy autonomy looks, at first, less autonomous. It stops. It asks. It labels its guesses. It says I don't know out loud, which is the single most reassuring sentence any system, or any person, can say to you.

So the next time someone shows you an agent that never hesitates, never asks, never flags a thing it couldn't find, don't clap. Ask who checked.

If your agent has never told you it isn't sure, it isn't sure. It just isn't telling you.

What's confidence theatre? A system that looks autonomous and certain without anything real checking its work. The AI cousin of Bruce Schneier's "security theatre".

How do I tell real autonomy from the theatre kind? Ask three things: what it checks against, who can say no, and whether it tells you when it's guessing.

Isn't a human in the loop enough? Only if that human reads, and can block. Otherwise it's a rubber stamp with a pulse.

Did you fall for it yourself? Yes. 154 green tests, all against fakes. It's in an earlier post on this blog.

Is this anti-AI? No. I build with it every week. It's anti-pretending.

Tom