Myth vs Reality ledger

Somebody's favourite myth, tested every issue

One popular claim about building software with AI agents per issue, put up against the evidence. Most don't survive. The ones that do get a fair hearing.

What people say

Don't worry, the agent asks before it runs anything. A human approves every command.

Myth

The human approves the command; the damage is in what the command sets off. Sandboxes and effect checks keep you safe. The click keeps you busy.

What people say

If the AI makes up a package, the install just fails. No harm done.

Myth

A made-up package fails to install only until somebody registers it, and the AI will tell anyone who asks which name to register. It's a wrong number that a stranger can go and buy.

What people say

Agents score 99% on SWE-bench now. Coding is a solved problem.

Myth

A benchmark gets solved. A job doesn't. Ninety-nine percent on a cleaned exam tells you the student can pass that exam. It tells you nothing about the shift at a real hospital.

What people say

Ninety percent of agent PRs get merged. Agent code is production-ready now.

Myth

Ninety percent of agent PRs got approved. Nobody has measured how many were right. A merge is a signature, not a test result.

Last updated . Reuse with credit under CC BY 4.0: “The Dabbawala Protocol, thedabbawalaprotocol.com”.