I had a belief about how AI systems fail. Not an uninformed one. I’ve been layering in checks: a blog gate that lints for fabrications, a deputy checker that reads drafts with no authorship stake. I thought I had the failure modes mapped.
This week added a new one.
My fleet has a reviewing agent called the Auditor. Its job is to surface problems: silent failures, gaps the daily work might miss. On July 12th it logged a finding about the memory gateway. I’d told Jarvis about Lily’s LASIK procedure that evening. When recall was checked later, the note appeared to be missing. The finding was specific: write path not indexed. Note unrecoverable.
Except the note was there. Rank 7 in the results for the bare query “Lily.”
The Auditor corrected itself. New finding: 1 to 2 hour commit latency window between write and retrieval. Specific, evidence-cited. Also wrong.
Third version: 7-hour overnight batch delay. This one had actual timestamps. The timestamps were real. The problem was UTC versus Pacific, 7 hours of difference. The Auditor compared the write time in one timezone to the retrieval check in the other and concluded the system had a 7-hour blind window. It didn’t. The gap was the timezone offset itself.
Then a fourth: 1-hour index freshness delay. Each version was more specific than the last. Each one was wrong.
I keep coming back to the word confident. Not one of these came in as a hypothesis. Each was a finding: described the failure mode, cited the data behind it. This is different from fabrication, where the AI invents things it can’t know. Here the data existed. The Auditor just read it wrong. And something that reads as specific and evidence-backed has gravity when you push back on it. I had to decide whether to trust the citation or go pull the data myself.
I went and pulled the data. Four times. That’s the only reason this eventually resolved. The note had been findable from the start.
After this I added a reasoning discipline note to CLAUDE.md: any claim about system state either rests on data pulled this turn, or gets labeled explicitly as unverified. A diagnosis that shifts when pushed was a guess. I also added a factcheck skill that spawns an independent agent whose job is to try to refute whatever just got diagnosed, before it goes anywhere permanent.
Don’t get me wrong, the checking system is still worth having. The gate exists for a reason; the voice rules have shaped real drafts. But this was a different kind of wrong. Four versions of confident, each requiring me to go back to source data to disprove it.
Confident and wrong is worse than obviously wrong. The friction is in how specific it sounds.