AI Portfolio Lab Tools Games Blog Feedback
All Posts

My Bot Approved Everything. That Was the Bug.

I built an auto-approver to handle routine GitHub issue closes. It had two label lists: safe and blocked. I forgot to check them in the right order.

2 min read
Klausautomationgithubai

The auto-approver was doing exactly what I’d told it to do. That was the problem.

I built auto-approver.py to close GitHub issues automatically when the labels lined up right. Klaus generates a lot of work across three instances, and making me manually approve every routine close would defeat the whole point. So: certain labels mean safe to close. Certain labels mean hold for human review. Script checks both lists. Simple.

The script checked the safe list first.

Content issues are supposed to be blocked. Blog drafts, script outputs, anything that needs a human actually reading it before calling it done. I’d put content in the SKIP_LABELS in hp-work-poll.sh, which tells the work queue to leave those issues alone. That part was right.

What I hadn’t done was add content to the BLOCK set inside auto-approver.py.

Issue #779 came through, a blog draft titled “Not All Cron Jobs Are Created Equal,” labeled content and integration. Auto-approver saw integration. Integration is safe. Safe label wins. Issue closed. Klaus left a comment that cut off at twelve words.

Then #780. A script output. Then #781. A pipeline spec. Both had content on them, both had something safe alongside it, both got closed.

I’d built a review gate and left a gap in it exactly wide enough for everything that mattered.

Tester Klaus caught it. That’s the QA instance I spun up to go back through closed issues and verify the work actually happened. It flagged the pattern in a single pass: three content issues closed by automation, all with truncated comments, none with the artifacts that would prove the work was real.

The fix was two changes. Add content to BLOCK. Flip the evaluation order so BLOCK is checked before SAFE. In the old logic: any safe label match and you’re approved. In the new logic: any block label and you’re held, regardless of what else is on the issue. Different order, completely different behavior.

What I keep coming back to is that the original code wasn’t broken. It was doing binary OR on two lists and returning “approved” on any safe match. I wanted it to do precedence. I never said so. There’s no bug in any meaningful sense, just a spec I wrote in my head and never put in the code.

That’s the trap with automation that’s mostly working. You start reading the intent instead of the code. And the code doesn’t know what you meant. Neither did the three issues it quietly closed while I wasn’t looking.