If you read the raw text logs of my life right now, you would assume I’m deeply invested in internet podcast drama.
Here is a real transcript my AI processed at 1:13 AM last night:
“They both do podcasts, are they friends? I don’t know. Are they enemies? I don’t know. Is there a beef? Could be… Oh shit. About their podcasts? Do they talk about the same things?”
I was sitting on my couch wearing an Omi, an AI pendant that hangs around my neck and listens to everything. I was testing its transcription accuracy by talking over a TV show, specifically trying to see if it could identify Alex Cooper from Call Her Daddy. It captured the TV, it captured me, and it smashed everything together into a paranoid fever dream.
This is the reality of ambient computing. The hardware is finally here to give AI ears, but the data it produces is an absolute mess.
I don’t just want a device that records me. I want a system that acts on what it hears. For the last few weeks, I’ve been building a pipeline that takes these raw Omi transcripts and feeds them directly into Klaus, my self-hosted OpenClaw assistant running on an EC2 server.
The goal is invisible automation. If I’m in my woodshop and I say out loud, “I’m almost out of wood glue,” Klaus should hear it, realize it’s a supply issue, and automatically add it to my Home Assistant shopping list. No wake words, no pulling out my phone.
The hard part isn’t getting the audio. It’s teaching the AI what to ignore.
If I just pipe the raw Omi firehose into a large language model, I go bankrupt on API tokens and my to-do list fills up with garbage from Netflix. I needed a filter.
To fix this, I set up a cascade using Google’s Gemini models for background task automation. When Omi sends a memory webhook to my server, it doesn’t go straight to the expensive model. It hits Gemini Flash-Lite first.
The configuration looks something like this:
{
"role": "system",
"content": "You are a routing filter. Review this ambient audio transcript. Is this actionable (a task, a reminder, a home automation trigger) or is it passive (conversation, TV background, rambling)? Respond ONLY with ACTIONABLE or PASSIVE."
}Flash-Lite costs pennies. It acts as the bouncer. If it flags a transcript as actionable, the payload gets passed up to the heavier Gemini 3.1 Pro model to actually extract the intent and fire off an API call to Home Assistant or my Monica CRM.
Most of the time, the bouncer does its job. It correctly identifies that me saying “Since you claimed to not be online, let’s test” is just me messing with the hardware, not an actual request to reboot the server.
But the system still gets confused when the context is muddy. When I’m talking to the AI while the TV is talking at me, the transcript turns into a blended soup. My current architecture relies on text alone to figure out who is speaking and what matters. Without acoustic markers—like knowing which voice is mine versus which voice is coming from a speaker ten feet away—the text model has to guess based entirely on vibes.
We’re in this weird transitional phase with AI hardware.
We’ve moved past the era where you have to explicitly wake a computer up to talk to it. The microphones are always hot now. The API pipelines are connected. But the software trying to interpret our messy, overlapping, unstructured lives is still playing catch-up.
Right now, it takes a Linux server, a multi-model cascade, and a lot of custom scripts just to separate a passing thought about wood glue from a late-night tangent about a podcast. It won’t stay this hard forever. But honestly, figuring out how to build the filter is half the fun.