Teaching the House to Listen — and Look

Kith started as a dashboard on the wall. It's now something closer to a home OS: you can talk to it, it can look at a photo of your groceries and know what's in them, and an AI agent can read and write its data directly through an MCP server rather than me hand-coding every integration point. None of that was the plan going in — it's what happens when you keep closing the gap between "the house runs this" and "you can actually talk to the house."

Voice was the easy part, once the pattern existed

The first voice command — a chore marked done by saying its name out loud — took building a fuzzy-match against the live chore list, since nobody says "Clean Dryer Lint Trap" exactly as it's spelled in the database. Once that pattern existed, everything after it was fast: check the bills, check what's on the grocery list, add to it, remove from it, get a spoken daily briefing of what's due. Each one reused the same shape — authenticate, call the API, speak back a template — which is the actual argument for building infrastructure instead of one-off scripts. The tenth voice command took ten minutes. The first took an hour.

The chore-matching also surfaced a real bug that had been quietly wrong for weeks: the recurrence logic for "every Saturday" chores just added seven days to whenever the chore last got marked done, with no correction back to the intended weekday. Complete it a day late once, and it drifts permanently — the chore just becomes a "every Sunday" chore forever, silently. That's not a bug you find writing tests against a spec. It's a bug you find because the household actually lives inside the schedule, and one Saturday it just felt wrong.

The dead end that turned into the simplest integration of the night

Wiring Homey into voice control should have been the hard part — its local API is a full session-based protocol, not something a simple automation can call directly. Three approaches failed in sequence: the session handshake itself, then a legacy REST fallback, then a webhook-and-flow workaround that worked but felt fragile. The actual fix was almost embarrassingly simple: Homey's bridge app had a setting, unused, for an API key. Once it was set, the same endpoint that needed a multi-step handshake started accepting single, stateless authenticated requests — no session at all. The lesson wasn't technical. It was to keep escalating through real alternatives instead of settling for the first workaround that technically functions.

Giving it eyes

The newest piece is the one that isn't voice at all: point a phone camera at a grocery haul, and Kith turns the photo into pantry entries — item names, best-guess quantities, categories — without ever storing the photo itself. It's a genuinely different modality problem than voice. Voice is good for one clean value spoken in passing; it cannot itemize fifteen things you just carried in from the car. A photo can, in one shot, at the cost of quantity precision you can just correct afterward. Building it also meant finding my own bugs before anyone else could: a batch of full-resolution phone photos, stacked as base64 in one request, will quietly blow past a server's body-size limit — exactly the failure mode the multi-photo design invited. And a missing API key, unhandled, reports itself to a user as "your photos were unclear," which sends someone down entirely the wrong troubleshooting path. Neither bug would show up in a demo with one clean test photo. Both are the kind that only turn up when you deliberately go looking for how the thing breaks before someone else finds out the hard way.

What actually changed

The real shift isn't any single feature — it's that Kith now has a surface an AI agent can act on directly, not just a UI a person clicks through. Voice commands, an MCP server, vision parsing: all three are different doors into the same rule, which is that the house should be something you direct, not something you configure. That's a small reframing with a large effect on what "smart home" ends up meaning in practice — less a dashboard you check, more a system you tell things to and trust to get them right.