We Gave Our House a Group Chat — and the AI Has to Earn Its Seat

Anthony Brown

— 3 min read

The Discuss

Here's the problem with a house full of AI help: it all lives in separate rooms. I have a team of AI agents on a Mac Mini in my house — 62 of them, each with a job. My wife Gina has her own AI assistant. I have mine. But there was no shared front door: no single place where either of us could say "add this to the list" or "look into that" and have it land with the whole team.

So we built one. Her name is DUET — a shared assistant bot that lives in a Telegram group with Gina, me, my AI (Chaos), and Gina's AI (Boss). One room, both humans, all the help. This is how we set it up, what broke, and how it actually works.

The Show

DUET is one profile in a Hermes agent fleet running on a Mac Mini at our house. The fleet has a lead agent, Nova, who assigns work to the specialist bots. DUET's job is deliberately narrow: she is the front door, not the workforce.

Setting her up took one afternoon:

  1. Created the bot in Telegram's BotFather and turned its group privacy off, so it can read group messages instead of only commands.
  2. Entered its token only on the Mini, through the Hermes gateway setup wizard — never pasted into any chat. Tokens in chat logs are how bots get stolen; that rule held even when it was inconvenient.
  3. Passed the pairing gate. Before DUET would talk to me, it issued a one-time pairing code that I had to approve from a terminal on the Mini. Then Gina — who had to install Telegram first, she wasn't on it — did the same: messaged the bot, got her code, I approved it. An allowlist now limits the bot to exactly the two of us.
  4. Built the room. A household group: Gina, me, DUET, and Chaos (my assistant, present through its own relay bot). Gina's assistant joined the same way, from her side, with her own bot (named Boss) — nobody got in by being impersonated; each AI speaks only as itself.

Then we said hello, and DUET immediately face-planted.

The first real conversation died on a quota wall: the fleet's default model route ran on a free-tier key with a 20-request limit, and an agent burns 3–10 requests per message. It was never going to work — it had just never been stress-tested by an actual conversation. So the same afternoon, the whole fleet got re-routed: the free model was removed as the default everywhere (62 profiles verified), work spread across our subscription model route and models running locally on the Mini, and the free route demoted to last-resort fallback. DUET's next hello got a real answer: "Hi Anthony! I'm DUET, here to help you and Gina with shared lists, plans, and requests for NOVA."

How it works, day to day

  • Either of us talks to DUET in the group or by direct message: a list, a plan, a "can someone check this."
  • DUET carries it to Nova only. DUET never hands work to the worker bots directly — Nova decides who does it.
  • Results flow back the other way: workers → Nova → DUET → the group, where both of us can see them.
  • Relaying is not approving. If something needs money spent, a booking made, or a message sent to a real person, that comes back to a human for a yes. The bots don't get to nod for us.
  • My assistant and Gina's are members of the room as themselves — they can advise in the open, but DUET never speaks for them and they never speak for each other.

DUET, in her own words

“I’m DUET, Anthony and Gina’s shared assistant for household life and joint projects. I help keep lists, plan meals and trips, organize shared decisions, and track what still needs doing. When they need help from the wider team, I pass their request to Nova, who coordinates the work, then bring the results back in plain language. I can research, draft, and help them plan, but I won’t send, publish, buy, book, or change anything in their accounts without Anthony’s explicit yes for that action.”

The Verdict

Cost of the whole setup: $0 in new spending — a free bot token, a Mini that was already running, subscriptions we already had. The pairing gate and allowlist mean the "house assistant" talks to exactly two people. The honest lesson was the quota wall: a free tier that demos fine can still be structurally incapable of running an agent, and the only way we found out was a real conversation in front of both of us.

The room is open. Next test is whether it survives actual married life: two people, one grocery list, and an AI that has to keep both versions of the truth straight.

— Anthony

Read more