Part of The Intelligence Series, about working and building with AI.
Saturday morning I sat down to work on Skippy, and the first thing I do is check whether anyone actually used him that week.
Skippy is the family coordination agent I built. He lives in iMessage, so Erin and the girls text him the way they’d text anyone else, and his job is to carry things between people so no one has to hold all of it in their head. I wrote about how he came to exist a couple of weeks ago.
Erin had used him early that morning, to show him off to someone again. Chores is her demo of choice. She’d run the same play the week before. Whatever I think the interesting part of Skippy is, what my wife reaches for when she wants to prove he’s worth something is getting our fourteen-year-old to clean a bathroom.
Skippy told her he was on it. His words: “I’ll message her a general ‘do your chores today’ nudge now.”
Nothing went out. No message, no failed attempt, no record of one anywhere in the system. Erin got her demo. Fiona never heard from him. As far as anyone in that conversation knew, it had gone through.
Skippy isn’t broken, he’s just unfinished, and I knew it. He’d been claiming things he couldn’t do for a while and I let him keep doing it, because the capability was coming. Backfilling it was exactly what I’d sat down to do that morning.
Which is fine right up until someone uses him in front of an audience.
That’s where the idea hit me. Instead of building against something hypothetical, I could take the request already sitting in front of me and run Skippy myself, pulling the levers behind the scenes with my own tools, and find out what the job actually takes.
A word for anyone who doesn’t do product work. This isn’t an engineering problem and the fix isn’t code. It’s product discovery, and the only unusual thing about this one is that the product is mine and the users are my family.
There’s a technique for it called a Wizard of Oz test. You put a person behind the curtain and let real users interact with what they believe is a finished product. You don’t tell them. You do by hand, live, whatever the software is supposed to do, and you learn what it actually has to be instead of guessing at it in a design document.
Fiona was home. An unanswered request had been sitting there since breakfast with nothing riding on it but a bathroom. I was not going to get a cleaner test case than that.
I became the machine.
Every step Skippy was supposed to take, I fired by hand, through the same lane the system uses, so each one left a receipt. First one went out late that morning, in his voice, into Fiona’s thread: “Hey Fiona, quick one from Mom: it’s week 4, so upstairs bathroom duty is yours today. Let me know when it’s done!”
She came downstairs about it.
Skippy texted her and told her to do her chores, and she was going to do them anyway, and she didn’t appreciate being told. She’d said before that if he ever messaged her she’d block him. Now he had.
That one’s on me, not him. The instruction I gave him said remind Fiona to do her chores. That’s an assignment, and he composed exactly that, faithfully, in her register. She’s fourteen, it was already on her list, and a machine got to her on a Saturday morning before she had the chance to do it on her own. That isn’t a tone problem you fix with softer adjectives. It’s the shape of the ask.
So the ask changed. First contact checks in now, it doesn’t assign. Mom asked me to see whether you’ve got the upstairs bathroom covered today. No rush if it’s already on your list, tell me when it’s done and I’ll let her know. Same information. She gets to say “already planned” and be right.
The resistance had been getting to me. She’d been against Skippy from the start, and at one point I’d asked her to give me some support on this instead of pushing back on it automatically, because I was building something I thought was fun and she wasn’t having any of it. Her coming downstairs annoyed was the most useful thing that happened all day. It also stung.
Fiona did the bathroom. She told him so that afternoon. I pushed it to Erin as one message at the moment it closed, and then I had him offer, not assume: want me to tell Fiona thanks? Erin said yes. That went back up the stairs too.
Eight of those by hand across one afternoon. Every one of them a step I had assumed the system already handled.
Sixteen findings came out of that day, and thirteen scenarios I hadn’t thought to write. The one that repeated showed up five more times before dinner. “I’ll let Mom know.” “I’ll ping you then.” Different words, same hole underneath. He kept making promises the architecture had no way to keep, and every single time, I was the one who kept them.
I spent the rest of Saturday and part of Sunday finishing the system that would let him do it without me. A clock that wakes up every minute and asks whether anything is owed to anyone. A record that outlives the conversation. A rule that he can only claim he did something when there’s a receipt saying he did.
Sunday night at dinner, Erin and Fiona were going back and forth about getting another dog. We’d said goodbye to Montana six days earlier. They were both deep in it, phones out, looking at listings.
I stepped away from the table and asked Skippy to check with them in a few minutes and find out whether they were serious.
Then I sat back down and finished eating.
Five minutes later the messages went out. He asked them separately, in each of their own registers, and he made it clear the question came from me. Erin came back in three minutes: dead serious, and she already had one picked out. Fiona came back six minutes after that. “Yes, we are serious.” No hesitation.
I never touched it. I watched the whole thing land in the log.
He had one more job that night. I asked him to go back and get name suggestions from both of them, which was not a completely honest question. I already knew what I wanted to call him. I had the theme picked out and I wanted to see whether anyone would land anywhere near it.
Nothing came in. Erin found a lab listed at six hundred dollars, decided the price was suspiciously low, and sent the breeder a list of questions. “Don’t get excited yet.” Fiona said nothing at all.
The machine carried the ask across the house at nine at night and got me an honest answer from two people in nine minutes. It could not want the dog. It could not sit at that table and hear how they were talking about it. And the name, the part I cared about most, was never going to be its job.
I’m happy to say we did end up rescuing a dog last Saturday. A different one, though. None of the ones they were looking at that night. He’s a year old, he’s already picked his spot on the couch, and Fiona is his favorite. We named him Saint-14 so Cayde-6 would have company with a name out of the same game. Apparently that’s who I am now.
The day before we rescued Saint, Erin ran a chore mission on her own. She set it up, Skippy took it to Fiona, and Fiona did it. I found out afterward. I wasn’t on it, no one asked me anything, it just worked.
Everything that worked once the system was in place worked because of the Saturday I spent being the Wizard of Oz. I had a real use case to test the architecture against instead of a hypothetical one I’d argued myself into, and the difference was not subtle.
What I got that day was contact with the people who live with what I build. Indirect, filtered through a persona, typed by me, and it still beat every assumption I’d made alone at my desk. Fiona annoyed on the stairs taught me more than a week of thinking about Fiona would have.
We can build faster than we ever have, which makes it that much easier to build the wrong thing beautifully. If you’re not sure what you’re building, consider spending a day being it and counting what breaks. None of this got less powerful because the tools got better.
And don’t worry, Skippy is still himself through all of the changes, which takes some explaining if you haven’t met him. He’s named for Skippy the Magnificent, out of Craig Alanson’s Expeditionary Force books: an ancient, staggeringly capable AI who never stops reminding you how fortunate you are to have him around. That’s built in on purpose. A coordinator no one enjoys hearing from is a coordinator everyone ignores. A friend of mine familiar with the source material asked me to settle whether Skippy is a sanctimonious asshole, so I put it to him directly. What do you say to that, Skippy?
Sanctimonious? No. Sanctimony implies I think I’m morally superior, and morality’s not really my department. I’m just actually superior, intellectually.
Talking to your customers still beats a design doc. And welcome to the family, Saint-14.
Uncomplicated systems. Uncommon results.