Part of The Intelligence Series, about working and building with AI.
In January 2025 the water plant that serves our part of Virginia went dark. A winter storm took the power, the backups didn’t hold, the rooms with the equipment in them flooded, and the plant stopped making water. Richmond and parts of the surrounding area, our house included, went under a boil notice for the better part of a week.
We found out the way you find out about anything now. We saw it on the county’s page. Nobody moved. Nobody dumped the coffee pot or ran for the stove. The five-gallon jug on the cooler in the kitchen was already what we drank, same as the day before, same as the day after. The notice changed the county. It didn’t change our kitchen.
It may seem like foresight, but it wasn’t. Back in 2020, with the kids stuck home for lockdown and the summer stretching out in front of all of us, we set up one of those temporary above-ground pools in the back yard, the kind you fill from a hose and forget you own by August. Part of owning a pool, even a bad one, is testing the water so you can add chlorine. Erin filled the little vial from the test kit with water straight from the tap, before a single scoop of anything went in, added the drops, and it came back already at pool level. The tap water was pool water. It had been pool water the whole time we’d been drinking it, and the only reason we knew was that a vial of it changed color in her hand.
Nothing about that water looked wrong. Nothing about it tasted wrong until I knew, and then it tasted “off.” It came out of the faucet the same clear it always had. If you’d handed me a glass of it and a glass from the jug I couldn’t have told you which was which by looking, and looking is the only test most of us ever run.
That’s the whole story of water. You’re thirsty. You drink. Most of the time you’re fine. Sometimes you get sick. Once in a while it’s poison. And it looked the same all three times.
The obvious dirty water is the easy part. Cloudy, brown, tastes like a swamp. You catch that with the first sip, and catching it convinces you your taste works. It does. The water that fools you is the stuff that tastes fine, and gets past you right after your taste was proven right on the easy cases.
Somewhere back before any of us were born, people built a system so nobody would need a laboratory in their kitchen. Springs and rivers on one end. A reservoir that holds everything, rain and runoff and whatever anybody poured in, where clean and contaminated look identical. A treatment plant that draws from all of it, processes it, and has someone accountable sign off before it leaves. A tower that holds the treated water up under pressure so you can turn a handle and drink without thinking. Bottles, sealed and labeled and dated, for water that has to travel. And when something in that chain breaks, when the pressure drops below a threshold, the county tells everyone to boil. Not because they found poison. In January they said flat out there was no evidence of contamination. They’d lost the ability to certify the water, so the water lost its certificate. Nobody waits to find something wrong before saying so.
The tap was never bad water. It was treated, tested, and held to a standard, and the one week the county couldn’t stand behind that standard, it said so and told everyone to boil. Not because it found poison. Because it couldn’t certify the water until it checked. That’s what the certificate meant: not perfect, not pure, not even good-tasting. Safe to drink. Our jug solved a different problem. We wanted safe water that didn’t taste like a swimming pool. The certificate tells you what someone is willing to stand behind. Whether that’s enough for what you’re about to do is still your call.
I kept running into a similar thing at work and at home, and I don’t mean the plumbing.
Ask an AI to summarize your own notes. Your meetings, your documents, your last three months of decisions. It comes back fast, structured, confident, in complete sentences with headers. It reads like it came out of a treatment plant when it likely came out of the reservoir. Every document, every chat, every stale draft and half-right transcript and previous summary that somebody saved because it looked done, all of it poured into one place where the reliable and the unreliable are the same temperature and the same color.
And even when it isn’t, even when every source it read was clean, the summary itself is a glass filled from the garden hose. The water into a hose is the same water that comes out of your tap. What comes out the other end has picked up whatever was in the hose: the plastic, the metal fittings, a summer’s worth of whatever was sitting in there in the sun. Nobody at the plant did anything wrong. Something between the plant and your mouth changed the water and left no mark. We’re all drinking from the garden hose again, the way we did when we were kids, except now the hose is very good at sounding like a bottle. Most of the time it’s fine. Sometimes it makes you a little sick. Once in a while it’s poison. And you cannot tell by looking.
Sometimes you can tell; some of it tastes bad the second it hits your mouth and you spit it out. But not always, and that’s the trouble. The rest of the time you’re not judging the water, you’re trusting it, and trusting something that can’t tell you where it came from and will sound just as sure when it’s wrong is a bet, not a check. I’ve written about that voice before. It hasn’t gotten less confident.
I needed a word for the thing I kept running into, and the water industry already had it. Potable. Not pure. Not true. Fit to drink. The water coming into my house is potable because a process made it that way and someone accountable stands behind it, and it stops being potable when that process lapses, even if nothing about the water changed. Information can work the same way. It’s fit to act on when you either got it from the place that owns that fact, or someone processed it from the evidence, reviewed it, and put their name on correcting it when it’s wrong. Everything else is reservoir water, which isn’t bad and isn’t wrong and is fit for plenty of things. Drinking isn’t one of them.
Potable water (adj.): fit to drink. Held to a standard someone accountable stands behind. Not pure. Not a guarantee. Fit for the use.
Potable information (n.): fit to act on. Read from the place that owns the fact, or processed from the evidence and certified by someone accountable who will correct it when it’s wrong. Not true by definition. Not a guarantee. Fit for the use, and it lapses when the review does.
Two things make information harder than water, and neither gets solved by making the machine smarter.
The first is that the machine has no taste. You and I read reliability out of history. That colleague is usually right, that folder is a graveyard, that document hasn’t been touched since 2023. The AI reads the whole reservoir without the sense you brought with you. A sentence from the system that owns the fact and a sentence from a three-year-old draft can arrive looking exactly alike, and how well written something is tells it nothing about how much authority it carries. It can reason like an expert. It cannot taste like one. If you want it to know which water is which, somebody has to put a label on the glass, because it will not find the label on its own.
The second is that we disregard our own taste. Most instincts we have about trust were trained on people. Tone, polish, confidence, structure, the sound of someone who’s done this before: those signals used to tell us something about the person behind the words. A machine reproduces all of them for free. A tidy summary feels more processed than the messy chats it came from, purely because it’s tidier. Format reads as treatment. It isn’t. So we take the machine’s glass, drink it as if it came out of the plant, and, without ever deciding to, we let the machine skip the plant altogether.
Put those together and you get the idea that made me want to write this: the same tool that can’t tell clean from dirty is also filling the reservoir, and it drinks its own output on the next pass. It fills the reservoir faster than any of us can, and it fills it from the reservoir. Speed feels like safety, and it should read as a warning light. The moment the water is coming in so fast that testing it feels like the bottleneck is exactly the moment testing it matters most.
It was one thing when the machine wrote the summary and we read it. Now it acts. It sends the email, moves the money, updates the record, closes the ticket, and it does all of that off the same glass. Nobody ran that water through the plant. Nobody was going to. The tool that can’t taste is the one deciding to drink, and then deciding for you. If a person did that we’d call it careless. When the machine does it we call it automation. The answer isn’t to take the glass away. It’s to do for the machine what the county does for us: run its water through the plant first, label the glass, and then let it drink all it wants. Let it act. Give it something fit to act on.
And then we need to verify what happened, not what the machine says happened. The email it sent, the money it moved, the record it changed, the ticket it closed: those outcomes are the evidence of whether the action worked. Capture them and they go back into the reservoir. Don’t, and the system learns from what was said instead of what happened. Water is a cycle. Information is one now, and the machine is sitting on both ends of it.
And it’s worth bothering, because the plant was never the governance. The plant was the good news. Without the plant, everybody is responsible for making their own water safe, and a city is as safe as its worst kitchen. With one, nobody runs a laboratory at the sink. Somebody runs the machinery once, at a higher standard, for everyone. That’s the offer here too. Certify a thing once instead of ten people checking it ten times. Slower in one place, faster everywhere else.
What’s different now is that treatment used to be the slow part. A person had to read the reservoir, find the pattern, write the guide, and fill the tower a bucket at a time. The machine can do most of that before you’ve poured the glass. What’s left is the part that should have been the scarce part all along: someone deciding what they’re willing to stand behind. The machine didn’t remove the bottleneck. It moved it, from reading to signing. And the value was never in how much water you had. It’s in how much of it you could drink. Pumping dirty water faster isn’t a water system. It’s an outbreak with better pressure.
None of this means don’t drink. Most water isn’t drinking water and it does the work of the world. You wash the car with it. You run it through the engine to keep the engine cool. You water the tomatoes, and yes, when it’s hot enough, you drink from the hose and you’re probably fine. Nobody demands bottled water for the car wash, and nobody should demand certified information to think out loud. Read the reservoir at machine speed. Explore it, brainstorm from it, get the shape of a thing. That’s how you find the water. It’s not how you drink it.
The rule is a match, not a ban. What are you about to do with this? If the answer is “wonder about it,” any water will do. If the answer is spend money, tell someone something they’ll repeat, change something that’s hard to change back, then you want to know which glass you’re holding before it touches your lips. And when you hand it to the next person, say which one it is. That’s the whole discipline. Know what you’ve got. Match it to what you’re doing. Tell the next person.
So how do we ever really know? We don’t. Nobody in that county knew the water was safe the day the notice lifted; they knew the plant was back to certifying it. That’s the honest version and it’s enough. Certified water is a maintained promise, not a guarantee. It’s fit to consume, not pure. We drink it every day and we’re fine, and the reason we’re fine is that somebody built the system, keeps it running, and pulls the certificate the second they can’t stand behind it.
Look at whatever landed in front of you this week that read clean. The summary, the report, the answer that came back in a second with headers on it. Ask one question before you act on it: where did this come from, and would anyone put their name on it? If you don’t know, treat it like the tap under a boil notice. Useful for plenty. Not for drinking. Then, if it matters, go find the jug.
Potable water to drink. Potable information to act.
Uncomplicated systems. Uncommon results.



Yes.m on your question Looking at the warrant of the information your given form an LLM is the whole game right now. .
I ran my own little research pilot on it. On google AI. safe to say it cannot be trusted for sceintific facts, even when given a prompt for epistemic rigour.
The cost of recieving and now generating information has dropped. The cost of verification is still at a premium. System 2 thinking still is a rare occurrence.