
When somebody tells me AI isn't real, or that it produces garbage at their company, I've stopped arguing. I ask what the review step looks like. Usually there isn't one.
A few weeks back I walked through the loop I teach.. interview me, make a plan, poke holes in the plan, build it, check the work. I'm not going to re-teach it here. I want to talk about where it breaks in a company, which is a different place than where it breaks for an individual.
1. It breaks at the plan, every time
Individuals skip the plan because they're impatient. Companies skip it because nobody owns it.
The plan step is the cheapest place to catch a wrong assumption and the easiest place to wave through. It's a wall of text that arrives before the interesting part, so people scan it, say "yeah, looks right," and hit go. Then the output is wrong in exactly the way the plan said it would be, and the conclusion everybody draws is that the model isn't good enough.
Here's the thing.. reading the plan is a management behavior. It's the same muscle as reading a scope document before signing it, and the teams who are good at one tend to be good at the other.
2. Same prompt, 194 defects
I recorded a podcast recently with a founder who's built a fairly serious harness around his coding agents.. a set of deterministic checks the output has to clear before it counts as done. He ran an experiment where he gave a frontier model the same prompt twice, once inside his system and once without it.
Without the guardrails, the run produced 194 defects that didn't clear his bar.
That's his number, from his setup, and I haven't reproduced it. But it's the most concrete version I've seen of something I believe from our own work: the scaffolding around the model is doing more for quality right now than the model itself. Same model, same prompt, wildly different output depending on what's checking it.
I wrote a while back about a founder burning tokens on a loop he couldn't see. Same root cause from the opposite direction.. no check in the middle, so nothing stopped it.
3. Where the check actually pays
We've built a lot of workflow automation over the last couple of years, and the pattern that shows up over and over is unglamorous.. classification/extraction/routing. Read the incoming thing, decide what kind of thing it is, pull the fields out, send it where it goes.
That work is a great fit for agents and a terrible fit for blind trust, because the failure mode is silent. A mis-classified item doesn't error out. It just goes to the wrong place and sits there looking fine.
So the check has to be structural, not a vibe. For that kind of work it usually means a confidence threshold with a human queue underneath it, a sample somebody actually eyeballs each week, and a hard rule that anything touching money or a customer commitment gets a person in the loop regardless of confidence.
None of that's exciting. It's also the difference between a pilot that survives contact with the business and one that quietly gets switched off.

4. This is the Agile mistake again
I wrote about this in July and it keeps getting more true. We took a set of practices that worked, kept the ceremony, dropped the discipline, and then blamed the framework when the results didn't follow.
The AI version is a team with an assistant on every desk, no review step anywhere, and a leadership deck saying they're AI-enabled. The ceremony is there. The discipline isn't.
5. What the review job actually looks like
When I say somebody should own the review, people picture a committee. It's much smaller than that.
On a given piece of work it's three moments. Before it runs, somebody reads the plan and asks whether the assumptions match reality.. does it know about the exception we always forget, is it about to overwrite the thing nobody documented. While it runs, somebody has a threshold for when the machine should stop and ask instead of guessing. After it runs, somebody looks at a sample of the output on a schedule, not just when something breaks.
Total time on a well-scoped workflow is maybe 30 minutes a week. The 30 minutes was never the obstacle. No name is attached to the job, so it never reaches anybody's week.
Put it in one person's objectives and the problem mostly evaporates.
6. Silent failure is the one to design for
AI failures are usually quiet.
Traditional software breaks loudly. Something errors, a page 500s, a job fails, somebody gets paged. You find out because the system tells you. An agent that misclassifies 8% of incoming items doesn't error at all. It produces confident, well-formatted, completely wrong output that flows downstream and looks exactly like the right output until somebody notices weeks later that a category has been quietly under-counted all quarter.
That's why the sample review matters more than the dashboard. A dashboard shows you throughput, and throughput is going to look great right up until it doesn't. 20 items read by a person once a week is a cruder instrument and it catches things no metric will.
I'd also say.. don't let the first version be autonomous. Run it alongside the human process for a few weeks and compare. It's slower and it's the only way you'll ever know what your actual error rate is, because you'll have something to compare against.
Wrapping up
The teams getting real output out of this are running the same models you are. What they have is somebody whose actual job is to read the plan before it executes and check the work after it does. That's a role rather than a line in the tooling budget, and in most companies right now it belongs to nobody. Pick somebody. Put it in their objectives. It's the cheapest quality improvement available to you this quarter and it doesn't require buying anything.
Putting this into action
On your next AI project, name the person who reads the plan before it runs. Write their name down, not a team name.
For anything doing classification or routing, add a weekly sample review of 20 items. If nobody's looked at the output in a month, you don't know if it works.
What I'm reading
Aaron Levie's notes from a couple dozen enterprise tech leaders. Seven trends out of banking, media, insurance and consulting, and two of them are this issue. The ROI shows up when companies change the workflow itself instead of layering agents onto the one they already have, and his open question is who actually owns driving that change. He also found that only a couple of those companies had any real grasp of evals on their own workflows.
The playbook for building an AI-native company, from YC. Most of what's in it is org design rather than tooling, which is the same argument I'm making above.
Jev, from TypeSafe. All over my feed the last several days, so I spent the weekend testing it. An LLM writes you a paragraph and leaves you to decide what to do with it. Jev hands your software the decision itself, with a confidence score attached.. ask it whether an invoice is a duplicate and you get back a yes or a no and how sure it is, not three sentences about it. That's the gate from section 3 as a product: above your threshold it routes on its own, below it goes to a person. It's early, and I've already found a few spots in our stack where it fits. Anyone else testing it? Hit reply, I'd like to compare notes.
Onward & upward 🤘
Drew





