The Pilot That Passed and Proved Nothing
Marisa ran the cleanest AI pilot her company had ever seen. It came back green, changed nothing, and taught her the most expensive lesson of her career: a pilot that can only succeed isn't an experiment. It's a demo with a budget.
The slide said 94 percent. Marisa remembers the exact number because the room applauded it.
Four months on a forecast-accuracy proof of concept for the company's highest-volume category. Last year's data, a frozen baseline, a clean question: could a machine-learning model beat the planners' numbers on a year that had already happened? It could — by fourteen points. The committee signed off, the vendor's logo went on the quarterly deck, and the pilot was filed under “win.”
Nothing changed. Six months later the same category was forecast the same way by the same people. No model in production, no decision different. The 94 percent had been reported upward, admired, and forgotten.
Marisa is a transformation lead with sixteen years across CPG and industrial — the person the CIO sends into a stalled program. And she had just run the most flattering pilot of her career: a number nobody could argue with, attached to a question nobody actually had.
The pilot answered one question — could this model have done better on a year that already happened — while the question everyone actually had was about a year that hasn't.
The morning the green light stopped meaning anything
It broke in a budget review, when the CFO asked the only question that mattered: "So we're rolling it out?"
Marisa opened her mouth to say yes and couldn't. Rolling out what? The model had beaten a baseline — but the baseline was last year's actuals, a year her planners had spent twelve months quietly repairing. The demand planner who overrode week nine because she knew the promotion would land badly. The scheduler who refused a changeover the system asked for. The buyer who held stock against a supplier that confirms cleanly and ships short.
None of that was in the file. What was in the file was the outcome those people produced. The model had been scored against a year human judgment already rescued — then that score was read as a prediction about a year where nobody would be there to rescue it. The 94 percent wasn't a lie. It was a mirror: the pilot measured the model plus the planners, and reported it as the model.
A pilot that can only succeed is not an experiment. It is a demo with a budget.
What a pharma scientist would have done differently
The reframe came from Marisa's new head of data science, who'd spent a decade in life sciences. Walked through the post-mortem, he wasn't impressed by the 94 percent — he was unsettled by it. "In pharma," he said, "a Phase II failure isn't a scandal. It's the process working. You paid to remove a possibility cheaply, before it became a commitment. Failure is the thing you bought."
Then the sentence that reorganized her whole job: your pilot was designed so it could only pass. You funded it against a business case, reviewed it against the same business case, reported the success upward — and never gave it a way to tell you something you didn't already want to hear.
This isn't a soft observation. When Microsoft's experimentation team measured its own well-designed, well-executed experiments, only about a third actually improved the metric they were built to move. Two-thirds failed — at a company that has run tens of thousands of them and treats that failure rate as the cost of learning. Marisa's portfolio had a near-100 percent success rate. That wasn't excellence. It was evidence that her pilots weren't testing anything.
A suspiciously clean pilot record isn't a sign your AI works. It's a sign your pilots can't fail — which means they can't teach you anything either.
The pilot she let fail on purpose
So Marisa ran the next one inside out. Same category, opposite design.
Before anything started, she wrote the number that would end it — the kill criterion, agreed in the room, before the work began. One line. Afterward, a stopping rule is rationalization; before, it's the only thing separating an experiment from a procurement exercise.
Then, instead of grading the model against a repaired history, she put it in shadow mode. The agent ran on live data, emitted its decisions, executed nothing — and every day her team watched where it disagreed with the planners. The disagreements were the product. Each divergence was a missing constraint, an expired rule, or a piece of knowledge nobody had ever written down. She went in thinking she was evaluating a vendor. She came out having audited her own operation.
And she stopped testing one agent alone. The value in a real network doesn't live in a single model — it lives in how demand, allocation, and scheduling agents negotiate across systems. Test one in isolation and you've measured the wrong unit, then scaled the wrong conclusion straight into production.
Eleven weeks in, the pilot hit its kill number on one sub-category and Marisa shut that branch down. In the old world that would have been a failure to bury. This time it was the most valuable thing the program produced — a bad idea removed for the price of a quarter's compute instead of a year's inventory.
The planner who kept saying no
There was one planner —Devin — who overrode the shadow agent more than anyone. Nineteen years on the floor. In the old regime his override rate would have been a red flag, a number to drive down toward the adoption target.
Marisa did the opposite. She sat with him and asked what he knew that the system didn't. Every override, it turned out, was a sentence in a language the model didn't speak yet: the supplier who confirms and ships short, the account that escalates to the CEO if a case slips, the line that can't absorb a changeover on a Friday. He wasn't resisting the system. He was protecting a customer nobody upstream had thought about.
Amy Edmondson found the shape of this in hospitals years ago, and it surprised her. The better-performing nursing units reported more errors, not fewer. She wasn't measuring quality — she was measuring whether people felt safe enough to speak. Any organization with a suspiciously clean pilot record should sit quietly with that finding.
Drive override rates to zero and you don't get a better model. You get a floor full of people who've learned to stop protecting the business — and a system that never hears the one thing it most needs to learn.
The leadership demand: someone senior has to say out loud that version one will be worse than the person it's helping — and that the planner who stops a bad recommendation gets credited, not corrected.
What the green light hides
Marisa isn't running more successful pilots now. By the old scoreboard she's running worse ones — her portfolio finally has failures in it. What she's running are pilots that can surprise her, which is the only kind that were ever worth the money.
The number that should worry a leadership team isn't the 95 percent of enterprise AI pilots MIT's NANDA initiative found delivered no measurable P&L impact. Read closely, that figure is less a verdict on the technology than on the brief — pilots scored for immediate P&L against baselines nobody adjusted, run by organizations MIT noted were reluctant to disclose anything but a clean win. The number that should worry you is your own success rate. If nearly every pilot came back green, you haven't been experimenting — you've been buying reassurance and booking it as progress. Gartner expects a large share of generative-AI projects to be abandoned after proof of concept, for poor data quality and unclear value. Every one of those was decided in the scoping conversation, long before anyone saw a result. The pilot's fate is written in the brief.
The 94 percent slide is still in Marisa's archive. She keeps it as a reminder — not of a win, but of the most expensive kind of success there is: the one that lets you believe you learned something when you didn't.
So here's the one to leave on your own whiteboard: think of your last pilot that came back green. What number, agreed before it started, could have made it fail — and if there wasn't one, what exactly did that green light prove?
Sources cited
● MIT NANDA Initiative, The GenAI Divide: State of AI in Business 2025 (July 2025) — 95% of analyzed pilots delivered no measurable P&L impact; only 5% created significant value; authors note a corporate reluctance to disclose failure. fortune.com
● Ronny Kohavi, Thomas Crook & Roger Longbotham, "Online Experimentation at Microsoft" / Trustworthy Online Controlled Experiments — only ~1/3 of well-designed, well-executed experiments improve the metric they were built to move; two-thirds fail. exp-platform.com
● Gartner press release (July 29, 2024) — prediction that 30% of generative-AI projects will be abandoned after proof of concept by end of 2025, citing poor data quality, inadequate risk controls, escalating costs, and unclear business value. gartner.com
● Amy C. Edmondson, Right Kind of Wrong: The Science of Failing Well (2023), and her earlier hospital-nursing research — better-performing units reported more errors because psychological safety let people speak; her four criteria for an intelligent failure (new territory, a real goal, a credible hypothesis, no larger than necessary). rolandberger.com
● D. Sculley et al., "Hidden Technical Debt in Machine Learning Systems" (NeurIPS 2015) — hidden feedback loops (the 'replay trap'): a system's own outputs shape the data it is later judged on. papers.nips.cc
