3.5 million dollars taught me why most AI rollouts never work
The outcome question most rollouts skip.
Hey Titans 💪🏻
Today I’m writing about the central challenge of AI: rolling it out without the outcome in mind.
Previously, I have posted about:
How I used a Claude skill to help my employee navigate a difficult life situation in I was too drained to write a delicate message. My agent wrote it in my voice.
Decreasing daily anxiety by creating an agent that highlights open matters in The end-of-day check that removes manager anxiety;
How I got surpring feedback from my agent-coach about my communication style in I thought I was coaching the engineer. I was coaching myself.
By the end of this one you’ll be able to distinguish a good AI rollout from a bad one, and walk away with a two-question filter you can run this week: what part of your work only your judgement can do, and what’s left over that’s actually worth automating.
Bad rollout - when excitement trumps desired outcomes
Why is AI introduced into so many companies right now? Is it addressing something real, or just making the org feel like it’s performing?
What is the desired outcome of an AI adoption? Is it:
accelerating a known process;
automating what’s manual;
or unlocking decision-making support?
The mindset behind adopting new technology should always be the same: what do we want to get out of it? What’s the desired outcome?
In my work with my team, we always look for the desired outcome of a project before we start it. Let me give you an example.
Back in 2024 I got promoted to senior manager, and a company-wide restructuring grew my team to 20 people. My job: audit every ongoing project for ROI, and decide what stayed and what got cut.
One project had been running for over a year, bleeding from requirements that kept changing out of the client team. There was no real ownership either - the research team kept commissioning changes, and engineering kept absorbing them. The audit was blunt: those moving goalposts had cost over 3.5 million US dollars, not counting the engineering time burned along the way.
What killed that project: no desired outcome for the team to work toward.
Two years later, I’m asking myself the same question about AI.
I feel the same excitement I felt when I started learning to code back in 2010. What ignited it was the first lecture on Java, where the professor built a window application in 5 minutes and maybe 10 lines of code. In that moment I thought, yes - I want to learn this.
But excitement alone didn’t land me a career that led to becoming a leader. There was always one stupid simple question underneath it: where do I want to go next?
Bad rollouts happen when that question never gets an answer.
Good rollout - when you improve what matters
A good AI rollout is one that addresses a specific metric that matters to you. Improve what matters.
This year, a metric that mattered to me was time spent reporting status. The more time I spent on it, the less time I had for building things.
I’ve never met anyone who told me they love writing status reports - not even the ones who pride themselves on “facilitating cross-functional stakeholder alignment.”
At the start of the year I was spending about 4 hours a week on status reporting of some kind - highlight/lowlight slides, update calls, stakeholder sync meetings. In “I got two hours back from the worst part of my job. Here’s where they went.” I wrote about reclaiming 2 of those hours - the weekly status report slide alone used to eat that much time on its own.
I’m happy to report that number is now down to 15 minutes.
Protect your judgement, automate what’s slow
AI needs to show clear value. Despite the productivity hype, only 29% of companies report a significant ROI from generative AI, and just 23% from running agents. Another report puts the number of companies running agents in production at 7%.
Leaders like you and me carry a specific pressure inside that gap: sit in a meeting, get asked “how can we accelerate with AI,” and have no concrete answer. That question grinds away at patience, mine included.
A better start is to flip it: what am I doing that can’t benefit from AI?
That’s where your judgement comes in - the reasons your work is indispensable to the company. Mine, for reference:
Networking - working with Google is like operating inside a macrocosm of a market. If you offer a service, you need a paying client for it. Without real relationships across the org, I’d never be able to find clients for my work;
Turning unclear into clear - I communicate with my team often, advising them on next steps even when the information is scarce or unreliable. I can’t share the what or why yet (that’s a story for a different article), nevertheless this is the cornerstone of leadership: creating clarity when you don’t have all the answers. No machine does that, no matter how much context you feed it;
Product development - my company builds a device that depends on the latest research and know-how to design and manufacture at scale. AI models can’t help design and implement what’s beyond the current innovation horizon, no matter how advanced the reasoning. This is because current models are a lossy compression of what’s already on the internet.
That work takes up about 60% of my time.
The remaining 40% is up for grabs for automation with agents!
That’s the part worth interrogating:
Which piece of that 40% costs you the most time or the most stress right now?
What’s the data that you can use for agentic workflows? Hint: ask your manager which AI tools are approved, and your CISO (or organizational equivalent) what data is safe to use.
Once you know both, you have a real rollout plan instead of a reaction to hype.
I encourage you to run the same exercise. Draw the line between what only your judgement can do and what’s just eating your week.
The learning: a good AI rollout starts with a metric, not a tool.
Excitement got me into code back in 2010. It won’t get you a good AI rollout in 2026 - answering the question underneath it will.
So here’s my question for you: what’s the one metric on your team that AI adoption should actually move, and have you named it yet, or are you still riding the excitement?
— Leszek
Would consistency across reports, time required to retrieve the data (answers to specific questions) count?
Thanks great read as always