LogoLogo
Book a Demo

AI and Change Management: From Pilots to Practice – Making AI Change Stick

AI Knowhow: Episode

113

watch-on-youtube-final
aaple-podcast
listen-on-spotify

Most leaders can say they’ve experimented with AI. Some can even say they’ve given their team a mandate to become AI-first. But far fewer can say with a straight face that AI has fundamentally changed the way their business operates.

So why is this gap so persistent? And more importantly, how can you close it in your company?

In the third episode of our miniseries on change management and AI, we tackle the challenge of moving from forever pilots to production-ready AI systems that actually transform how your business operates.

Escaping the Forever Pilot Trap

According to new Harvard Business School research covered by CNBC, AI is reshaping white-collar work in unexpected ways. The research studied workers at Procter & Gamble and Boston Consulting Group, revealing critical insights about where AI excels and where humans still reign supreme.

The findings are striking: individuals equipped with AI can perform at the same level as full teams without AI. Yet AI-enabled teams outperform lone AI power users when quality and innovation matter most. The message is clear: AI transformation isn’t about replacing humans, it’s about redesigning collaboration for a new era of a AI-enabled teamwork.

But here’s the catch: while AI drove a 17% performance increase among top performers, it drove a 43% performance increase among lower-skilled performers. This suggests there may be more value in leveling up performance across the masses than trying to make the very best incrementally better.

Why AI Initiatives Fail to Stick

Knownwell CEO David DeWolf identifies several reasons why organizations struggle to move past the proof-of-concept phase:

“I think we have to move past this proof of concept phase. There’s a lot of just experimentation. And people don’t believe that it’s actually here to stay, that we’re actually trying to change the business, that we’re trying to fundamentally operate differently.”

The problem isn’t just technological. It’s cultural as well. Too many leaders give lip service to AI transformation without truly understanding what to expect or how to inspect progress. As David notes, “It’s hard to inspect what you expect when you don’t really know what you expect.”

Four Principles for Making AI Stick

Knownwell Chief Product & Technology Officer Mohan Rao outlines four critical principles for embedding AI into business operations:

  1. Embed AI into workflows, not layered on top. AI must be designed into the flow of work, not treated as an add-on tool employees have to remember to use.
  2. Use AI in rituals. Tools and products can start the change, but rituals sustain it. Weekly account reviews, planning cycles, and team meetings need to assume AI usage, making it expected rather than optional.
  3. Escape the forever pilot trap. At some point, “learning at scale becomes more valuable than perfection in isolation,” Mohan says. You have to move AI into production and iterate from there.
  4. Design for evolution, not stability. In the AI world, product-market fit must be found every three months. The goal is adaptability with guardrails, not locked-down systems that become obsolete.

The Leadership Gap in AI Management

Perhaps most striking from the Harvard research: there’s a significant readiness gap when it comes to managing AI-powered teams and agents. As one researcher noted, managing AI takes an entirely different set of skills—orchestration, evaluation, and system design. It’s a fundamentally different leadership experience than managing people.

This has profound implications. Management training is already challenging. Now leaders must develop entirely new competencies to manage teams where AI agents and AI-enabled collaboration are central to the work.

From Pilot to Performance: A 90-Day Playbook

For leaders ready to stop running forever pilots and start building permanent capabilities, our guest Scott D. Anthony, bestselling author and clinical professor at Dartmouth’s Tuck School of Business, offers practical guidance.

Scott, who’s consistently ranked among the top management thinkers globally by Thinkers 50 and was #5 on the 2025 list, teaches a course on AI and consultative decision-making at Tuck. His framework for preparing students for the new world of work emphasizes treating AI as a teammate. And like any human teammate, it’s one with strengths and limitations that you can only learn by working closely together.

The key insight: the AI revolution won’t necessarily reward early adopters. It will reward effective operators. Those who move beyond pilots, embed intelligence into workflows, and engage leadership at every level will find themselves on the right side of the transformation divide.

The Hard Work Ahead

The Harvard research makes one thing abundantly clear: the hard work of AI transformation won’t be about adoption rates or active licenses. It will be about organizational design—sorting through all the ways work comes together differently when AI is truly embedded in operations.

The winners will be companies that have deliberately addressed team structures, training programs, management development, and workflow redesign. They’ll be the ones who escaped pilot purgatory and built AI systems that compound intelligence over time.

As Mohan puts it: “Leaders should start expecting that these AI systems and outputs are used at certain meetings. It’s not even a question of is it being used. It is used in these meetings. It is in the workflows. Once you set the expectation that this is how we do business, there’s a much higher probability of these systems sticking.”

Watch the Episode

Watch the full episode below, and be sure to subscribe to our YouTube channel.

Listen to the Episode

You can tune in to the full episode via the Spotify embed below, and you can find AI Knowhow on Apple Podcasts and anywhere else you get your podcasts.

Show Notes

You may also like