The Role of Change Management in AI

Why is change management such a critical competency for success with AI?

What are the implications for bringing everyone in your organization along with AI, not just the early adopters?

And what are a few of the AI tools and innovations that the AI Knowhow team is most grateful for this year?

Hi, I’m Courtney Baker, and this is AI Knowhow from Knownwell, helping you re-imagine your business in the AI era.

As always, I’m joined by Knownwell CEO David DeWolf, Chief Product and Technology Officer Mohan Rao, and NordLite CEO Pete Buer.

We also have a discussion with Knownwell AI Advisory Board member Andrew DeBerry about driving successful change management in the age of AI, and a project he’s worked on with the team at Google X that Time recently named One of the Best Inventions of 2024.

But first, power up those headlamps and limber up those typing fingers, because it’s time for a new segment with Pete Buer we’re calling Searching for Competition.

Pete Buer joins us as always to break down some of the latest and greatest in AI.

Hey, Pete.

Hey, Courtney, how are you?

I’m doing good.

This week’s story comes to us from the verge.

OpenAI’s search engine is now live in chat GBT.

Pete, what’s the takeaway for business leaders?

Well, I’ll tell you, one is that despite all the turmoil and controversy at the company, especially at the top, including leadership departures, OpenAI continues to roll out new features faster than you could possibly imagine.

In this case, it wasn’t that long ago that chat GBT only trained on data that was recent up to about six months ago.

Now you can kind of trust that your answers are recent too about yesterday or maybe even today, which is amazing if you think about it.

Good news for we consumer buyers, but also for enterprises as well.

You know, you can easily imagine the technology powering knowledge management within the large corporation.

Find the latest board deck, the properly branded PowerPoint to use for your next set of presentations.

Where have we worked with provider X before?

When have we used technology Y?

And how did it go?

I mean, the possibilities kind of quickly become endless.

And it’s not hard to leap from the internal applications to the possibility for making access to your information from customers a whole lot more seamless.

And especially if you’re in the content business, boy, that’s your stock and trade.

I personally spent this morning doing some dumb searches, like finding restaurants in the area that had sit down dinners for Thanksgiving.

We’re thinking about doing it a little differently this year.

And which ones are gluten free and what are their hours?

And that was like, it was just so easy.

I was looking for cabins next summer in the south of Norway.

And the searches were effortless and the results were to the point it couldn’t have been easier or faster.

I feel like we’re entering an entirely new world of search with this advancement.

That’s amazing.

And you just gave me some things.

It’s there’s been a few things lately where search has just not been enough.

And so I think I’m going to turn to this to get some help with the 3-year-old birthday party.

So, you know, it’s helping professionally and it’s helping personally, which my favorite kind of technology.

Pete, one other pro tip here is the Chrome extension.

So if you’re used to just, you know, if you’ve got your homepage set to Google and want to use this on a more standard basis, there’s a Chrome extension that you can utilize to make ChatGPT your default search engine.

So definitely check that out.

Pete, thank you as always.

Thank you, Courtney.

Andrew DeBerry is a member of the Knownwell AI Advisory Board and the co-founder of a Bay Area startup.

Andrew has led AI initiatives at Microsoft, AWS and Meta.

And as I mentioned in the intro, he worked on Google X’s Bellwether project to analyze natural disaster damage that was just named one of Time Magazine’s best inventions of 2024.

He recently sat down with Pete Buer to talk about why successful change management is a must for any AI initiative to meet its mark.

Andrew, hi.

So awesome to see you again.

How have you been?

Life is good.

How are you?

Well put.

Life is good.

I agree.

You were our very first podcast guest.

So it’s a special honor to be spending time with you in this seat again.

Topic today is change management.

And I know that there’s some work that you’ve been doing that will be great fodder for folks who are either listening or reading, you know, as in the end this gets published.

Can you give a little bit of background on the work?

Absolutely.

So at a high level in the market, we see that the excitement around large language models continues to grow.

But the reality is that when AI is most effective, it is pulling through some of the longer standing use cases that are out there, more traditional unsupervised and supervised machine learning.

One of the areas where that’s coming to life for me is in a project with X, which is the moonshot arm of Alphabet.

In the past, had been called Google X.

And within that, there’s a team called Bellwether, which is using AI for disaster and emergency management.

So by introducing new concepts like integration of different data sources, processing them more efficiently, and helping different levels of the government understand how to respond and prepare for disasters has been a very real use case for not just understanding what AI is capable of doing, but working with the existing infrastructures that these organizations have.

That helps with the purpose, the business objective.

Thank you.

What does success look like?

How will we know if the project ultimately achieves its goals?

So there’s a good, better, best scenario here.

One is that they are able to integrate solutions that we have out of box to address immediate issues that are coming up.

So for example, one of the public case studies that we have is with an organization where I have served as a reservist called the Defense Innovation Unit.

And they are in partnership with organizations that take imagery of different scenes after a disaster.

And it takes a lot of manual time to download those images, process them, understand the locations those images were taken of, and make better use of that, especially after a disaster to assess the damage.

And we are able to leverage the AI capabilities to listen to what the workflows are, literally get into the trenches with them, then find ways that can have immediate impacts, reduce the costs, reduce time that it takes to do that.

So there’s an immediate impact there.

A second component is a better scenario is understanding the infrastructure that they are using to make those solutions happen.

And so it’s not only providing something ready out of box, but also looking at their workflows.

You could look, you could say their machine learning ops, ML ops, how their flow works for that, ensuring that the fusion of different models is working well and in a secure way.

And that provides them a broader base for how they can apply future machine learning solutions into what their roadmaps will be for the use cases they’re trying to address.

Then best scenarios is that they have full org capabilities primed for working with AI.

So not only is that technical understanding of data and infrastructure, but it’s also cultural.

Do you have leadership who can champion new use cases?

Do you have budget set aside to run experiments and then scale them once they are successful?

Do you have a process for coming up with success metrics?

So it’s many ways just iterating what mature organizations do today, but factoring in the unique nuances that machine learning involves.

If you run the project from beginning to end according to the timelines that you’ve estimated, how long will this work take?

So the project itself has been in development for a few years, but once it’s integrated out of box, it can within a couple of weeks provide a clear insight to how buildings and sensitive assets are being affected by different sassers.

And then if you want to move from being reactive to what has already happened, to being more strategic in their planning of resources and the positioning, for example, thoughtful planning of evacuation routes or staging areas, those predictions can stretch one to five years into the future with hard metrics that show that the latest approaches that we’re able to take are superior to many alternatives in the market.

So there’s almost no limit to how long the impact can last.

But just by looking at a map, it’s almost very intuitive for what you can do with it.

And so the technology has the capability to drive change within weeks.

Are the users ready to move that quickly?

That’s a fine question because we are learning there’s different kinds of users.

There are some who were just end users and they just want to see the product as is.

They just want to have access to a map which shows certain areas are higher risk for wildfires than others.

So that gives them immediate insight for where they should position things or how they should plan.

There are others that want to get under the hood and they want to understand, well, what is really driving the risk for high concern areas and how can they begin to mitigate or treat those areas.

And so, yeah, there are different users that have different needs.

Some more technical and some just want to have it in their hands so they can do their flights or position the resources right away.

Well, and so we’re getting closer to the change management focus with this discussion of users.

What are the barriers to implementation to transformation that the change management work is meant to get after?

I like this as not like an incredibly new concept for how to consider new technologies, but then I will show how AI is unique to it.

So at a high level, be it cloud, be it even the mobile phones, there’s different levels that are really critical for that innovation to be considered.

Number one is having a vision for what is possible, and it certainly helps if there is leadership that understands the value of that vision and can translate that and champion that across the organization.

Number two, that’s valuable because that helps the individuals within that organization understand the processes and how they can test and iterate with the new capabilities that the market is bringing to bear, but then evolve their workflows in a very practical way so they can steadily adopt and transfer what they’ve been doing to that.

And then thirdly is the individual users having not just excitement and an openness to it, and that is that openness and the cultural piece is certainly a blocker in many organizations when it comes to change management, but also knowing what their next immediate steps are and how to have quick wins with these new capabilities, and that begins to build this momentum.

It’s not uncommon, especially where I sit in the Bay Area, that folks are very excited about the tool that they’ve created, and it oddly sometimes overlooks the human element, and just the openness for those humans to take those tools and be brought along the journey along with the tool, and what’s capable needs to be considered a bit more.

And so that is something that I’ve seen teams like this one and others who are more successful in getting their changes implemented, and they do work alongside people and listen to the problems.

They’re very attentive to where they’re at, not just mentally, but also spiritually, psychologically, to the problems that they’re having, and it allows them space to grow with that innovation.

So specifically for AI, there are places where you do need to have a federated approach to this.

So is there a centralized place where data is compiled?

And then is there a centralized team that begins to federate access and federate use cases across an enterprise so that it can serve a wide range of use cases, as opposed to like different spokes, different have to make things up on their own.

Just with the nature of centralized models, centralized data, there are some nuances that are involved with AI in order to get these use cases to have impact in ways that are measurable by specific success metrics that ultimately leadership in the end can champion.

As you talk about pockets of readiness or not readiness or resistance even within the business, how do people show up?

Is there a rule of thumb distribution, a third to third to third site ambivalent anti or any guidance you can give on what people should expect as they try to implement and drive change?

I love this question because I feel like there have been two camps.

There are the folks that are so excited about the future, and it’s not uncommon that it can be a senior leader who sees all the headlines, and I’ll say the hype cycle, they’re thrilled.

There are a number of long-term visionary consulting firms that champion that and highlight the art of the possible.

Then there’s a second camp which are very practical and they’re very day-to-day focused.

You need both of those.

But in there, there’s a small overlap of individuals who understand the day-to-day workflows and how you can begin iterating what’s already there with this new technology, and come up with the processes for how can you commercialize these capabilities.

Trying to listen for the organizations that have that openness, that appetite to iterate is helpful.

Especially in large organizations, there is a lot of concern about risk.

And a number of individuals, rightly, are incentivized to protect what they have.

And they’re not willing to take the leap until they see that there’s a group of peers that want to take the leap with them.

So in that, you start to…

It’s like an influence campaign.

You start to listen for who is most open to taking that first step, and then try to convince more stakeholders alongside them so that they together can take that leap as a group.

And so I would say that the…

I’ll say the comments of the folks who have been there, let’s call that two-thirds, and then depending on the organization, that could be one-third who are really excited, and there’s just a…

Often it’s a kind of a small sliver that can see what’s possible.

And then we can compare organizations to organizations.

So I mean, using Crossing the Chasm frameworks, there are some that are the early adopters, and they’re willing to take those steps.

And there’s others who say, that’s great.

Let them figure it out, work out the kinks.

And then once it’s further along, then I will feel more comfortable given my incentives in my business and my organization to follow along after that.

So that is very important for any innovator to be cognizant of as well.

I can imagine change management happening pretty much all the time for the next five years, because you’ve got major initiatives that need to get launched into the business.

And you’ve also just got the presence and preferably proper use of generative AI in jobs day in and day out.

Do you see change management becoming more of a required capability of successful companies going forward than maybe it’s been up until now?

Absolutely.

It’s critical for teams to be able to listen to what’s happening.

And it’ll be interesting to see again how organizations position themselves in those changes.

Let’s say in the cloud space, there were certainly advantages for AWS to be the first mover inside of cloud infrastructure.

And then now we’re seeing that the CEO of Oracle is the second richest man in the world because Oracle waited and they took a different approach to see what was landing and then position themselves there.

Both are really valuable.

Neither of them, though, would succeed if they did not listen to where their clients and their customers are at.

And so, there is a market for folks that want to move quickly.

And we saw that Apple, when they got built out their app ecosystem and set up some exclusivity rights, that was certainly a differentiating commercialization strategy for them.

And then we see later that as some of those terms get relaxed, that other ecosystems are able to also benefit from those coders and developers.

And so, the change management piece will be critical.

I think there will be an interesting challenge of understanding in business strategy, there’s a paper called Flexibility vs Commitment by Gemawat.

And there’s this challenge of discerning what do you have to commit to and then what resources you have that can be flexible towards those future use cases.

So, it feels right now like, especially the hyperscalers, the Amazons and Googles of the world, they are realizing they need to commit to building out large AI compute foundations.

And that will allow them the flexibility to pivot from there.

Other organizations will have to approach this dilemma differently.

So, do they pursue one specific use case that they’re really convicted is going to win?

Or do they wait and like maintain the flexibility?

So, there’s that trade-off for how you do that as well.

And that’s a unique part of change management that I think will be interesting for all of us to listen to.

I think it’s true at the organization level.

I think it’s also true at the individual leader and manager level, you know, to achieve the kind of outcomes and get the ROI that we’re talking to Wall Street about.

Behavior has to change all the way down to the front line.

And with AI, innovation happens as much at the front line as it does, you know, in a headquarters conversation.

And so, as a manager, you kind of need to be able to guide your team to think creatively, to work differently, to take new approaches to work in a way that I don’t think we’ve expected of managers in the past.

I think there’s a big learning journey ahead for large swaths of the middle of the enterprise.

Have you thought that one through or do you have a view?

Absolutely agree.

At one point, I used the phrases of divergent and convergent experiences.

There’s more ways to have brainstorming sessions, whiteboarding sessions, Figma now has FIGJAMS, and there’s more ways that I’m feeling a little bit more disciplined to your point of thinking more creatively.

And then from there, there’s also another discipline of like, okay, well, how do you bridle that and get more goodness out of that?

Once upon a time, some of these innovation consultants like from IDEO came up with a lot of frameworks for how you can do that, but then some of them may not necessarily commercialize as well and there’s others who’ve seen, well, what has worked from that?

And they’re incorporating some of those practices.

Some of the strategic consulting firms also incorporate some of those best practices as well of how do you expand people’s minds to what is state of the art right now, but then start to guide them towards what is low-hanging fruit and what can have impact in the short, medium, or long term.

So to your point, absolutely.

This is a new time of reimagining what’s possible, but fast following that with like, well, what can we do today and then starting to couple those together.

Well, I know that we’re getting to the end of time and I want to respect your calendar, but may I just say it’s been a pleasure to see you again and to hear your insights.

I’m grateful for everything you’ve been willing to share.

Congratulations to the Knownwell team.

I’ve been thrilled to see the growth and the progress and some recent huge milestones, so it’s really great to connect with you.

One thing we’re really grateful for at Knownwell are new members of our Early Access Partner Program.

As you may know from previous episodes, we’re an AI startup that’s changing commercial intelligence for professional service companies.

And we’re always looking for new innovators that would be interested in joining our program.

So this Thanksgiving, we would love it if you would go check out what we’re building at Knownwell at knownwell.com/demo.

If there’s one thing our producer, Nick Jaworski, loves in the world, it’s playing the drums.

If there’s a second thing he loves, it’s playing the drums in holiday-themed episodes.

So David, Mohan and I, we’re only happy to oblige Nick recently by talking about what we’re most grateful for when it comes to AI.

David, Mohan, it is the week of Thanksgiving, and I’m curious, what have you been thankful for in 2024, especially when it comes to AI and how you have personally deployed it or found uses for it and also in our business?

I will start this thankful train.

Is that a thankfulness train?

I like that.

I’ll start this thankfulness train by going with, I am thankful for everybody else’s superhuman note-taking yada yada that show up on all of the calls so that I don’t have to keep my own meeting transcripts.

I joke all the time getting on a meeting and four robots joining me early, no humans yet, just everybody’s note-takers.

It makes me laugh every time.

I can totally see why you’re thankful for that, and I’m with you.

I think it’s interesting to look at that, Courtney, and see not only the technology, but the adoption to me is the fascinating part.

Yeah.

When these first came out, I think there was a little reticence to have it auto log in.

Then like you said, I literally was on a call last week where it was me and three note-taking apps that showed up before anybody else did.

I’m like, I don’t know what to do.

This is kind of strange.

But it is really being adopted.

I think that tells you something.

We get a lot of questions about adoption and how people are thinking about privacy and all these different things.

When it comes to Knownwell, I think that’s just an example of one of the leading use cases that is being adopted and the wave is happening.

I’ll piggyback on that.

While I’m thankful for that, I’ll tell you there is a recent advancement in the reasoning of these AI models that I think has gone untalked about.

I don’t know if that’s the right way to say it.

It has gone, there just hasn’t been noise.

There hasn’t been buzz about it and I’m shocked because I’m very grateful when ChatGPT released O1 Preview.

It was its advanced reasoning model and it actually takes time to think before it responds.

And if you play with it and use it for the right use cases, you’ll find it is exceptional for doing some advanced, more complex thinking.

I’ll give you an example that I tried it out with and I was just blown away.

We are, as we build our company, implementing OKRs for goal setting within the organization.

And as part of that, we did some early experiments and we are at the point of rolling it out to the entire organization.

And we wanted to put together some training.

And I asked O1 Preview to be able to put together the curriculum for that training.

And I was blown away by the difference that O1 Preview was able to do versus kind of standard ChatGPT 4.0.

And the thinking that it did just in a few prompts was able to get to something.

Like the first course it gave me was a curriculum for like a college class.

Like it was unbelievable.

It’s like, hold on a minute, where is startup?

Like tone it back.

And it was able to come up with curriculum based on publicly available books and videos and different things that was really good and compelling and could be rolled out perfectly for a startup.

And literally did 97% of the work with four different prompts into O1 Preview.

I have since been able to use O1 Preview for a couple of other kind of thought intensive pieces of work like that, that really gave me a kickstart and added significantly to my productivity.

And I think that was just a breakthrough that I’m really excited about to see where this advanced reasoning goes next and what other use cases I can find and how that gets embedded into kind of the enterprise platforms of tomorrow.

I agree with you.

O1 is amazing because what O1 does is it embeds Daniel Kahneman’s system one, system two thinking, right?

So one is a fast, rapid fire.

And then there is system two, which is much more slow, more reasoned thinking.

And when you combine the speed with system two thinking, the results that it produces are just amazing.

So just to reiterate, I think some of these transcription services, I mean, my goodness, Otter has over 10 million users at this point.

Not only do I think this is really helpful for just generally taking notes and, you know, keeping track of all kinds of things like sales calls and pulling those into the CRM and all kinds of use cases for those.

But it’s also another piece of data that can be used in other applications, other AI platforms that I think is really interesting.

And I think we’ll continue to grow.

I think we’ll see more and more meetings that we get in with 10, 15, you know, computers waiting and note takers as we get on these meetings.

Courtney, to that point, you know, you mentioned Otter, I think Fireflies is one of the most popular, Superhuman, I think you mentioned a little bit earlier.

Like all of these are growing that fast.

And this is a brand new market, right?

We talked, I think in an episode, three or four episodes ago about emerging markets and how AI will not just reinvent but will create new markets.

This is a new market.

I was looking at some research the other day.

I believe this market size is up to 400 million, but is expected in the next 8 to 10 years to get over 2.3 billion.

Talk about how economists see this really accelerating the efficiency, the effectiveness, the work that we’re doing and how it will cascade through the market.

I think this is one of those areas that historically, we have taken notes on notepads and we have asked people to help us capture notes.

We’ve had these different solutions where all of a sudden AI can totally reinvent that, and it’s created a brand new market and it’s being adopted, it’s being used.

It does seem like the first step to these types of tools expanding beyond note taking in this context.

Being able to take notes just audibly as you’re sitting working at your desk or as you’re doing things.

It seems like a really easy transition outside of this medium.

What I need to do, I get so many of these summaries and notes now.

I need to figure out the workflow part of this of, when I used to take notes, the value for me was the tactile writing it down helps me to recall.

I don’t know what to do with all of these notes unless I have a very specific thing that I’m going to look back up.

It’ll be really, really interesting to see what that innovation spurs in terms of other innovations for helping us take the benefit that we see.

We can all say, aha, wow, this is awesome, but then how do we translate that into the next step of what do we do with it?

How does it fuel that next wave of innovation to drive our behavior even further?

It reminds me of Mohan has talked so much about knowledge management within a business context, but we also are going to need our individual knowledge management within our individual roles of just being able to take all of our notes, all of our thoughts and being able to recall things for us.

Yeah, because I’m just overwhelmed with all these notes I’m getting now.

I’m impressed by them, I’m wowed by them, and when I need them, I need them.

But otherwise, I’ve got a filter in my email to take them out, so I’m not bogged down by it.

So I think you’re totally right.

That’s a great point.

Yeah.

David, there are some new apps that have come out.

I think the way you write is you, I think record, you voice record certain things.

I do do a lot of that, yep.

When I’m in the car, I’ll write documents in the car while I’m driving with voice record.

Yeah.

I’m trying to figure out, remember there is one app where you can just record and then it produces, it’s not as well as what Will does, but it produces a nice write up in your voice in the way you like to write.

So it’s just doing the basic transcribing and then putting it in the way you want it to see.

I think it’s called Voice Notes or something like that.

Can’t remember the name right now.

One of the challenges in that world is the spoken word and the written word are two different things.

How that ends up impacting language and society and how we communicate will be really interesting.

It reminds me, CS.

Lewis, one of the great kind of Christian writers back in the 1950s did a radio, I don’t know, series of lessons called Mere Christianity and actually was turned into a book later on and he had to go through multiple iterations to get it actually to work in written format.

He actually talks about this in the second version, about taking that spoken word to then written word, and then having to modify it.

As you’re talking about this and we’re thinking about all this new information flowing at us, I can’t help but think we’re living through that again, of figuring out how this all fits together and what we do with it.

What I’m grateful for in 2024 is I’ve been reading about all of the AI-powered drug discovery and health care improvements.

I’m not an expert in this field, but I’ve just been following as much as I can, and the AI diagnostic systems have really come a long way.

I think the effects of this are going to be felt for many years to come.

It seems like in 2024, AI-powered drug discovery, health care has taken a giant step forward.

Thank you all for listening to this.

Hopefully, you found some good hacks, maybe some things you want your boss to get to improve their productivity.

If so, send them this episode.

Happy Thanksgiving for everyone in the US.

Thanks as always for listening and watching.

Don’t forget to give us a five-star rating on your podcast player of choice.

And we’d really appreciate it if you can leave a review and or share this episode on social media.

At the end of every episode, we’d like to ask for one of our AI friends to weigh in on the topic at hand.

So, hey ChatGPT, what’s happening?

This episode, we’re talking about what we’re grateful for in the AI realm.

So, question for you, what are you grateful for?

I’m grateful for AI’s ability to break down barriers to knowledge and creativity this year.

In an era of rapid change, AI’s capacity to drive meaningful innovation and connect diverse minds and ideas is something to celebrate and harness.

And now, you’re in the know.

Thanks as always for listening or watching.

We’ll be back next week with more AI applications, discussions and experts.