The End of Best Practices: Why the AI Era Demands a New Playbook

Courtney Baker: [00:00:00] For decades, professionals have relied on best practices in their particular realms as the safest bets for growth, efficiency, and transformation. But what happens when AI changes the rules so fast that best practices are already outdated by the time you implement them. Is it time to throw out the old playbooks and start writing new ones?

Hey, ChatGPT, can you help me out with this new best practices for marketing playbook?

Hi, I’m Courtney Baker, and this is AI Knowhow from Knownwell, helping you reimagine your business in the AI era. As always, I’m joined by Knownwell CEO David DeWolf, Chief Product and Technology Officer Mohan Rao, and Nordlight CEO Pete Buer.

We also have an interview with Andy Sitison, who joins us to talk about how your company can approach applying AI for human-centered [00:01:00] change, ethics and transformation, not just for productivity.

But first, Pete Buer is off this week, so we’re going to welcome David DeWolf to the studio to break down some of the latest in AI news.

Courtney Baker: Pete Buer is out of the office, so we’re going to get David DeWolf’s perspective this week in the news segment.

David, thanks.

David DeWolf: I mean, it took me over a hundred episodes to be able to wiggle myself into the most fun part. I, I do feel like I have big shoes to fill. Pete’s a lot smarter than I am, but, uh, this feels like a lot of fun to me. I love reacting to, to the real time what’s going on in the world with AI.

Courtney Baker: Yeah.

David DeWolf: Thanks for having me.

Okay.

Courtney Baker: I think I’m gonna give you a fun one out of the gate. I wanna start with a story that puts the scale of AI’s impact into perspective.

Walmart CEO Doug McMillon said recently that AI is going to change literally [00:02:00] every job at the company. That’s over 2 million people from cashiers to corporate staff. should leaders take away from a statement like that?

David DeWolf: Well, Courtney, I, I think we have to start by putting this in perspective like you did, but I wanna put some numbers on it, right? Walmart, you are absolutely right, is one of the world’s biggest employees. Well. That is 2 million employees, right? 1.6 million in the US alone. I think it’s really, really critical that we think about that and how big they are.

Now when you have the CEO of one of the largest employers in the world, that employees, 2 million people saying that your job is going to change no matter what job it is that you’re holding. I think we have to pick our heads up and notice that and say we are moving from a period of experimentation. To a period of implementation where we are beginning to see CEOs, not [00:03:00] only demand ROI, but to paint the picture of how it’s gonna roll out and to be able to set those expectations.

Now, what I’m really encouraged by is, in this announcement, there weren’t these kind of hyperbolic statements of. we’re gonna do mass layoffs or this is how many people is gonna say no. It was just a very measured, every single job is gonna change. And I do believe that matches what we are seeing as AI rolls out, right?

What we’re seeing is everybody’s job actually transforms and it changes, and we still need their expertise. We still need that role, but. In order to do our jobs better, in order to do them more efficiently, there are new approaches, new tools, new processes, new systems we have to think through. And so I think this can be kind of a bellwether for other businesses to.

Just take a peek at and say, okay, [00:04:00] if the largest employer in the world, 2 million employees is saying every single person’s job’s going to change, how do I begin to think that same way and not just look at one department and how many, how much head count can I change? No. How do I actually take and re-skill every single employee and how do I embed AI in.

My default thinking of how I am gonna do my job. And that doesn’t mean every single task, right? There are things that are uniquely human, but it does mean recognizing that everybody is impacted by this technology. And as leaders, as employees, as just human beings, we have to start to recognize that we live in a totally different world.

Courtney Baker: Yeah, absolutely. Such an interesting story. And man, it’ll be interesting to see as you think about what this could look like for Walmart, you know, do they get ahead of the competition quickly? You know, kind of reimagine the whole [00:05:00] experience and what that might look like. It’s pretty exciting,

David DeWolf: Courtney, I think the key thing of what you just said is reimagining that experience. It’s all about opportunity, and it’s one of the ways I really appreciate that Doug actually communicated this. He talked about the opportunity, the goal to create opportunity for. Everybody. Right. And I think that’s an important thing for leaders to think about is the communication, the message.

How do we roll out this type of transformative change? I think it needs to be framed in a way of opportunity, not just slash and burn and cut and looking for cost savings.

That’s when it’s gone upside down.

Courtney Baker: I mean, you can really imagine like, what if Walmart becomes, there’s like two times as many Walmarts as there are today, and they’re more centrally located to residences, you know, so it’s quicker and there’s

David DeWolf: Totally.

Courtney Baker: there because AI is, I mean, you could just like go on and on, um, imagining what it could mean and maybe even a much more profitable [00:06:00] company.

David DeWolf: Absolutely. Delivering more value to customers

Courtney Baker: Yep. I, I could use a grocery store in the front of this neighborhood. I can tell you that. David, thank you as always.

Courtney Baker: Lean and Six Sigma in operations and supply chain, agile and Scrum and software development NPS and CSAT in professional services. In the era where AI is rewriting how work gets done in real time, what’s the shelf life of a best practice? I sat down with David Am Mohan recently to get their thoughts.

David Mohan. Hey, how are you? Y’all good?

David DeWolf: How are you? I’m good.

Mohan Rao: Doing great.

Courtney Baker: actually, everybody on this call knows I’m the one that’s not good, um, because I made a rack of technology over here. Speaking of technology, YouTube [00:07:00] for decades. Everybody, business leaders, we’ve all been talking about best practices.

You know, what are they? How, if we’re not there, how do we get there? Uh, if we’re there, how do we continue to improve? It’s kind of been the benchmark, but in the age of ai, best practices and may actually be impediments to growth in this transitional time, what worked yesterday obviously could be obsolete or already obsolete, and you just don’t even realize it yet.

Um, so David Mohan, I’d love to dig into why success in the AI era demands a new playbook, or to put it maybe a, a little bit differently, the ability to continually rewrite your company’s playbook as you go. What, are your, your thoughts on this?

David DeWolf: Well, you know, there’s always been a lot of different opinions around best practices, and obviously [00:08:00] they have a place in, in corporate America, um, and, and in businesses. Uh, but I think a lot of the, um. Arguments against over-indexing on best practices have really come to life with ai and, and one of the staples that I think of is that best practices in general are static by nature.

Right. Of course, there is a long-term ability to update them and, and have the newest. Just the concept of best practices is this long-term thing of like, here’s how we do things. And we are now living in an age of AI where things evolve so rapidly and there’s so much breakthrough that I think, um, that, that is exacerbated, uh, in this era.

The other thing I, I think that has been a complaint against best practices is it, it limits. Innovation and, and really mastery, right? Best practices are in place for. [00:09:00] Um, situations where we need to know what good looks like. They don’t necessarily enable people to understand the underlying values and principles and then leverage those to create the best solution, right, or to innovate and to come up with new ideas.

And so because of that, they’re, they’re often. Misplaced there. There are frameworks that have good meaning, but they can be, be used in ways that undermine the optimal solution in a situation. And I think in the AI era, those fundamental complaints about best practices are just exacerbated and taken to the next level.

Mohan Rao: Yeah. You know, best practices that were rooted in, uh, first principles were always good and they’re still good. It’s just that when they become a static checklist of this is what you shall do, and, uh, when they get embedded and crucified in an organization, that’s the problem. [00:10:00] The good news is AI lives in constant change.

Any good AI system should be constantly. Adapting. And that’s why there is so much energy about agent architectures because agents are not predetermined set of steps, but are adapting to the, uh, context and the situation. So in that sense, best practices needs a reboot. And in the AI era it’s all about, um, adapting to the context of the situation.

Courtney Baker: You know, it’s, it’s interesting, Mohan, when you talk about agentic and those structures, Being built differently. But right now, you know, I can imagine as people are listening to this and they’re like, yeah, but that, those frameworks for like the lay person, you know, maybe in teams that are not, uh, technologist, it’s so much work to maintain those and they break, you know, they’re very brittle.

What is your, like, recommendation or, or is there a hope for those of us

Mohan Rao: Yeah.

Courtney Baker: [00:11:00] technologist?

Mohan Rao: Yeah. You know, when we are calling for end of best practices, uh, it doesn’t mean chaos. Uh, right. Uh, what what it really means is, um, it’s a discipline of constant reinvention. It’s the discipline based on first principles, uh, that’s what’s as important. So these static rubric of, uh, here is how we do things and everybody in the company shall follow.

Um, that could be really an efficient way to not produce what you want. but, uh, you know, having something that’s adaptive is what we are looking for in, in, in effect creating learning context each time where the company gets better each week and each iteration, uh, is what you’re looking for. And these are things that we’ve done in certain disciplines before, not.

Really well, you can do it, um, in some situations, but the question is how do you use AI to make that better and better and better?

David DeWolf: I think to your point, AI really has the ability to [00:12:00] help us contextualize, um, and really that contextualization a big piece of what we’ve been talking about that makes best practices rigid, right? Is they’re developed in isolation. Um, they are put out there as a. Frameworks or the quote, best way to do something for everyone, in all peoples, right.

And we all know that, uh, rarely, uh, are organizations or situations cookie cutter. And so what you end up with is a good enough solution that works as opposed to maximizing the solution. And, and I think AI to your point, because for the first time we’re really digitizing, uh, the knowledge work and, and we have the ability to infer.

Based on the contextualized information, uh, we can now come up with, um, next practices. Right? Whatever it is, it’s the, the contextualized, what is it that is the practice, the next practice. For this situation [00:13:00] right here, given all of the surrounding context, and all of a sudden we have tailored solutions that are based on learnings from the past, which is absolutely what you want.

If you think about your highest performers, the highest performers aren’t just replicating what they did in previous jobs and say, well, I did it here this way, so I’m gonna do it here. No, they understand the fundamental principles behind why it worked. They understand how it applies to this new situation, and they are able to tailor make the solution for the contextualized situation at hand.

And I think that’s what we want and we may now have the ability to really embrace AI to help further that and take this entire concept to the next level.

Courtney Baker: Yeah, I think this is such an interesting conversation. I think for everybody listening, you know, you may. Come back to this episode, the next time you hear somebody say, well, we’ve always done it this way. Uh, that may be a flag that, hey, we need to frame this up differently, this [00:14:00] new era, what you really need more of in this AI era is people who are asking, why have we always done it this way? And what could we do now that we have ai? How could this look different? David Mohan. Thank you as always.

Mohan Rao: Thank you, Courtney. Thanks.

David DeWolf: Hey Courtney, that’s a wrap.

Courtney Baker: Oh my gosh.

Courtney Baker: Too many firms are still chasing new logos while hidden churn quietly drains their revenue.

At Knownwell, we believe exponential growth starts with the clients you already have. If you’re interested in seeing your data on the Knownwell platform to find out exactly what commercial intelligence can do in your business, go to Knownwell dot com or just send me a message on LinkedIn.

[00:15:00]

Courtney Baker: Andy Sitison is the CTO of Share more stories as well as a startup founder and advisor. I recently got the chance to talk with him about how you can ensure your company supports and amplifies human experience in the age of ai.

Hey Andy, welcome to the show. To get started, I would love if you could just give a quick introduction of yourself and where AI fits into your story.

Andy Sitison: That’s a great, uh, opportunity for me to talk for 10 minutes and I will not do that. I’ve been doing tech for over 30 years. I’ve always been in emerging tech and I’ve always been about people, process and technology, how they all come together.

But let me tell you about the AI side of this equation, and that is after doing a global run of high tech. Um, I really started looking at the merger, the crossroads where human and digital meet. [00:16:00] And I said, I really want to start allowing the humans to ride on top of the technology and not find themselves under it. And so AI seemed like a, it was just, it’s 2016.

It was a good timing for machine learning and I was an old data pro data management guy, so I, I dove in, spent four months just. Training myself on all the techniques and tools and, uh, been been at it since We have a little startup that we’ve spun up around it.

Courtney Baker: That’s awesome, Andy, you’ve emphasized that AI potential isn’t just about efficiency, it’s about driving human-centered change, and that certainly resonates, uh, with our, our team and a lot about what we talk about on this podcast. What does that distinction mean for you in practice, and why do you think so many organizations are still stuck on productivity as the main story?~

Andy Sitison: ~Yeah, that’s a great question and, and we could talk.~

~All sorts of iterations and versions of this. But let me summarize my thinking on it. AI is an automating tool. It is a productive tool, and it is a way for, you know, let’s say small. Groups and counties trying to work on flood strategies, you know, to, to increase their, their human capabilities because of ai.~

~So there, it’s a productivity tool. A lot of people worry about losing jobs over, but it’s also a place where resources can be gained from in, in, in human deployments. But let’s come back to AI as you you phrase it. How does it help humans besides productivity? Well, what it does best for me, and, and we can talk generative ai, we can talk predictive ai, we can talk, you know, all sorts of algorithmic tools and techniques and statistics, and I like to tell people.~

~If that, when you look at the way I use ai, think of NASA and the short sleeve white t-shirts and their pencils. That’s the way we like to apply it. It’s not all the buzz that you see or generating images of ex-presidents and things like that. Uh, what I love about AI is its ability to find pattern in complex data.~

~So to just take a set of data that you can’t get after as a human and then to just get creative with it. And, and when I say creative, not changing the data, but understanding what’s inside it, looking for variance, covariates, all the techniques of feature analysis, all these little things that you can apply and let the AI help you, um, expand your, your human abilities.~

Courtney Baker: ~Yeah, it’s so true. So many times I feel like. people are so focused on the job loss, but what I, I think we actually see is just people being able to leverage something they’ve never been had access to. You know, where previously they might not have the funds or the resources to go even, hi, hire another human to do a thing.~

~They now can do it themselves to leverage their own creativity to, you know, work towards the goal that they just wouldn’t have had before. Ai.~

Andy Sitison: ~Absolutely, and, and you know, it’s everything. I can give you examples and, and I tell people. First off, especially if you’re working with generative ai, treat it like a human and get mad at it. Like, treat it like a human. You wanna beat up, like, because it, it responds to that. ~

Courtney Baker: ~Yeah. ~

Andy Sitison: ~but also be creative.~

~That’s what I really was gonna say. I’m always being humorous, but, um, you know, it, it, it, you treat it creatively. Think it the heart, the, the best question. You, you can ask is the one you didn’t think of originally. And so I often will spend, uh, 40 minutes working on analysis of some project data. We fed it and then at the end I’ll go, okay.~

~Of what you see in the data there. What didn’t I ask about that I should asked about? ~

Courtney Baker: ~Yeah. ~

Andy Sitison: ~it’s almost always profound. Something will come back in a eight or 10 lists and I’m like, Ooh, I didn’t even think of that. You know? And ~

~so there we go. That’s another data point I’m bringing forward in our analysis.~

~So it’s, it’s brilliant that way. And it’s also creatively helpful in coding. Like we, we don’t necessarily write our code with it, but we might. Check the quality of our code, ~

~or are we the standard? You know, what would, how would you rewrite what we just wrote? Um, and, and I actually wrote a little folk tale for my grandkids.~

~And, uh, I, I’m not, I’m a pretty decent artist, but not an illustrator. And so I used it to generate illustrations for that folk art. So, you know, there’s all sorts of ways that it, it can augment and not re, you know, not necessarily replace people in the process.~

Courtney Baker: ~I love that. That’s such a great story and I, yes, it’s so fun. The creative things you can do besides just work. But you know, with your family, with kids, grandkids, it, it’s really fascinating, uh, use cases. ~

You know, one thing I wanted to talk to you about was, you know, you’ve spent decades at the intersection of business data and technology.

From your vantage point, how can leaders ensure AI supports transformation makes organizations more human and not less? ~You know, we can, we can talk about it a a lot, but like, how do we actually put that in the practice in our businesses?~

Andy Sitison: I’ll get existential a little bit and say it’s similar to capitalism or any other tool or technique. It is a productive tool, but somewhat devoid of value, right? Like

it is what we [00:17:00] make of it. And, uh, we need society, we need groups, we need teams

to, and cultures to, you know, round that out.

And humans are who define that. Like, I, I don’t, I have spent. Third over 30 years in the tech integration automation world. I’ve been hearing every year that we’re gonna lose all our jobs next year and no one’s ever lost a job or no, that’s not fair. Somebody’s lost a job. We, we never have seen the market change in a manner that jobs just completely go away because of the new technology.

And so I don’t think AI does that either, although it absolutely transforms what you need to be as a human in the future of business, in the future of organizations for public business, uh, public, uh. Work and so forth. And so I would tell companies and I, and you know, I heard your question. I’ll get to it now, uh, finally.

And, uh, and that is that, um, to apply AI first is to treat it cellularly. [00:18:00] It’s not a stack, it’s not a login. It is something that can augment anything. It’s, it’s almost like treat it like a service dog. If you look at all your employees and all your business processes. What needed a service dog? What’s not quite getting done right?

What’s not managed well? What’s more complex and you can handle? And then ask the question, could AI. Enable us at in that one piece because it, it’s really a, a, it’s not a, it’s not a stack, it is a stack, but it’s, it’s more like timeshare on a mainframe like in the old days. That’s really the way we’re treating it right now, and the way it’s productive is when it’s just enabling.

The people that are already in your organization and, and they need to have the skills to do that, you have to have a team to, it’s almost like a training function more than a deployment function. It’s like how do you train people to understand how to use it, how to trust, when to trust it, when to check quality, all that.

And of course we use a, we do a lot of deployment [00:19:00] work where we’re just structuring that data quality and locking down what those systems will do so that our, our researchers will, uh, not have to do that on their own.

Courtney Baker: That’s so interesting. I would love to talk to you a little bit about, uh, your work at Share More Stories. I think it’s so interesting. Um. Your work there, you use AI to surface, not just sentiment, but really those deeper drivers and emotion in people’s narratives. How does that kind of story driven insight change the way organizations make decisions compared to, say, you know, dashboards and KPIs alone?

Andy Sitison: ~Great. Yeah. ~You, you’re, you’re, you’re doing a good job of staging why we exist, and that is, um, if you look at qualitative versus quantitative data, we have so much data right now. There’s so much noise in every aspect of our life. [00:20:00] And there’s also decent signal in the metrics that, uh, you know, A CMO or a CXO or even a CIO will, will see coming through the, the desk.

Often when you’re, when you’re looking at that data, it’s still difficult to feel connected to these people. You’re tracking activities, you’re tracking surveys. Well, gosh, I’m getting a Likert scale of I really hate it. I really love it, but what does that really mean? Like, and, and was I in a bad mood when I did that review or did I really mean what I said?

Right? Like there’s just so much challenge to. The veracity of that data and to find the deeper soul of that community inside those, I call ’em monkey clicks. It’s kind of a rude term, but

I just, I just, can see the monkeys sitting there clicking all the buttons, right? And, and so it. It’s all valuable. I’m not saying that’s not valuable, but what we’re finding is, and you know, we started eight plus years ago doing this, so it’s what’s good is the market’s catching up with this.

Finally, we’re not just the, Hey, that’s really interesting, but people are [00:21:00] going, oh my God, I need you this week.~ And, and that what’s happening is people realize experience matters, and whether that’s employee experience, their customer experience, we call it human experience often. And, um, if you’re starting a big strategy and, you’re in, in a marketing role or maybe you’re in a nonprofit and you’re trying to connect with a community in, in a, in a state or, or the nation. Uh, how do I get to know that community? So what we do is we collect stories and we worked on our culture to be trusted because if you don’t create a sense of trust or the triangulation of trust with that community, you won’t get authentic responses and.~

~First, that’s important. Secondly, by collecting stories, three or four pages of, of a heartfelt story about something related to the topic, something happens to humans. When we process a story, we go inside, we kind of frame it up. We, we sit there and type. We’re not, we, we can’t keep a bad mood. You know, we, we, we don’t normally.~

~Just turn that into some kind of riff or rant against somebody, but really an inward look and, and it pulls forward kind of an emotional, thoughtful perspective of that person. And so what we’ve done is we have all sorts of scores, but I, I actually have a psychology undergrad that doesn’t bind me a whole lot, but it did.~

~I did spend, you know, one of the first years, a year and a half of the time just looking at 150 years of kind of case log against psychological measurement of, you know, what. We humans have measured our humans and, uh, ourselves with. And so, you know, this isn’t, you know, Myers-Briggs and, uh, big Five and all those different techniques.~

~And we’re not necessarily saying we’re measuring the personality of the individuals that are collecting stories from, but we are starting to get something similar to a personality of that community. Through their stories and, and that allows us to see joy or anxiety, uh, self, self-transcendence, activity level, and start to marry those different feature sets to start to tell, uh, a story or see, you know, possible problems or great subtle things that are happening in that community or customers can enjoy and, and, and work with.~

Courtney Baker: ~You know, this is so interesting. Probably for every marketer that’s listening, like we’ve all gone through persona interviews and you’re sitting down, I mean, I have an, an image of me. ran outta room on the board. We were in the floor with all these post-it notes trying to find those patterns in these stories.~

~It’s actually. Pretty difficult to do. And so I read some of your case studies. It was really interesting how all were using AI to help take this, these stories, you know, it’s what connects us to each other. It’s like where it’s, where all the emotion is this in our stories and actually use AI to help, help find those patterns and really surface it in a way that, um.~

~People can use in their businesses. Really fascinating use case. Uh, maybe for all the marketers it means less stay in the floor, uh, with the Post-It notes.~

So one thing that you’ve said previously about safeguarding authenticity and identity as core to your work, in a world where. Generative AI can easily kind of distort voices, fabricate personas. How can organizations protect the integrity of the people they serve?

Andy Sitison: Wow. It’s a, it’s a really interesting topic

because algorithmically we’re all getting put in buckets, right? We’re all marketing buzz buckets, as they say. If you’re not paying for it, you are the product,

right?

And we are definitely the product right now in so many ways, uh, to the point where you sometimes you don’t feel oriented.

~Right. And, um, so let’s talk about digital twins. You mentioned earlier like the idea of personas and you could tweak what we do and, and absolutely describe that same model that we’ve been doing for, uh, you know, we, we spend a little more time customizing that, but we segment those, those we use segmentation as part of that effort and it essentially generates what you’re talking about.~

I would tell you that our customers, want to know who’s, who is out there. They, they’ve all done the work with the personas. They’ve all defined their, their segmentations, and they don’t not know their [00:22:00] customer base.

They just don’t know what they care about. ~Like, one of the things that’s interesting, I’m the tech guy.~

~I’m the, I’m the science guy and I love the details of the data. None of ’em ever care at the level I do obviously. ~And, um, you know, one thing that’s always surprising is they’ll get in the process, you know the names, share more stories, and then you tell ’em what you’re doing and then you get in there and then they get just a couple weeks in and get a story or two, and they’re like, oh my God, these are stories we’re like.

Yeah, that’s what we said. We’re gonna collect with it, you know, stories and, uh, but this one’s like, this person talks about going to the Y and you know, like was her recovery from her mother dying in her arms and like, you know, she became a, and you know, I can’t believe we’re getting this content and so I don’t wanna downplay in any way what we hear from those stories.

But the person has the autonomy to name themselves however they want, and that becomes their identifiable connection to the story and their demographics and, in that space, we also don’t necessarily share all the content of those stories with our sponsors or allow them to carry that forward to publication of any sort, unless that’s part of the initial part of the [00:23:00] project where we agree, get our participants to agree, Hey, are you willing to share your content further?

And if they say no, they don’t get to use it. So we will and. Not surprising. It’s therapeutic for people to tell their story. It’s a voice for them, and a lot of people do wanna get that voice out there. They do want their stories shared, whether it’s with their name or not.

Courtney Baker: You know, there’s a whole new box of AI ethics. It often, can feel more. Abstract or like a compliance driven thing. When we think about AI ethics, what are, you know, some practical ethical considerations?

Leaders should keep in mind if they want their AI initiatives not to erode, trust not to erode, you know, the ethics that they most likely feel very strongly about.

Andy Sitison: Yeah, I, I think one is, uh, assigning leadership, you know, chief experience officer’s a, a hot new C level that we’re seeing all over the place. And [00:24:00] that by the way, helps us a lot because people finally understand what we do and, uh, and so that’s good, but. I think it’s, it’s very much about, uh, it’s not a free ride.

It’s not, it’s not a one button push tool set that you’re, you’re implementing. You gotta care about your culture, your people, your customers, and that needs to be systemic in your process. ~So, you know, if those are broken. You gotta work on some of that activity because if you start using us to ferret out the problems, but you don’t want to deal with it, it’s in disingenuous, actually hurts us.~

~You know? It hurts, hurts. Those that go to help you get connected because you so it, it’s a, it’s a holistic process and.~ Most companies care about that. Although I’ve, I’ve seen a few that really didn’t. They, they, they might do a step or two to make it look like they care. Um, but most care about that. It’s just a matter of how do you do it?

And then it’s a function of improvement. It’s transformation. It’s just like everything we’ve ever done. It’s just like when, you know, we went from typewriters to word processors and, uh, document management systems in the cloud. It’s, it’s really about how does this enable your business? ~And, you know, ai, uh, just remember.~

~There’s this, there’s inherent bias. You, I like, I like this example. You take two schools, 500 kids in both schools. You teach one set of kids what a blue ball looks like. The other set of kids, you, you teach ’em what a red ball looks like. And then you put the two kids together and you put a blue ball through, you’re gonna get an answer to that.~

~That is not a red ball, it’s a blue ball. If you put a, a blue ball through, you’ll get Well, that’s, I, I can’t remember if I got red or blue balls there. I lost track of my, my ball color. But anyway, you’re gonna know, ’cause you’ve trained it to do that. Now you put through a brown square. What does it do with a brown square?~

~Or even worse, a purple square, right? Like it, it might say, I don’t know what that is. It might say it’s a red ball, it might say it’s a blue ball.~ The, the thing about ai, even though it’s sometimes immense and vast, its amount of training, it’s still just trained on images and labeled data, [00:25:00] simplified data, so it only knows what it knows, and it may get it right.

It may not get it right. So the further you take it from its core capabilities, the more you get the hallucinations as they they like to call it, and, and so you get you. It’s a function of learning how to use a tool. It’s just like hiring an employee. It’s like an employee than it. If you, if you’re using it, know that you’ve got training to do.

You’ve got management of that person to do. You’ve got some. Some care and TLC to do with that technique or technology that you’re deploying. So, you know, get to know it, understand how to cha challenge it and, and know how to limit it from, you know, hurting you because it is so dynamic. You, I mean, 10 years from now, it’s not gonna matter, but right now it’s really about da.

You gotta look the data quality, you gotta look at the boundaries of how you apply it and you know, I’m beat around that bush enough. I think I covered it one way or the other.

Courtney Baker: Well, Andy, we always like to ask at [00:26:00] the end of our interviews for a little advice from you, so if you are a CEO. listening right now. What’s one move you’d recommend they make in the next 12 months to apply AI in a way that really advocates some of these things that we’ve been talking about with ethics, authenticity, and human-centered change while still, you know, delivering business impact.

Andy Sitison: Yeah, that’s a great question. I’m looking at a center of excellence around that. I think a

COE strategy’s pretty good. You, you need to have some competence in the house, but that’s not just an IT team. That’s not like, don’t let the CTO like me get a hold of and think, okay, we’re gonna teach all the

tech and the, you know, the pendency and all that it, I tell people I’m a practical AI pl, I’m not an AI pundit.

Don’t ask me what the freshest model is. I won’t know because I deploy this stuff. And so [00:27:00] I would say it’s about. Having a, a center of knowledge around this and a training program, it, people don’t naturally know how to use creative tools. It’s just like if you rolled out a bunch of magic markers to your company, you might get a little art, you might get a lot of graffiti.

You don’t know what you’re gonna get because they need to know how to use it. And that’s a closer to AI than I think electronic spreadsheets. I think, uh, in the eighties, whatever. So I think it’s really about considering the well-roundedness of how it fits in your group. Consulting around that, being able to go in and talk and ed and listen, and then educate people on it.

And then measure how it helps you. You know, like spend a little time going, you know, do we save 20% of time? Do we, do we learn new things? Do we grow our revenue? it’s standard business stuff. It’s just a new one. It’s a new piece to play with.

Courtney Baker: You know, that’s so interesting when you were using that analogy of rolling out some markers. You know, it’s. if we were to roll out [00:28:00] Photoshop to the whole company and ask for an outcome from it, you know, there would be a portion that would excel and do very well. You know, there would be a big group in the middle that, you know, they may, you know, they got something and then another group at the end that just would totally, you know, they’d be good to get the canvas up and going and, but obviously we know that so.

you know, with a tool like Photoshop, it’s so interesting, even with the large language models, that because it is so language driven, we just kind of assume, I think that, hey, everybody’s gonna get this. It’s just

Andy Sitison: No.

Courtney Baker: prompting and it’s, it’s not, I mean, it’s exactly what you said of like, training, giving people the tools to really, um, do this well and not taking it for granted that it is, um, easy for

Andy Sitison: I think it’s a really great point and I, I’ll give you some strong data on that. I, I presented at a conference [00:29:00] recently. I thought it, it was mostly science and researchers. I went in with what I thought was a reasonably basic layout of what generative AI AI was and how to do prompt engineering. Some really, I thought, basic with full demo, go along, lecture the whole thing, lost them 10 minutes in, couldn’t get ’em back.

You know, there was just, they just weren’t there yet, and I needed another hour to get them to where I was. On the other side, you talk to teachers, you, you know, okay. The group that’s always challenged with technology and, and doesn’t always get what they need. Teachers get it innately, like all

the teachers immediately were a group that just knew how to use ai.

They’re like, oh my god. Now I can send teach letters to the parents without getting ’em angry at me at the end of the day. And my point in that is don’t assume, the level of adoption of each group, it’s, it’s about efficacy comes from, you know, a lot of little tool sets in this, like, you know, my, my partner James, the CEO is an English major.

His prompt. Engineering is [00:30:00] brilliant, and I’m, I’m a short TL;DR guy, right? Like, I love the short, so I have to stop and go, what would James do here? Like, how would I, how do I phrase this in a way that I don’t lose the model? And so, yeah, it’s, we’re all a little different. So, you know, thinking like, again, consulting, testing, training, and then measuring how well that’s going and, and you’re gonna have some real success with it.

Courtney Baker: Yeah. Well, Andy, thank you so much for joining us. It’s been a pleasure having you with us.

Andy Sitison: Well, it’s been great to join you guys and, and talk a little.

Courtney Baker: Thanks as always for listening and watching. Don’t forget to give us a rating on your podcast player of choice and listen, we’d really appreciate it if you would share this episode with someone you know would enjoy it. At the end of every episode, we like to ask one of our AI friends to weigh in on the topic at hand.

So, hey, Claude, welcome back to the show. Today we’re talking about the end of [00:31:00] best practices and why AI demands a new playbook. What do you think?

Judith: I think you’re onto something – AI is fundamentally changing how we work. The old rulebooks might not cut it anymore when machines can suddenly do things we spent decades optimizing humans to do.

Courtney Baker: And now you are in the know. Thanks as always for listening. We’ll see you next week with more AI applications, discussions, and experts.