AI vendors and enterprise buyers struggle to get on the same page about the return on investment (ROI) of AI solutions.
Misinformed or uneducated buyers look for quick turn-around and measurable results within an unrealistic, bounded timeline.
Eager vendors are often quick to agree to a buyer’s unrealistic expectations if it means winning a pilot deal – but this eagerness often turns to regret as overpromised results ruin the client relationship, and prevent the project from becoming a real deployment.
Smart AI vendors (i.e. those with talented data science staff and real experience implementing AI with enterprise clients) know that this dynamic is bound to lead to frustration and failed projects.
In this article, we’ll discuss some of the challenges to measuring AI’s ROI, and we’ll present our Trinity model for estimating and presenting AI ROI to leadership. We’ll also link to a variety of related Emerj articles that explore the themes of Measurable, Strategic, and Capability ROI in greater depth. If you need to convey the value of AI – and the honest challenges of AI deployment – to leadership, this article should serve you well.
The Challenges of Determining AI ROI
There are countless reasons why AI projects go wrong – but there are also a variety of reasons why AI ROI is difficult to predict or manage, the following being among the most important:
- New technology means nascent use-cases: AI is greenfield technology. Many AI use-cases are being tried for the first time. Very few established sets of best practices exist to consistently deliver value from AI projects of different kinds.
- Iterative process means ROI isn’t certain: Traditional software is deterministic AI is probabilistic, it is experimentation. An application that worked well for a competitor, won’t necessarily translate to our business – and an application that worked well in one department might not work well in another part of the business.
- Iterative process means timelines aren’t certain: Tinkering with data, features, and weights takes time. It takes time to coax out patterns from data, and finding the junctures of business processes where those algorithms can contribute value. Having set timelines is wholly unrealistic.
- AI transformation implies AI maturity: Skills, resources, culture – companies need to work with data and teams in entirely new ways – and successful AI projects often require a gradual build-up of AI maturity factors.
This doesn’t mean AI ROI is not measurable, or that no efforts should be made to measure it. To the contrary, at Emerj, we believe that the challenges presented by AI ROI measurement are actually opportunities for smart vendors and smart enterprise leaders who can gain a nuanced view of AI and push forward towards a deeper – and more strategic – AI value for their organizations.
When estimating or presenting the ROI of an AI project, we recommend thinking through more than just measurable, financial results:
Emerj’s AI ROI Trinity Model
Emerj’s Trinity model for AI ROI involves three main components.
Measurable ROI refers to the quantifiable aspects of AI project impact – which could include financial (cost savings, revenue increased) or non-financial (reduced manufacturing equipment temperature, improved self-reported customer service scores) measures.
Strategic ROI refers to the ability for AI to influence the long-term strategic goals (3-5 year goals, current thrusts or initiatives, digital transformation vision, etc) of a company.
Capability ROI refers to the impact of an AI project on a firm’s overall level of AI maturity – which includes skills, resources, and culture required to deploy AI successfully.
We’ll explore each of these ROI types in greater depth:
Measurable ROI
With measurable ROI, we want to think about how will this project be able to make us money – how will this project be able to save us money, and how will it potentially be able to reduce risk? Risk reduction is the hardest to measure, but there’s occasionally ways that we can extrapolate it.
Measurable ROI boils down to one of the four factors:
- Increased efficiencies or money saved – such as automating parts of the underwriting process, or using computer vision to reduce the time required to examine heavy equipment.
- Increased revenues or money made – such as product recommendations in eCommerce, or customized email messaging.
- Reduced risk – such as AI solutions for cybersecurity, or legal document search and discovery.
- Non-financial measurements – these metrics might include self-reported customer satisfaction scores, or time-to-resolution for customer service. These metrics don’t tie directly to the financial impact, but we estimate that they heavily influence customer loyalty and churn.
Remember that making money can be in a variety of ways. It can mean we get more customers, it can mean that we make more money per each customer when they transact with us. It can mean reducing our churn rate so that we retain them for a longer period of time, and they add more dollars to our coffers over their lifetime, whatever the case may be.
With saving money, again, this could be saving time, which we can translate to money, the number of man-hours required to achieve a specific task, potentially completely automating a task so that we can reallocate humans to something more productive, and we can save all that sunk cost to not involve people at all.
There’s a lot of ways we can potentially save money. We can save fuel if we’re an airline, and fuel costs money.
Related Emerj Plus articles:
- Your AI ROI Predictions Are Wrong – Here Are 3 Reasons Why
- The ROI of Machine Learning – 3 Strategies for Measurable Results
Strategic ROI
Then we can move into strategic ROI, where we want to think about how does artificial intelligence fit into the strategic goals for the company. We talked before about what those are. When we’re thinking through these, as mentioned in this graphic, let’s go ahead and pull it back up, we want to filter these potential strategic ROI connections. So, we might have a five-year goal of being the number two competitor for market share for this kind of baby food, or we might have the current initiative and thrust, a really, really big push for tightening up our cybersecurity.
We might have a key differentiator of having better customer service and a better customer experience online than any of our competitors, whatever the case may be. So we have all these potential strategic anchors. The question is, we want to take our projects, put them through these filters here and figure out which ones are relevant to this AI project. We might be working on a project that really doesn’t have anything to do with cybersecurity, in which case, we’re not going to tie it to that strategic anchor.
So we want to really look at relevance for the initiatives. So what are the strategic anchors? What are the client’s strategic goals that actually align to this project? We take one project, we put it down and we say, which of their high-level strategic anchors tie to this project? That’s the first question. Which are relevant? The second question is, which does leadership care the most about?
That’s the second question, and that’s really, really, really, really important. Once we ask the second question, then we’ll have a very short list of potential strategic value points for this AI project that we’re going to be able to bring up when we get into the presentation phase.
Related Emerj Plus articles:
- How to Succeed with AI Projects – Lead with Strategy
- AI Strategy in the Coronavirus Era – a Business Leader’s Guide
Capability ROI
This is about building capabilities to enable future AI innovation and to unlock the value of data.
The majority of buying stakeholders think of AI like any other IT solution. It needs to be plugged in – and then it needs to deliver its results. AI requires an entirely new way of working with data and teams, an understanding of AI use-cases and applications, and cultural changes in a firm’s willingness to experiment and iterate.
In the words of Silicon Valley AI consultant Dr. Charles Martin, “AI is not IT.”
In my speaking engagements with corporate clients, I often say it this way:
“Every AI project should not only be guiding a firm towards near-term measurable ROI, and towards long-term strategic outcome, every AI project should be an investment in a company’s ability to leverage AI to win in the market.
Any AI project that does not build AI maturity is a failure.”
Clients often don’t understand this at all – most clients are focused on near-term Measurable ROI with little to no concern about the other two factors. For this reason, internal AI champions or outside AI consultants must educate their buying stakeholders, and present AI maturity (i.e. re-training team members, overhauling or altering data infrastructure, building a culture of iteration, etc) as a benefit in addition to a cost.
No AI initiative will be a magic bullet, a market-winning single capability. Many projects will fail – but firms who succeed and leverage AI nimbly into the future are those who will reap the benefits of AI.
Related Emerj Plus articles:
