After spending the last two weekends putting the finishing touches on our new AI ROI Cheat Sheet, my mind is swirling with quotes and AI ROI “rules of thumb” from some of our smartest interviewees and research advisors.
There are two lessons that come through time and time again in our most recent batch of AI ROI research:
- Determining AI ROI isn’t easy. It often takes AI startups two or three years to determine the core “needle-moving” metrics that they use in presenting a business case to clients. There is a balance between measurable (often financial) ROI and strategic ROI that must be maintained. There is a balance between “metrics our client cares about” and “metrics we can reliably impact” that must be considered. Smart AI services providers know how critical it is to present an ROI case, and take the time necessary to do it well.
- Framing AI ROI the wrong means failed (or short-lived) projects. For AI consultants – or for internal innovation leaders within an enterprise – nothing is more frustrating than taking months and months to select a vendor, hire data scientists, sign a pilot project, and simply have the entire effort wasted by poor expectations of results. Calibrating ROI the right way allows companies to benefit from more than just the financial gains of AI, and to turn the journey into a benefit as well.
In this article, I’ll quickly highlight three reasons that AI ROI predictions go wrong – based on the testimony of enterprised AI buyers, and experienced AI vendors.
1 – Insufficient Measurable ROI Assessment
Issue:
Consultants or advisors who go along with a client’s stated financial ROI goals can run themselves immediately into trouble.
So the client wants “cost savings” in their call center.
Given the range of AI applications that might be deployed, we may have better success in improving customer satisfaction rate, or in improving closing rates for phone sales staff, or in any number of other potentially measurable benefits that might (or might not) tie immediately to cost savings.
Solution:
No project should begin without a deliberate, cross-functional effort (involving in-house data scientists, relevant vendor partners, SMEs, and leadership) to map the potential “needles to move” for this particular AI project.
In an ideal world, all parties are involved in this process, but the fact of the matter is that most of the time, the onus for doing this right – for determining the range of possible metrics to use and how they might be measured – will fall on the vendor. The executive cutting the check often isn’t interested in generating these ideas. SMEs often don’t have the time or context about AI to generate these ideas. It behooves a vendor to ensure that all parties are involved in ranking, prioritizing, and deciding on measurements.
This is because people are more likely to move forward with a vision that they themselves helped to create, and because context from people within the business is important to pick the right goals in the first place).
Keep the initial options wide. It’s not just “how much money will this project save?” or “how much money will this project make?”
It’s “do we have strong baseline measurements for this part of the workflow to know if we can help shave that time down?” or “can we develop a more real-time customer satisfaction metric to genuinely attribute the improvements from this AI project to customer experience?” Zoom way out before zooming in and deciding on measurements. Going along with what is suggested first is limiting, and often unrealistic.
There is no substitute for the team buy-in and genuinely good ideas that spawn from this process when it’s done well.
2 – Neglecting Connection to Strategic ROI
Issue:
Projects are often sold with an emphasis on measurable financial ROI only. Even if nods are made the strategic value, they are often vague ties (i.e. “Well, the company said they want to be innovative,” or “Well, the client said that reducing their delivery times would improve customer retention and market share”).
When the going gets tough – as it often does in the messy world of AI enterprise deployment – these token references to strategic value aren’t going to convince leadership to double down and make the investments they’d need to keep a project afloat (or to pivot a project in a new direction and retain the valuable lessons learned).
Solution:
Ank questions and get a deep understanding of the strategic value that you can tie your AI initiative or project to. Talk to many stakeholders, and get a sense of company-wide, department-level, or individual-level strategic value that you can appeal to. Examples might include:
– 3-5 Year Goals
– Key Thrusts (Major Projects for This Year / This Quarter)
– Company Vision / Digital Transformation Vision
– Differentiators / Competitive Advantage
– Etc…
After collecting many ideas for strategic value alignment, rank them based on which kinds of strategic value matter most to the client, and which can most be impacted by this AI project or initiative.
Having stakeholders see an AI project as being in line with long-term value gives the project freedom to breathe, to grow – and the potential to pivot and iterate – instead of being canceled or phased out as soon as inevitable adoption hurdles come up.
One of our best interviews about connecting measurable (near-term) and strategic (long-term) ROI is with David Carmona, General Manager of AI at Microsoft. I highly recommend listening to this episode in full:
3 – Downplaying Adoption Barriers
Issue:
Every AI project will involve overcoming unforeseen challenges. The assumptions we had about our data is wrong (no small problem). We don’t have subject-matter experts who are willing. Our in-house data science team doesn’t understand the problem domain well enough. The list goes on.
When these issues aren’t addressed up front, the inevitable challenges (to data access, to data quality, to iteration time, etc) become seen as “waste”, and enterprises lose hope, cancel projects, or get rid of vendors after a “sandbox” project, with nothing more to show of it. No lasting value for the enterprise, no lasting AI transformation relationship for the vendor or service provider.
Solution:
Read our article Critical Capabilities to get a more complete framework for thinking through and addressing AI maturity, and the prerequisites to AI deployment.
Many AI vendors and service providers put off these issues, seeing them purely as barriers to “closing the deal”, instead of treating them as they should be treated: As part of the value of the AI solution itself.
Taking this mature approach means building trust, setting expectations you can deliver on, and turning an AI project into a lucrative AI transformation journey (there is no free lunch here, there is nothing “plug and play” about AI, this is a journey). Take the core strategic objectives of your client (what we refer to as “strategic anchors”) and tie Critical Capabilities to each of them.
Does the company have a long-term goal of having the best possible customer retention in their specific eCommerce market? The overhauling of the CRM and purchase data infrastructure will be an enabler of this long-term aim, not just a “cost” for “plugging in” a one-time recommendation engine.
Does the company wish to strategically differentiate by having a phenomenal corporate culture? The education of C-level executives and functional business leaders on AI use-cases and applications is a facilitator of this aim, not a “time suck” related to a one-time AI project.
