In the final installment of the “Pitfalls to AI Adoption” series, we talk about predicting the ROI of AI. There are a lot of misconceptions running rampant around the ability to gauge the return on investment of artificial intelligence. In this article, we talk about what can and can’t be done when it comes to investing in artificial intelligence and predicting what the return might be.
- How much will an AI project cost?
- How long will the integration take before the software is ready to use?
- How will it be able to be used to drive value?
- Will it be able to do what you want?
- D0 you know for sure that the software is going to achieve the outcome that you set out to achieve?
None of these questions are easy to answer. We’ll lay out exactly why this is and go into some of the critical reasons why it’s very hard to gauge the cost of an AI project, the time to completion for an AI project, and whether or not the project is going to work out.
Pitfall 3: Projecting the ROI of AI
Artificial intelligence projects require data. If you’re doing an AI project for fraud detection, you need to have data about your customers and about fraudulent and non-fraudulent payments, for example.
Any kind of application is going to require data, and the fact of the matter is you may not have enough data. You may not have the right data to train a system to get done what we’d like to get done.
The subject-matter experts and data scientists at the company could all agree that AI is right for the company, but the fact of the matter is there is no way of telling how long it will take to clean, organize, and harmonize the data so that you can feed it into the system and train the AI.
Once we have the data infrastructure in place, now we have to start training models. Once we’re training models, we have to see if the AI is generating a potentially better recommendation than it was before. The only way to test those recommendations might be to expose them to our customers. Now we have to have a way to measure that.
We have issues with training the algorithm, seeing if the data is even capable of being able to train this algorithm in the first place, and it may not. We may spend six months trying to train different algorithms with the kind of data that we collect and realize it’s not better than the status quo.
A lot of AI applications will interact with customers, and these happen to be the most challenging to bring to life for a variety of different reasons. One of these reasons is the percentage of AI talent at a company, data scientists, machine learning engineers, that have actually built AI software that have been deployed in business. This is rare.
In addition, how do we measure the results of the AI product? Even structuring the assessment of measurement is a challenge.
Up to this point, it may sound like artificial intelligence is a poor investment. If we can’t gauge the returns, if this is going to take longer than we think and there are so many points of failure, why would we invest in artificial intelligence in the first place?
To be frank, that’s a very good question. Right now, most midsize companies with absolutely no data science savvy probably should not be investing very much in artificial intelligence.
Even in spaces like banking and pharma, there are a lot of low ROI applications. They are running at a loss, but it’s still very time-consuming. It’s still very expensive.
It’s right to say, “Wait a second. Should we really be using AI?” In the future, a lot of these challenges to projecting ROI on an AI project are going to fade away.
If we can’t predict how artificial intelligence is going to produce a return on investment for our company, what kind of ROI considerations can we make? What can the boardroom and the C-suite actually be doing when it comes to thinking about AI ROI?
There are two things that they can do.
Discovering the Landscape of AI Applications in a Sector
We may not be able to know the ROI of individual use cases, but what we can know the landscape of AI applications in a sector.
If you’re a banking executive and all you do is read banking press releases before you spend money with AI vendors, you are inevitably going to be investing in the wrong places. It makes sense to get a deep understanding of where a return on investment is being garnered with AI in your sector.
In addition, it’s worthwhile for companies to consider what core AI capabilities are in their sector and how those capabilities overlap with their company strategy. For example, a data scientist at Staples told me the other day that Staples is focusing right now on recommendations. They’re re-imagining their business in the era of Amazon, and recommendations is a very big priority for them.
When we look into eCommerce, we see AI capabilities in email marketing, recommending content, recommending large ticket products, recommending smaller consumer products. There are different trends and different AI capabilities that have different levels of traction in different industries.
Do we want to use conversational interfaces? Do we want to use email? Do we want to do on-site product recommendations? The likelihood of garnering a return can be quite different from those different areas, but it is at least worthwhile for us to know as a business.
Then we can go out into the world of AI and find capabilities and applications that plug into that. Even though we might not know what the return is going to be, at least we’re aligning technologies that we can assume work from robust research in the space with existing company strategies that are not going to steer us off course.
Identifying Self-Contained AI Use Cases
It’s also worth identifying self-contained AI use cases that can be brought up to speed relatively quickly. Many AI applications in the enterprise are extremely challenging to set up.
There are some AI applications where the barrier is much lower. For example, there are some applications of computer vision for security where the systems are more or less trained for the most part. In other words, the vendor already knows what a person looks like when they’re walking on a screen. Identifying certain kinds of vehicles or people may already be hard-coded into those algorithms. An application like this can often be a foot in the door to see what it’s like to use these technologies, understand these technologies, speak with vendors without having to open up all of our proprietary data and rehash everything that we’re doing in terms of data infrastructure.
So when we’re looking for an ROI as a business, we might want to look for a technology we’re already shopping for that might be served via an artificial intelligence vendor in a way that is going to be relatively easy in terms of integration.
In fact, Madhu Shekar at Amazon, who we’ve spoken to multiple times for our podcast, mentioned that it’s important for businesses that have very low data science skills to often work on more self-contained projects.
In order to achieve success with AI, companies need to grasp what the AI capability space in their sector actually is.
The fact of the matter is that general business infrastructure does not allow for the kind of iteration, testing, and experimenting that AI requires. Instead, enterprises can think about AI adoption with a venture capital analogy.
If we think about a venture capital firm managing their returns, they’re not looking at a “to-the-penny” assessment and a “to-the-day” assessment of how much and when a certain return happens. Instead, they’re making informed bets. They have a strategy. They decided to invest money in different parts, presuming some of them could really explode and become very high capability investments.
Instead of looking at each individual project as something we need to project an ROI for, instead, we’ll manage a fund. A fund within an enterprise can roll the dice on a number of different projects. Again, we’re not picking random projects. We’re doing them in line with our strategy, but selecting a number of them and then looking at the return of the whole fund as opposed to individual projects.
It may very well be that philosophies of that kind have to evolve for AI to gain traction in sectors like banking.
Gauging when an AI project will be operational or if it’s going to be operational is extremely hard. The data needs, the training of the algorithm, the interaction with the customer, the way we’re going to measure this application, are all very large challenges and can take significantly more time and money than enterprise leaders often presume.
However, it’s important to understand our desired outcomes in our current business strategy and to mesh those strengths and priorities of ours with the capabilities of AI that actually are showing real promise. This is the most likely confluence to give existing enterprises a leg up as AI becomes more accessible and starts defining the winners and losers in different industries.