Introduction: There’s a lot of hype out there about conversational AI. Although according to our guest, we’re nowhere near the day when AI can generate accurate conversations for the average business to integrate into their customer service, chatbots still have practical applications. In this episode, we interview the head of research at Digital Genius, Yoram Bachrach. Yoram succinctly outlines the current applications of chatbots—what they can and can’t do—and details how business can best prepare to automate their customer service.
Expertise: Machine Learning, Deep Learning, Artificial Intelligence, Chatbots
Brief Recognition: Yoram Bachrach is the head of research at Digital Genius, a customer service augmentation/automation service and technology provider. Before Digital Genius, he worked in deep learning for Microsoft for eight years. He holds a PhD in computer science from The Hebrew University of Jerusalem, focusing his thesis on “computational and algorithmic aspects of cooperative game theory”.
Current Affiliations: Digital Genius Ltd.
Readers with an interest in exploring current applications of chatbots in business should read our full article called “7 Chatbot Use Cases That Actually Work“.
Big Ideas:
Yoram outlines five key parameters that one has to think about when putting together a chatbot system. Here’s a quick rundown of each of them:
- The volume of conversations: in order for the system to come up with suitable answers to customer questions, it needs to reference past conversations between human agents and customers. Although there’s no magic number with regards to how many prior conversations one should have cataloged before integrating a chatbot system, Yoram suggests that a couple of months worth of conversations as a starting point. That said, the more conversations the AI can reference, the more likely it is to finding a suitable answer for a customer’s question.
- How consistent are the agents currently? If your agents aren’t making use of templates and canned responses, how likely are they to respond similarly to the same customer questions? The more similar the human responses, the more likely the system will be able to define those responses as appropriate for a given customer question. If your agents are giving different or perhaps more nuanced responses to the same customer questions, the system is going to have more difficulty with finding answers to those questions.
- How hard is it to train agents? If it’s difficult to train human agents in handling customer questions, it’s going to be just as difficult for a machine to handle it. If agents need to search around for answers to customer questions because every question involves significant nuance, any integrated AI system is going to have difficulty coming up with suitable answers.
- What’s the complexity of questions incoming and outgoing? Similarly, if the questions customers tend to ask are extremely specific or somehow different depending on the customer, the machine is going to struggle with coming up with answers to those questions. The system might easily be able to handle a cancellation request if the majority of your customer support tickets are for cancellations, but it may have trouble with IT problems, for example.
- How long are the conversations? Right now, chatbots are more able to handle quicker, simpler conversations. If the customer asks for something specific, such as a cancellation, the system should be able to handle that request fairly easily. When conversations tend to drag on, they tend to involve more nuance and complexity, which would have the system struggling to come up with appropriate answers to questions.
Turning Insight Into Action:
If you’re looking to automate your customer service, think about the big ideas listed above. How complicated are the questions you receive from your customers? If they’re simple enough that you’ve already created templates or canned responses to answer most of them, then you’re on the right track if you’re looking to automate. Maybe your customer requests are simple, but you haven’t taken the time to create templates and canne responses for them. Start with that. After using these templates for a couple months, you’ll have a backlog of references from which an integrated chatbot system can pull.
Interview Highlights on How Chatbots Work
(3:25) Dan: What does it take to train a conversational agent system?
YB: Basically we’re integrated with CRM systems. We grab historical data, actual responses that actual agents said in conversations with customers. We take all these conversations and process them. Once we do that, we build a machine learning model capable of figuring out what’s the best response for a customer question. To be more precise, we look at the conversation prefix, everything that’s been mentioned so far, [we end] with the last customer utterance, and then we try to figure out, “what is the best agent response to this?”.
Currently, to make this work, we have to grab the original conversations and process them. The processing takes a couple of hours. For more complicated systems, at most a day. We’re talking about a matter of days to set up a full working system. Having said that, the current product (Digital Genius) does not handle all conversations. If the expectation is the system will be able to answer any incoming customer, ten that’s not the case at this point.
I think we’re quite far from that. What we can do is handle the most simple cases if someone is asking for something very routine that has been observed previously in the conversation. If most of the past incoming calls are asking for a cancellation and that’s what the new calls are about, chances are the machine will be able to answer it more easily.
(7:18) Dan: How far back do you generally have to go to take this information into account?
YB: In our cases, we go back a year or maybe even more. Generally, for us to build a good model, we need to have enough data. If we have a huge call volume, we might not need to go back too much in time. But generally we need to go back enough in time, let’s say a couple months or a year, for the model to start working well.
On the other side there is freshness. In some cases things change, and if you take into account conversations that happened before things changed you might make the wrong recommendation. There are a couple of ways to tackle this problem. What we’re doing is we’re trying to handle an answer bank.
These are commonly used answers and those can be kept fresh by the customer. This is something we can take into account. We can also say “this answer is wrong.” As we’re training the system, we’re learning on the fly. We would get that information from the agent supervising the machine, basically.
(9:27) Dan: Humans would provide the core up to date general templates and the machine would have to take those into account, rather than inventing them on what it saw in the historical two years. It sounds like a combination of the two. How does that work?
YB: You’re asking basically how organized the customer is, and it varies. Some customers come very prepared, and what happens is they already have a good bank of templates [for the machine]. Some CRMS offer templates or a canned response. If they make good use of that, then perfect, we already have a good template candidate to plug into the system.
Other places are less organized, and so we’ll see an agent opening a notepad on the side, typing in stuff they say routinely and doing copy and paste.
The reality is, for the current offering, the best solution is that you have a customer that comes ready and says, “I understand I want to automate things, here are answers my agents routinely use.” Any historical utterance by a customer can be a candidate [to plug into the system], but there’s huge variability to what agents say. That’s the harder, more challenging thing to tackle. The more organized you are, the better the system we build will work.
(13:52) Dan: Let’s say there are some some weird questions that have been asked that have been answered by agents on the fly. When another ticket of that sort comes through, is it possible that a machine would be able to draw in those non-banked answers and say “based on historical data, this is what one should say here.” Is that possible with today’s tech?
YB: The state of the art models will do that. But when it comes to actual customer service, we found models operate much better when it’s the more frequent cases, and if you apply answers from the answer bank. There are models that can generate their own responses, but your probability of issuing a correct answer drops quite drastically. There’s been tremendous progress in the field of deep learning, but still, even when you employ the best technology available, it’s still helpful when people come to us and say “this is a good answer for an agent to give.”
(18:20) Dan: From your projections into the future, who are the folks who are most likely to automate the highest percentage of their tickets in the next 5 years?
YB: I wish I could say, “here’s a simple answer.” If there’s a high variability of questions and answers, that’s going to be harder to automate. How long does it take to train a new human agent? If the training process takes a huge amount of time and even after that they still would need to do a lot of research for every new customer inquiry that comes in, that’s an indication this is hard. If it’s hard for a human to do, it’s going to be hard for a machine to do as well.
If you come better prepared as a customer [of Digital Genius], it’s going to help to train the AI. You mentioned volume: how many transcripts do you have? It depends on how varied the questions and answers are. Another parameter is the length of the conversation. If the conversation goes on, and it’s a hundred utterances, that’s going to be harder to tackle.
Even within the same industry, some customers are much easier than others because the domain is simpler. Certainly you need to have enough volume of conversations, especially if you have a complex product. People also ask us about languages. Some of our customers even offer more than one language. You need to have enough data in each language, and the technology can support it if you have enough data.
Head image credit: pexels.com
