Chatbots have in large part dominated the AI conversation in the enterprise. This has ignited interest in the technology that makes this possible: natural language processing, or NLP. But NLP is not limited to chatbots and covers a broad range of AI capabilities that can be used in several applications.
In this article, we give an overview of natural language processing for business leaders, avoiding the technical where possible in favor of real-world examples and use-cases in industry. We also highlight three AI vendor case studies that reveal NLP’s use in banking, lending, and marketing.
What is Natural Language Processing?
Natural language processing is everything which is related to human language. If you have a system that needs to recognize what a human wrote, that’s NLP. If you have a system that tries to understand what a human said with his voice or with her voice, that’s NLP as well. If you want a system to speak and to do some speech synthesis, that’s NLP as well.
If you want a system to understand the sentiment behind a tweet, that’s NLP as well. If you want to classify an email between spam and no spam, that’s also NLP.
So everything which is related to having a machine understand something or do something with human language that would be the “definition” of NLP. – German Sanchis Trilles, PhD
Natural language processing is a broad term that encompasses many different techniques that allow computers to understand human speech and text. In the past, some of these techniques involved rules-based statistical systems and later, machine learning.
NLP can be used to analyze a vast array of different types of speech and text data from different contexts. For example, it could be used to analyze or transcribe audio recordings of incoming customer service calls or help extract relevant clauses from a legal contract.
The variety in the types and context of the data being input to the NLP algorithm and the different objectives for building the NLP system are what make different techniques necessary.
NLP involves technical techniques such as tokenization, parsing, tagging parts-of-speech, identifying sentiments or semantic relationships in sentences, and so on.
NLP systems break sentences down into smaller segments, identify what type of word or phrase each segment might be and what the relationship between each of these segments is, that creates the overall meaning.
Some of the capabilities that NLP-based systems might grant businesses include:
- NLP (Categorization): NLP can be used to categorize text, create summaries of documents, allow indexing and searching or look for duplicate content in vast volumes of text or speech data.
- NLP (Topic Identification): NLP can potentially identify the meaning and larger themes within vast volumes of text data.
- NLP (Extraction and Abstraction): NLP can also be used to pull out the relevant paragraphs from a long text document or even create a summarized version of the document that includes only the most relevant paragraphs.
- NLP (Information Retrieval): NLP can be used or searching for specific information within digital text.
- NLP (Intent Parsing): NLP can discern the context within human language. Chatbots necessarily involve NLP for intent parsing because they deal with recognizing the intent within text data, as well as responding to customers with text. In essence, the NLP software needs to “learn” the appropriate text responses to text it receives.
- NLP (Sentiment Analysis): NLP can determine the tone and inferred opinions (positive or negative) behind human language in text form.
- NLP (Speech/Voice Recognition): NLP can transform natural language human speech into written text, and vice versa.
- NLP (Machine Translation): NLP can be used to automatically translate text or speech data from one language to another.
The overall goal across all of these applications is to take raw human speech or text data and use AI and machine learning to extract insights or add value to that data in a way that makes it more valuable.
How Natural Language Processing Works
In order to train an NLP algorithm to understand human speech and text, the algorithm needs to be input with hundreds of thousands of labeled audio or text samples.
For example, an eCommerce company may want to build a chatbot that can process refunds for customers. The company would need to train the chatbot on hundreds of thousands of customer support inquiries involving refund requests. It would then label these emails as “refund request.”
The labeled text data would then be run through the software’s natural language processing algorithm, and this would train the algorithm to figure out the aspects of the “refund request” emails that, essentially, make it a refund request.
Perhaps it’s obvious: the word “refund,” or perhaps it’s a certain string of words or punctuation. Not every “refund request” is going to include the word “refund;” customers might ask for their “money back,” or they might ask about an unknown charge on their account, that, after enough back-and-forth with an agent, almost always ends in a refund.
If the algorithm is trained on a variety of different “refund request” emails, it will be able to determine if incoming messages are in fact refund requests even when the customer’s message is less straightforward.
A customer could then message the chatbot, and the NLP algorithm would be able to send the customer a pre-written template.
The business would also likely set a confidence interval for the chatbot algorithm so that if the algorithm is only, say, 80% sure that the message it receives is a refund request, it can instead route the customer’s message to a customer support agent.
Intent Parsing (Conversational Interfaces)
Vendor Spotlight: Nuance
AI vendor Nuance offers virtual assistants that the company claims use NLP to help clients improve their customer service operations. Nuance offers, Nina, a virtual assistant they claim can assist customers with business services.
At a bank, for example, Nina could provide a customer their account balance. When the assistant is unable to assist a customer or when the issue is more complex, Nina would redirect the customer to a human customer service agent.
The video below shows how Nina works:
Nuance claims to have integrated Nina with Swedbank’s customer service systems. According to the case study from Nuance, the company claims Nina helped Swedbank ensure that a majority of their incoming customer queries could be handled via digital channels such as chatbots.
The company claims this gave the bank’s call center agents more time to focus on selling and recommending more products to customers.
Nina was integrated into the Swedbank homepage, and users can directly type questions in a text box and receive answers for basic transactional questions.
For instance, If a customer said that they needed a new ATM card because they lost their old one, Nina either redirects the customer to a relevant web page or asks follow-up questions, such as whether the customer wants to block their old card.
Nina uses NLP to understand customer questions and retrieve relevant answers from a database or redirect customers to the correct webpages or human agents.
According to the case study, before the integration, 58% of Swedbank’s customer base was banking digitally, such as through Swedbank’s Mobile Bank.
The bank’s 700 contact center agents were handling 3.6 million customer interactions per year, which included more than two million basic transactional queries, 500,000 emails, and 10,000 social media interactions.
The case study claims that Nina handled over 30,000 conversations per month with a 78% “first-contact resolution” within its first three months of deployment.
About 55% of these conversations did not require the customers to take further actions, such as calling the contact centers. According to the case study, Nina can now handle 350 customer questions and answers.
According to Nuance, Coca Cola also deployed Nina for their My Coke Rewards and Ask Coca-Cola pages. Nina purportedly had 15,000 and 30,000 conversations a month on these pages respectively. Nuance also claims that Coca-Cola saw a 40% reduction in voice calls after integrating Nina.
Information Retrieval
Vendor Spotlight: LenddoEFL
LenddoEFL is a Singapore-based company that offers a product called The LenddoScore, which they claim can help banks and financial institutions assess an individual’s creditworthiness using NLP.
The LenddoScore allows financial firms to offer loans to individual customers or small and medium businesses in developing countries where credit histories might not be readily available for all customers.
The company claims that they use natural language processing to analyze and extract useful information that goes into generating the LenddoScore from the customer’s digital footprint. Customers need to download the Lenddo NFL app on their smartphones in order to avail any loans.
Once the app has been installed, users can grant permissions to allow the software to comb through their digital footprint including social media information, browsing histories, shopping histories, locational data and so on.
The company claims they have developed a model that uses NLP to extract all of this information and identify how they relate to creditworthiness leading to the Lenddo Score.
The app then shares this credit score with credit unions and other lenders that have tied up with Lenddo FL to potentially expand customer bases while simultaneously reducing the associated risk with new customers who have little to no credit history.
Below is a short 3-minute video demonstrating how LenddoEFL works:
LenddoEFL claims they worked with FICO to develop the FICO Score. FICO is using the score in countries like India to help offer loans to customers who have no credit histories.
Sentiment Analysis
Vendor Spotlight: IBM
IBM a product called Tone Analyzer, which it claims can help businesses in marketing operations using NLP-based sentiment analysis. The Tone Analyzer is a software that can be integrated into business systems in such a way that the software can analyze customer feedback data.
For instance, the software can be connected to a business’ social media page where customers post their opinions or to a customer service survey page.
IBM claims its product uses Watson’s sentiment analysis capability to learn which customer feedback is likely to be positive, negative or neutral. The software was trained on labeled datasets of existing customer feedback and social media posts to categorize customer experiences for various brands.
The company claims Tone Analyzer uses NLP-based sentiment analysis to identify what tags should be associated with customer statements.
The software can also be used to ask other questions of the data, such as identifying top customer issues or determining which products customers are talking about the most and whether these discussions are positive or negative overall.
The software could also list out the subject and tone within new customer reviews or social media posts.
IBM claims to have helped marketing firm Influential understand the social media presence of their digital influencers. According to the case study, Influential used Tone analyzer input with the social media posts of influencers about certain brands.
IBM claims the software was able to judge whether a digital influencer’s social media presence was aligned with the sensibilities of the brand they were advertising for bases on 47 different characteristics of influencers’ personalities from the social media posts.
The Future: Natural Language Understanding
We believe that in the next five years, the direction in which NLP use-cases might evolve would be in natural language understanding (NLU). NLU is a term that describes NLP systems that have a far deeper “understanding” of the concepts it analyzes.
NLU goes beyond the structural understanding of language to interpret intent, resolve context and word ambiguity, and even generate well-formed human language on its own. This is a hard problem to solve, as this means developing NLP systems that function as well as humans, even in new situations.
Header image credit: Kim Komando Show
