Emerj Technical Advisor, German Sanchis Trilles, PhD, defines natural language processing as:
“…everything which is related to human language. If you have a system that needs to recognize what a human wrote, that’s NLP. If you have a system that tries to understand what a human said with his voice or with her voice, that’s NLP as well. If you want a system to speak and to do some speech synthesis, that’s NLP as well.
If you want a system to understand the sentiment behind a tweet, that’s NLP as well. If you want to classify an email between spam and no spam, that’s also NLP.
So everything which is related to having a machine understand something or do something with human language that would be the ‘definition’ of NLP.”
While not all industries and sectors have obvious and plentiful use-cases for computer vision, all businesses work with text, and have to manage documents and communications.
Established sectors like banking, insurance, energy, and healthcare (collectively accounting for a large portion of global GDP) are buried in legacy systems and text documents. In some respects, NLP is a critical technology for unlocking the value in established and older firms, and enabling future AI innovation.
In addition, it is often much easier for a firm to adopt NLP than it is for a firm to adopt computer vision or voice technologies, as NLP often works with structured data (text documents or chat logs for example).
While a machine must scan and make sense of every image as a unique assessment, the letter “L” is always the letter “L”, providing less ambiguity and often less upfront training time to deliver some degree of business value. NLP also typically requires vastly less computational resources than the processing of images and audio.
While any AI application may have potential value in a business context, NLP will be frequent, and an understanding of the use-cass, potential, and limitations of NLP is important for anyone considering AI adoption.
Related article: What Is Natural Language Processing? – A Definition for Business Leaders
Chatbots and Conversational Interfaces
Capabilities:
- Replying to relatively simple questions or commands. Siri and Alexa are good examples of this general ability – and chatbots in banking or eCommerce are also often limited to a set range of questions that are determined to be the most common and useful for users.
- Classifying specific messages as being of a certain type (for example: Refund request, product inquiry, etc), and sending a customer to a human customer service agent when the machine is unable to handle the issue itself. Timing this “routing” is often the difference between a great chatbot experience and a poor one.
Limitations:
- Inability to understand conversation across time. Alexa doesn’t recall your previous sentence, and can’t build on the context of your requests over time (for example, to infer what you mean).
- Inability to actually hold a conversation over time – to explore the nuances of something that is said, or to ask deep questions to clarify intent.
- Requires a large corpus of previous conversations to train on.
- Requires a lot of context from data scientists and subject-matter experts in order to determine (a) what use-cases the chatbot should be used on to deliver value for users, and (b) which applications are feasible given the company’s data, talent, and financial resources for the project.
Related article: Chatbots for Customer Service – 4 Current Use-Cases
Enterprise Search and Discovery
Capabilities:
- Clustering and classifying documents by a certain type. For example, a machine that sees labeled examples of a few thousand “classified” and “unclassified” documents within a bank and then manually label other documents (in a corpus of potentially millions of documents) as either “classified” or “unclassified” based on labeled examples. This provides human users with a greater ability to sort and find relevant abstract information.
- Ability to search for more robust terms and concepts. A natural language model for search might be able to search medical records for specific drug side effects, looking for all the known research papers about “coughing” as a side-effect of a specific drug. This search application might automatically search for synonyms of “coughing”, such as “wheezing” or “hacking” or “tickle in the throat”, etc.
Limitations:
- AI-enabled search and discovery applications require a lot of upfront effort in determining the ontology of terms and labels that humans will use to search for information.
- Often, users are unsure of the exact workflows that will use the application. Users have their own ways to developing workflows when information is hard to find, and it is challenging to build an application that users will then use in their daily lives. This also makes measuring “before and after” ROI very challenging.
Related article: Information Extraction in Oil and Gas – Current Applications
Related article: Information Extraction in Banking and Financial Services
Related article: Text Mining in Banking – Currenct Applications
Related slide deck: The Digital Workplace Powered by Enterprise Search
Customer Feedback
Capabilities:
- Determine the sentiment of specific messages, or themes of messages. Answer questions such as:
- When people talk about my main competitor, do they usually do so in a way that reflects well on our company, or poorly?
- What is the general sentiment of social media posts about our brand in the last week?
- What are the angriest and irate email support tickets we receive, and what words or phrases do they have in common?
- Searching for entities (products, places, people, etc). A pharma company might search through all of it’s customer service call transcripts for the name of one of its products, in order to find the kinds of issues that were brought up with that product specifically, or any side-effects mentioned about the drug
Limitations:
- Except in relatively limited, structured examples, NLP is unable to find genuine insights from raw customer data without human guidance. A human can use NLP tools to navigate customer service data – or a human can have a machine monitor a specific metric (ie. the ratio of positive to negative sentiment expressed on social media), and have the program notify the business user whenever the ratios or numbers vary significantly from a baseline range. However, a machine is unable to interpret the “why” behind the questions being asked of the data, and human context and intelligence is necessary to determine the workflows and applications where this technology will have the most value.
Related interview: Using NLP for Customer Feedback
Related article: Discovering Customer Experience Trends with Natural Language Processing
Voice to Text
Capabilities:
- Transcribe speech to text. This text information can then be used to reply (Siri), purchase products (Alexa), or reply to basic questions (in a chatbot or conversational interface).
Limitations:
- There are a number of factors that make noisy voice data challenging to work with. Local accents can be challenging to understand for a machine (unless it has previous exposure to that particular accent) – microphone quality, air movement, and background noise can also impact sound significantly. Applications need to be trained on speech data in a variety of environments, accents, volumes, etc – in order to ensure that the application will be robust.
Related interview: What Industries Will Adopt Voice-Enabled Search First?
Related article: Voice and Speech Recognition in Banking
Related article: AI for Voice Transcription – Comparing Upcoming Startups and Established Players
Images to Text
Capabilities:
- Converting images or PDFs to structured text. With the right training, it is possible to take paper documents with unpredictable formats, and structure their data into an accessible and use-to-use format. For example, computer vision could be applied to invoices from different companies, which use different fonts, different page layouts, etc. The vision system could determine which text is the “Invoicing Company Name”, which text is the “Amount Due”, which text is the “Due Date”, etc – even if the system has never seen an invoice with that particular page layout in the past. Once converted to structured text, NLP can be used to search ann explore the contents of what once was just an image (for example, of a paper document sent by mail).
Limitations:
- Computer vision systems typically require a lot of manual training time in order to achieve accuracy. Careful pre-labeling of various documents is often laborious and monotonous. Some vendor companies offer “pre-trained” solutions, but even these solutions require careful calibration to the unique needs and workflows of the business adopting them.
Related article: AI for Document Digitization in Banking
Related article: Document Digitization in Healthcare
Related article: Information Extraction in Insurance
Best-of NLP Interviews from Emerj
Learn from some of Emerj’s best podcast interviews about NLP fundamentals. Before starting and kind of proof-of-concept with an NLP application – we recommend learning the fundamentals in these insight-dense interviews:
- NLP Systems Have a Lot to Learn from Humans – with Catherine Havasi of Luminoso
- NLP for Customer Service, How Does it Work? – with Narjes Boufaden of Keatext
- How Businesses are Making NLP Work – with Paul Barba of Lexalytics
- How Chatbots Work, and How They Evolve – with Yoram Bachrach of Digital Genius
- What Chatbots Can and Cannot Do – with Dr. Sid J. Reddy of Conversica
Header image credit: aimm.edu
