Business leaders in pharmaceuticals may be concerned with how machine vision will affect their operations in the clinical and scientific departments more than those of packaging and administration. However, prioritizing these types of operations may not provide the ROI as it’s hyped up to be. Instead, there are many possibilities for machine vision in pharmaceuticals related to packaging, shipping, and data entry.
The pharmaceutical industry and others alike have many companies with large stores of data from years of business operations, but companies may be less focused on innovation with data science than with other business areas. This results in fewer data scientists hired by these companies, as outlined in our interview with Abinash Tripathy, founder and CSO of Helpshift:
Basically what’ll happen is that specific industries like telecoms, like pharma, they have lots of data so they own the data. But they don’t have a culture of hiring scientists. Especially computer scientists and researchers to go and solve AI problems and they’re probably going to succeed by taking platforms like Googles and Amazon where the algorithms are getting created every day and applying them to their data.
Despite this, there are still numerous vendors competing to provide machine learning models best trained to suit each business problem or scenario. We have included an example of one of these vendors in each of our sections in this article.
In this report, we’ll detail the possibilities of machine vision software in the pharmaceutical industry. Because we have found that manufacturing, shipping, and data processing are the most common use cases for machine vision in this space, we will focus on the possibilities within those departments.
We’ll explore the following potential applications of machine vision software in the pharmaceutical industry:
- Quality Control for product and Packaging
- Shipment tracking and product Traceability
- Digitizing physical documents for database entry
Our exploration of machine vision possibilities in the pharmaceutical industry begins with quality control and packaging inspection for pharmaceutical products:
Quality Control for Product and Packaging
A prominent application for machine vision software in pharmaceuticals is in the quality control and packaging departments. The possibilities for these range from quantity and condition detection to the inspection of packaging and included items such as printed instructions and dose applicators. Companies can utilize machine learning on the production and packaging lines to ensure each unit meets a company’s quality requirements.
The business problems within pharmaceutical quality control and packaging inspection most often addressed by machine vision software are as follows:
- Counting the number of pills each bottle is filled with
- Inspecting each pill for accurate shape, size, and any damage
- Inspecting packaging for quality control or damage
- Quality control for secondary items such as instruction leaflets
- Label validation for product information and barcodes
Pharmaceutical companies can set a standardized fill amount for each container of their products, and train a machine learning model on the shapes and sizes of all pills currently produced by the company. A machine vision software solution would be able to detect how much of one product is being dispensed into the container at a time, as well as if any of the pills are damaged or otherwise misshapen due to a production defect.
Quality control can also be extended to a product’s packaging when used to detect the presence of child-proof caps, plastic wrappers over bottle caps, the correct box, or any other packaging element. A camera at the end of the production line could feed a machine learning model footage of each unit being packaged, and it would scan the video for each item being attached properly.
Printed barcodes and labels for pharmaceutical products also need to adhere to strict quality guidelines. Machine vision software can help companies make sure their barcodes are visible by industry standard scanners as well as verify all product information for accuracy and visibility. For printed text, solutions will typically require optical character recognition (OCR) technology.
Depending on how specific the label validation system is, the machine learning model for OCR may also require a natural language processing (NLP) component. This is because if a system is made to correct text mistakes as well as detect them, it will need to have more substantive training on the language than simply detecting which characters are used.
A machine vision software for any of these applications may be programmed with the ability to temporarily stop the production line in the case of any mistakes or deviations from the standards set by the company. Microscan Systems is a machine vision hardware and software provider that offers solutions with those capabilities.
Microscan Systems offers solutions to manufacturers across industries and pharmaceutical companies are among their most frequent clientele. Their cameras and software can be integrated into many existing manufacturing and production lines, as well as individual pieces of machinery. This could involve mounting a camera somewhere along the production line and connecting it to a machine vision algorithm within the system.
Alternatively, the software could be incorporated into the production machinery itself, allowing for “self-checking” devices. For example, Microscan Systems offers their LVS-7510 print quality inspection system, which is used to detect errors in printed labels for manufactured products. The software can signal the machine to change a light affixed to the top from green to red, indicating that it has stopped printing labels so a human employee may check for any errors.
Below is a video from Microscan Systems which explains how the LVS-7510 print quality inspection system works in more depth:
According to a case study from Microscan Systems’ website, the company worked with ISW GmbH to create a system for inspecting ampoules for pharmaceutical products. Ampoules are disposable plastic or glass vials typically manufactured in strips of five, with caps that are connected to each other. The software was used to monitor the process of producing, filling, and sealing the ampoules.
The software purportedly determined the dimensions of each ampoule and fill level of the solution inside as well as inspected their necks and openings.
John Agapakis is the Director of Advanced Sensing at Omron Automation, which is the parent company of Microscan Systems. He holds a PhD in Vision Guided Robotics from MIT. Previously, Agapakis served as Director of Sales in the Americas at Microscan Systems.
Shipment Tracking and Product Traceability
In pharmaceuticals, it is imperative that drugs and all other products be trackable on their way to stores and distribution centers. The products must also be traceable from the patient’s possession back to the manufacturer in case of any defects or unforeseen side effects.
Pharmaceutical companies are responsible for providing tracking and manufacturer information with all of their products, and machine vision solutions can help detect that information and deliver it across channels during many steps of the shipping process.
Companies can use machine vision software to ensure their tracking information is both accurate and instrumented across the channels that would need to access it. This could be in the case of an emergency or a simple customer service inquiry. The possible departments that could be instrumented with machine vision technology for this purpose are as follows:
- Factories and each of the numerous production lines inside
- Distribution Centers
- Warehouses
Applications for the technology for each of these departments also follow below:
- Shipping label inspection. Similar to our previous example of Microscan Systems’ print quality inspection system.
- Barcode and operational code detection and scanning
- Serialization Codes for further traceability
Companies could use a system such as Microscan’s technology for scanning printed labels in order to ensure the accuracy of every code or date printed on a product. This could also be used to detect any errors in shipping labels which could prevent mailing errors as well.
A machine learning model could also be trained to detect information from barcodes for retail inventory and serialization codes to prevent counterfeiting. Serialization codes could be related to the date of manufacturing as well as the factory or production line where the product was manufactured. This would allow pharmaceutical companies to determine where and when a given product unit was made if they were ever asked.
Cognex Corporation is an industrial company that offers machine vision hardware and software solutions to companies across industries with goals in manufacturing. They work with medical and pharmaceutical clients such as Boston Scientific to ensure their manufacturing practices meet industry requirements and prevent defective products from making it to market.
Regarding his company’s experience with using Cognex’s solutions, Research and Development Manager of Boston Scientific, Chris Pelerine, said, “By implementing 2D marking and In-Sight reading technologies, we now have a foolproof and efficient way to ensure that our products are labeled properly.”
The company offers a case study in which they claim they helped global pharmaceutical company Sanofi-Aventis track their products in real time using their machine vision technology. Sanofi-Aventis utilized the Cognex In-Sight micro vision system to read codes from all over the surface of the products and their packaging, as well as their Dataman handheld code readers. The handheld units were used by human employees when handling packages manually, while the In-Sight system was used for processing information from multiple packages more quickly.
The company had created multiple identification modules for products at different levels of packaging. These modules produced codes for serialization, bundling for multiple similar units, case packing for the amount of units that could fit in one case, and palletizing for grouping multiple cases that can all fit on one pallet.
According to the case study, the Sanofi-Aventis was able to maximize identification automation within their manufacturing and shipping system. This allowed them to ensure the traceability of all their products from there on.
Below is a video from Cognex which explains the differences between the models of their Dataman handheld barcode reading device. The video describes how each model is powered by a separate algorithm and the purposes of each:
Joerg Vandenhirtz is the Business Development Manager of Life Science OEM in Europe for Cognex. He holds a PhD in Elementary Particle Physics from NASA, Kennedy Space Center. Previously, Vandenhirtz served as founder and CMO of LemnaTec.
Digitizing Physical Documents for Database Entry
Another possibility for machine vision in pharmaceuticals is the automation of white-collar business processes such as document digitization and data entry. A machine learning model for OCR trained on clinical and pharmaceutical forms would be able to discern and extract data points from those forms and deliver them to a database for a client company. This is especially useful for companies with a large amount of physical forms containing data that they want to use for future projects.
Machine vision could help digitize the following documents and more:
- Clinical Trial Documents
- Patient reports containing clinical data or medical records
- Lab Notebooks
Clinical trial documents from past trials may be a valuable resource to companies looking to compare all of their clinical trials to decide on best practices. For example, a pharmaceutical company looking to gather data for their business intelligence analytics solution may want to digitize documents from clinical trials that are not saved digitally yet.
Patient reports may contain useful information about a patient’s response to a drug and the severity of any side effects. This is especially important if a new side effect is discovered or if the patient has an allergy to the drug. The pharmaceutical company and their clients each benefit from digitizing and centralizing this data because the company can factor in every past patient reaction and use that awareness to drive patient health and satisfaction.
Lab notebooks contain recorded procedures and results from a pharmacologist’s experiments across weeks of work. These notes would be digitized as new results are found and incorporated into all data associated with the corresponding experiments. For example, if an experiment was filmed and the video was saved as experiment data, the OCR data from the scanned lab notebook would be categorized under the same experiment.
One company that has purportedly found success in offering machine vision software for pharmaceutical document digitization is ABBYY. Their FlexiCapture solution is made for OCR and data capture from detailed forms with multiple distinct fields. They claim they helped Lambda Therapeutic digitize a high monthly volume of Anonymised Single Patient Reports (ASPRs) for pharmaceutical clients.
Lambda Therapeutic’s clients used the newly formatted data to drive their pharmacovigilance campaign, or their motivation to find new side effects and alternative uses for drugs. The case study states that a single employee typically processed eight ASPRs per day before they adopted ABBYY’s FlexiCapture system. The system was able to process 160 ASPRs per day once it was fully installed.
Although the case study also states that this represents an increase in productivity by twenty-fold, it is unclear how many workers this system replaced.
Andrey Isaev is the Chief Product Officer at ABBYY. He holds an MS in Management and Applied Mathematics from Moscow Institute of Physics and Technology. Prior to his position at ABBYY, Isaev served as Chief of Mobile Software Development Division at Paragon Software.
Header Image Credit: In-Pharma Technologist
