According to the US Government Accountability Office, the Federal Bureau of Investigation’s database contains over 30 million mugshots of criminals and ID card images from 16 states. This is just one of many law enforcement databases which also contain further identity information, including fingerprints and text data.
With needs to improve investigation times and streamline the task of matching suspect images within a pool of numerous identities, government officials, law enforcement offices, and commercial vendors are researching how AI, specifically computer vision, can be used to improve facial recognition.
Through our research, we aim to show insights on how various law enforcement agencies and companies are implementing facial recognition technologies.
This research will be broken up into the following two categories:
- Government and Law Enforcement Applications: Law enforcement agencies and government organizations that have implemented facial recognition software.
- Commercial Vendor Applications: Vendors that claim to offer facial recognition solutions to law enforcement or security-related organizations.
Readers interested in AI for law enforcement might be interested in our founder’s presentation at a joint INTERPOL-UN conference on AI in law enforcement given in the summer of 2018.
Government and Law Enforcement Applications
New York’s Department of Motor Vehicles Software
Implemented in 2010 and upgraded in 2016, New York’s facial recognition system connects to the database of ID photos and identity information recorded by the state’s Department of Motor Vehicles offices.
Since implementing its newest upgrade in January 2016, New York Governor Andrew M. Cuomo claims the software has led to more than 4,000 fraud arrests involving criminals who attempted to steal the identities of New York residents listed in the DMV’s database.
According to the press release, when a license, passport, or ID photo is taken at a New York DMV office, it is automatically entered into the facial recognition software. If a photo entered into the system doesn’t match up to a single identity, or if it matches the mugshot of a differently-named criminal, DMV investigators are notified.
When the DMV staff member clicks on the possible fraud notification, they are able to expand to a page which shows an old and new ID comparison. The DMV can then choose to investigate further internally or to report the potential fraud to law enforcement.
The 2-minute video below from the New York State DMV demonstrates how DMV staff use the program to investigate possible cases of fraud:
It is not clear from demonstration videos or the press release if this notification happens live, while a person is still at the DMV office, or if the system takes longer to alert DMV staff.
Aside from ID photos taken at various New York DMVs, the system receives mugshots and other forms of identification from both federal and state law enforcement databases.
Since 2010, the system has notified DMV employees to 21,000 potential fraud cases, with 7,000 of those alerts coming after the 2016 upgrade. The press release does not note how many fraud arrests occurred before the system upgrade or if an outside vendor was involved in any aspect of building the system.
The FBI’s Next Generation Identification System
The FBI says their Facial Analysis, Comparison, and Evaluation Services Unit uses a database called the Next Generation Identification System (NGI).
The FBI states NGI could be used for the following:
- The software allows law officials to scan and log fingerprints, upload facial images and enter other biometric data.
- The user can also upload security photos or photos with multiple faces, as the software will identify and log faces detected
- The uploaded facial images, fingerprints, and other data can then be used to search the FBI’s database for facial or fingerprint matches. The database is made up of facial photos and fingerprints taken at law facilities and state motor vehicle departments, according to the FBI.
- The software can also provide FBI or law officials with a Rap Back, a notification from the program that is activated when someone already entered in their system, such as a person on parole, gets their prints or facial data entered in again after an arrest or booking.
We could not find any demonstration videos or descriptions of how any of the FBI’s technology works. However, it seems that an FBI official logs into the NGI software, uploads an image or video of a person of interested, and is then presented with facial images that match the individual photographed. It is not clear how officials would access or be notified of a Rap Back.
While it is unclear if the FBI built NGI internally, NEC claims to have helped provide NGI with Rap Back capabilities.
While the feature might be used to monitor individuals under investigation or on probation, the FBI says the feature can also be used to monitor government, security, or law enforcement employees and notify their organization or supervisor if they are arrested for a crime which may disqualify them from employment.
While the FBI did not note whether NGI technology was involved, they reported that they used unspecified facial recognition software to identify and gain leads when apprehending a fugitive who was on the run for over 14 years.
After getting tips that the fugitive was traveling internationally, an FBI agent contacted the US Diplomatic Security Service to gain access to a passport image database. He then uploaded a photo of the fugitive’s wanted posters to cross-reference him with the database. The software linked the fugitive’s face to a similar headshot on a passport with a different name listed.
When investigating where the passport was used, the FBI said they were able to track and arrest the fugitive who was living under a false identity in Nepal.
While we could find no case studies specific to FACE or NGI, the FBI’s website offers detailed fact sheets with figures on the software’s quarterly use.
Commercial Vendor Applications
Rekognition
Although one might think of recommendation engines when they think of Amazin, the Orlando Police Department and Oregon’s Washington County Sheriff’s Office are among those using Amazon’s Rekognition software.
According to a case study, Marinus Analytics claims to use Rekognition to find women kidnapped by sex traffickers. When looking into computer vision applications for facial recognition and face matching, Marinus found that the software they tested did not have the capability of properly analyzing and matching “grainy” security camera images.
Marinus determined through its pilots that Rekognition was the most accurate image analysis software for their purposes. They integrated Rekognition with their Traffic Jam software, which the company claims allows them to match the photos of missing people and criminal suspects to an online database of ID photos and mugshots.
The case study details how a detective in Northern California uploaded a family-provided photo of a missing teen to the Rekognition-integrated Traffic Jam software. The software then searched the web matching photos as Rekognition searched its database for the same. Rekognition then scored the matches on accuracy with a confidence score and Traffic Jam sent the detective images with the highest confidence scores. The software also sent the detective information related to the images, including the geographic location of the person who posted the images. With this data, the detective was able to find the suspect and the missing teenager alive.
The 3-minute tutorial below demonstrates how a user can upload an image, such as that of a dog, and Rekognition can find matches online that show a confidence rating of how likely other online photos are also dogs:
Rekognition requires that a data scientist or similar professional integrate the software.
Kashif Imran is a senior solutions architect on the Amazon Web Services team. He received his Master’s in computer science from the University of Florida.
The team also includes Rakesh Nambiar, a senior software development engineer who worked on Amazon Fire prior to this project. He received a Master of Science and Engineering in computer science and engineering from the University of Michigan.
NEC
NEC claims its facial recognition software can be used in two ways, one of which is called Watchlisting. With this feature, investigators, law enforcement officers, and security guards can connect NEC software to a security camera live feed or upload a video to the platform as it is playing. The company says its software can highlight and capture faces that pass in front of the camera in real time, adding them to a list of facial images.
NEC’s second feature allows users to run face images through the system by uploading them or selecting them from images collected in Watchlist-mode in order to see if the image matches with mugshots or ID images in various law enforcement databases.
The company is not clear on whether it offers its own database of face images; however, law enforcement officials can connect or upload databases that they already use to the software.
It is unclear how this connection or setup process takes place on the user’s end. That said, it does seem that NEC’s features can be integrated into control-room computer systems.
The 2-minute video below demonstrates features of NEC Facial Recognition software, including Watchlisting, which records and lists face images found in real-time security footage:
Below is another 4-minute demonstration, from the Calgary Police Service, an NEC client, demonstrates how NEC’s software matches an image with mugshots and other images in a criminal database:
According to a case study, the Irving, Texas Police Department looked to NEC’s NeoFace Reveal software.
Due to time and staff constraints, the study explains that Irving police officers were regularly forced to make decisions on whether to act on or not act on crime scenes with minimal evidence. For example, when called to the scene of a crime, the officer could get there and realize there is not enough current evidence to convict possible suspects. If security footage or cell phone videos were recorded, the officers had to then investigate images that could be poor quality and grainy, according to the study.
The increase in cell phone camera use lead to officers becoming more backlogged with investigating an influx of videos and images sent in by residents. Many of these images and videos still did not provide active suspect leads when officers investigated them manually. As a result, the police department was seeing higher costs and time allocated to investigations that were not leading to arrests.
Irving PD used NEC’s NeoFace Reveal to quickly pick out faces from the plethora of videos and images they received from residents and that were already logged in their database. According to the study, this software was managed by investigators at Irving PD’s Crime Information Center.
Irving PD also claims NeoFace Reveal ran videos and images through Irving PD and other local police department mugshot databases in order to find matches. When images were matched with those in one of the databases, investigators could then call in related police officers and discuss whether or not to further investigate a situation.
While NEC does not note how many matches its software made for Irving PD nor whether those matches lead to arrests, it did say that investigators had proceeded with cases based on facial matches found with the software. The case study also purports that investigators only needed one day of training before implementing the software.
NEC’s Chief Acceleration Officer, PG Madhavan, has held product leadership roles at Microsoft, Bell Labs, Rockwell Automation, and GE Aviation. He also received a PhD in Electrical and Computer Engineering from McMaster University, as well as a Masters in Biomedical Engineering from the Illinois Institute of Technology.
William Tan, NEC’s Director of Global Face Recognition, previously held the position of Regional Alliance Manager at IBM, where he managed global systems integrations for the Association of Southeast Asian Nations. The amount of AI work related to this position was unclear from his LinkedIn profile.
FaceFirst
Los Angeles-based FaceFirst claims to offer software that can match images uploaded into its platform by a law enforcement officer or other clients against images from a face recognition database of enrolled images and mugshots.
According to the company site, officers can use FaceFirst by uploading a database of images into the system. From their mobile phones, officers can add mugshots, ID images, or other facial images by uploading them or taking them on-scene with the application.
When an officer enrolls a suspect in the system, the officer uploading the image will be prompted to add information, such as a name or notes, that will be linked with the image, if that individual isn’t already in the database.
The 2-minute video below from FaceFirst notes potential problem points an officer may face without facial recognition and then demonstrates how they claim an officer could use the FaceFirst app to identify a suspect:
FaceFirst’s clients sign a non-disclosure agreement, and so the company does not list clients on their website, except for those mentioned in case studies. However, the company’s homepage states that it helps to provide face recognition to Fortune 500 retail companies and organizations affiliated with security and law enforcement.
In a case study, FaceFirst claims that the Automated Regional Justice Information System (ARJIS), based in San Diego County, uses FaceFirst. ARJIS Partner agencies including the San Diego Police Department, the FBI, the Drug Enforcement Agency, and US Marshals and others have been using the system since 2013.
In its first four years of working with the aforementioned law enforcement clients, FaceFirst claims its software led to a total of over 12,000 “police actions,” such as arrests or questionings.
There were no further case studies that went into FaceFirst’s work with specific law enforcement agencies.
The company’s VP of Engineering, Gary Brown, assisted in building Encase, a risk management and predictive analytics product while working as Director of Software Development at the endpoint security company Guidance Software. He also received his Master’s in computer science from Florida Institute of Technology.
The Current State of Facial Recognition for Law Enforcement
Facial recognition software is gaining traction in the law enforcement world. It may solve the problem of law enforcement officers having to sift through millions of mugshots and ID photos in order to detect fraud and match suspect images with identities.
Software such as that provided by Rekognition, NEC, and the FBI’s NGI may help law enforcement agencies cut time out of investigations, leading to quicker arrests and possibly more lives saved.
Commercial applications, such as Rekognition and NEC’s NeoFace Reveal, require a prior database of images into which the software can be integrated. For most of these applications, these databases could include public criminal records and photos from federal identification departments, such as the DMV.
Header image credit: teslarati
