Concerns over facial recognition software use
Your face is unique; it's your calling card. Now imagine if you were recognized in a crowd via a surveillance camera – linked perhaps to your driver's license from the state database, or another database – and identified as a possible criminal and hauled into a police line up of suspects.
While that may sound farfetched, it could be a reality through the use of facial recognition by law enforcement to catch suspects for all kinds of crimes, from shoplifting to car theft to fraud or terrorism. Facial recognition can provide a certain amount of deftness and simplicity to our lives through some commercial applications, such as airports that are now using facial recognition as a security tool, allowing us to get through TSA pre-check lines with ease and quick express. At Detroit Metropolitan Airport, Delta Airlines is using facial recognition at some gates to expedite boarding. Casinos, including Detroit's MGM and Motor City Casino, use facial recognition to keep track of cheaters entering their establishments – as well as to notify staff when a “whale” – or a big spender – is coming in who they should lavish drinks and attention upon. “Eyes” are on us in crowds such as sporting events and concerts. Shopping mall security staff can scan through surveillance camera feeds to see if noted shoplifters or other criminals are floating around in the throngs.
But facial recognition also captures images of you and I – for good and bad. And a question that has authorities and ethicists pondering is how accurate and reliable facial recognition is as a tool of law enforcement, and what invasion of privacy concerns there may be.
Facial recognition is technology that is capable of identifying or verifying a person from a digital image or from a frame of video that comes from a video source. Also described as biometric artificial intelligence, it is an application that is supposed to uniquely identify a person by analyzing patterns based on their facial textures and shapes. While there are multiple methods for facial recognition, they all come down to comparing a scan of your face to an image of faces within a database.
Facial recognition was initially a form of computer application that has now been expanded to wider applications on mobile platforms and other forms of technology, such as robotics. It is often used as an access control in security systems, and is compared to other biometrics such as fingerprints and eye iris recognition systems. It is considered more reliable than a fingerprint and nearly as accurate as iris recognition, and is less-invasive and contactless than fingerprinting. As the technology evolves, it is finding commercial uses, such as in identification and marketing, video surveillance, automatic indexing of images, human-computer interaction, video databases and tracking.
An example of a well-recognized commercial application that many of us utilize on a daily basis is our iPhones. The latest incarnations of Apple iPhones switched to facial recognition over a fingerprint to open in the last couple of years. Apple said the version of the technology, which they call Face ID, uses a suite of sensors to map our faces in 3-D. Infrared light illuminates the face, and a projector in the phone projects an array of infrared dots at it, while an infrared camera snaps an image of the dots, which the phone uses to authenticate against an already stored image of your face. Apple claims the feature is so secure, there is less than a one in a million chance that someone else could spoof you, which is much more reliable than a partial fingerprint on the former “home” button to unlock the phone.
Anil Jain, University Distinguished Professor in the Department of Computer Science and Engineering at Michigan State University, said he has been working on facial recognition for 25 years, and it has changed a great deal from its infancy.
“It's a fairly mature technology now, but it will drive us for many more decades,” Jain said.
Moji Solgi, director of AI and Machine Learning at Axon, a technology company, said that face recognition is a broad term that lumps together a collection of technologies. It includes face detection, face tracking, face re-identification, face matching and face attributes. Face detection finds faces and their locations, he noted, and “most commodity digital cameras, including mobile phones, run face detection to enhance image quality.”
Face tracking corresponds a face from one frame to the next, and allows for the face of one person to be matched between two frames. “This is useful for a police agency when they need to blur out an individual's face in a body-worn video (such as a body camera) that they want to release to the public,” Solgi explained.
Face re-identification is similar to face tracking, but frames are not necessarily consecutive in a video. Your face may be at the beginning of a video, and then not seen again until the end. But, Solgi said, face re-identification can recognize it's the same face without identifying your face by comparing it to a database of faces.
With face matching, a target face can be given along with a set of candidate faces and candidate matching will find which one of the candidate faces belongs to the target face. “This is where algorithms meet databases for face search and retention,” he explained. Some photo storage applications use face matching, such as Facebook, to tag a face that appears in various photos, and it's the application technology that allows smartphones to unlock the phone.
Face attributes can take it further. More data has been fed into the algorithms to allow face attributes to extract information such as gender, ethnicity, emotions, age, facial landmarks and other identifying recognition.
Jain said the primary difficulties, however, with facial recognition is in its acquisition of faces. The problem is often they're too perfect in their setting and exposition.
“It is usually in a constrained imaging environment,” he noted, “where the subject is cooperative. You have to pose in front of the camera, take off your sunglasses, your hat. You are indoors, in controlled neutral lighting, posed, six-inches in front of the camera. The poses eliminates expressions. That makes identification from facial recognition much easier.
“In these applications, for access and government identification, the subject follows directions, and the algorithms are much easier to create,” Jain explained.
He said the guidelines on how to take a mugshot are very similar. We've all seen it on TV crime shows – look this way, straight at the camera. It gives law enforcement a clear, head on photo of the suspect.
The bigger question is how, and if, law enforcement uses facial recognition technology to help them apprehend bad guys or identify suspects in line ups. Birmingham Police Chief Mark Clemence said, “We don't have any equipment to use it.”
Oakland County Sheriff Michael Bouchard said they do not use facial recognition technology – yet. “That's one of the technologies we're constantly monitoring,” he said, noting it currently has too many false positives, but that commercial applications are there, so he anticipates its use is on the horizon.
“The ability to know you've got a bad guy on your hands sooner rather than later would be great, because lots of times when they come in the back door, they lie to us. If there's highly accurate facial recognition just as they enter the system, it's invaluable.”
Wayne County Sheriff's Office Chief Robert Dunlap said they also are not using facial recognition technology “at this time.”
“I use it for my personal use on my (Microsoft) Surface Pro (tablet), and I think it's excellent,” Dunlap said. But for the department, “We have just not acquired the technology yet. It's fairly new.”
Ming Dong, professor and co-director of the Department of Computer Science in the College of Engineering at Wayne State University, is also working with Ford Motor Co. on facial recognition to help determine the gender, age and other characteristics of drivers and passengers in cars for autonomous vehicles, as well as the use of camera-mounts in cars, “capturing the facial image of a person, because it can provide greater safety precautions.”
He said his department at Wayne State has someone collaborating with Detroit Police Department, “and how the cameras mounted on a police officer can recognize a person and certain events. That potentially can detect valuable information very quickly in the database. The camera can recognize a person from a body-mounted or vehicle-mounted camera within seconds, in real time.”
Dong continued, “The technique is substantially similar to detecting facial recognition for any application. The next step, I see face recognition as a part of AI (artificial recognition) with potentially a lot of applications. It will transform our lives.”
Despite the proliferation of surveillance cameras throughout the central business district of downtown Birmingham, Clemence said, “We've never once used the surveillance cameras,” for facial recognition, “and we don't anticipate using them.” Part of the reason is the inability of getting clear, usable images from the cameras.
He noted that Delta Airlines is beginning to use photos for facial recognition to speed up the boarding process. “The Delta photos will be head-on, perfect photos,” Clemence noted. “But most surveillance photos are angled, or people are looking down, and they're not perfect.”
Jain noted that having machines verify through facial recognition images caught from surveillance cameras “is more challenging because people are passing by, but they're not posing,” even though there are more and more surveillance cameras all over.
In downtown Detroit, a network of 2,000 surveillance cameras was installed in late 2017, paid for by Detroit Conservancy 300, a non-profit organization which manages Campus Martius under a contract with the city of Detroit, providing a live feed to Detroit police and the security operations of several downtown businesses, including Dan Gilbert's Rock Ventures, DTE Energy, Blue Cross Blue Shield of Michigan and Ilitch Holdings. The cameras, like in Birmingham, are not capable of facial recognition.
That doesn't mean that Birmingham or other police agencies have never thought of accessing a facial recognition database. They just turn to outside help for their expertise.“Sometimes if you have a serious crime or good picture off of surveillance (at a business), we'll request assistance from Michigan State Police – they're the ones who have the expertise. Michigan State Police is on the cusp of all of that – although it's not like CSI on TV,” Clemence said.
Dunlap said of the plethora of surveillance cameras in Detroit, “Any type of camera that can capture an image can be useful – the question is privacy. Every citizen is not someone we need to acquire intelligence on. Airports are very useful places to do that. After 9/11, I thought that was advisable. It's an imply/comply consent – that way airlines have a record of who is on that plane. You're giving implied consent when you get on that plane. But it's a different story when you walk down the street, going from point A to point B. You might not have a choice of where you're going, and you don't have a choice of having your image captured.”
At last count, there were 24 surveillance cameras around downtown Birmingham, primarily to provide assistance to police for safety issues, to provide a feed to dispatch at the police station in order to send officers if a crime was seen on camera, or a citizen was seen in trouble. In contrast, Clemence said, in London, England, “every license plate coming in and out (of central London) is identified and tracked. As technology is improved, it's the wave of the future.”
London uses automatic number-plate recognition, a technology that uses optical character recognition on images to read vehicle registration plates to create vehicle location data. According to the Electronic Frontier Foundation, 200 U.S. cities use the technology, with a current database of 2.5 billion license plates scanned – 95 percent of which were not under suspicion of any wrongdoing. Similarly, facial recognition databases do not determine if someone is suspected of doing anything wrong – they just hold images. Some databases include mug shots of criminals or suspects – and others are of Michigan drivers' licenses.
“I do hope it gets to that, with the amount of surveillance cameras and the number of pictures out there from cell phones, home and store cameras, it could really help the number of retail frauds done by retail fraud groups,” Clemence said. “We have tons of photos now. But the problem is, we have absolutely no idea who these people are. There are tons of retail, fraud, identity theft, tons of credit card fraud at gas stations – but we don't have the technology to identify them – and the bad guys know it. They are organized and they move around from town to town. If we get to the point where we can use facial recognition successfully, it'll be a different game. When technology gets to that point, it will be great because we'll be able to catch way more bad guys.”
Angie Yankowski, section manager, digital analysis and identification section (DAIS), Biometrics and Identification Division, Michigan State Police, said that facial recognition is available for the state police to support all criminal investigations for which there is a law enforcement purpose.
“Facial recognition is used successfully to identify subjects without identification on a traffic stop and can assist detectives in developing a suspect in a criminal investigation when surveillance video or other suspect images are available,” Yankowski said. “Additionally, we have a project in which we are using facial recognition technology to detect potential fraud within our copy of the Michigan Department of State's driver's license database.”
The Michigan Secretary of State's office stores all of the information each of us has on our driver's license or state identification card, as well as the information that is given when a person applies for a license or ID card, said Fred Woodhams, former communications director for the Michigan Department of State. “As required by state law, the department allows law enforcement agencies access to the images.” Beyond that, Woodhams, who left his position with the incoming administration changeover, “I can't speak to what happens with the photos.”
In other words, they provide the database, but offer no interpretive expertise.
The Pew Research Center countered that, noting that “fraudsters or drivers with serious violations try to beat the system by getting multiple licenses using different names. The implications of the deceit are far-ranging: People use driver's licenses and state IDs to do everything from cashing checks to opening bank accounts to boarding domestic flights. States increasingly are foiling the crooks and scam artists by employing a high-tech tool – facial recognition software. The software uses algorithms of facial characteristics to compare driver's license or other ID photos with other DMV images on file.”
The Pew Center said that at least 39 states currently use facial recognition technology in some format or another, with excellent results.
“You have an opportunity, using this technology, to find people who are trying to skirt the system. It has really helped to identify fraudsters,” said Geoff Slagle, director of identity management for the American Association of Motor Vehicle Administrators.
The technology in the database works by taking a photo of anyone who has a photo for a driver's license or state ID – and the image is converted into a template created out of the individual's unique physical features, such as their cheekbones or the distance between their pupils. An algorithm compares the image with others in the database in a search for a match.
Sometimes, someone has gotten married or divorced and changed their name. But often, Slagle's agency said, it's “a deliberate attempt to violate the law,” whether because someone lost their license due to DUIs, and wants to obtain another one, or they want to get credit, buy a car or get a mortgage using a false identity.
Kevin Bowyer, Schubmehl-Prein Family Professor of Computer Science and Engineering, University of Notre Dame, whose specialty is computer recognition, as well as facial recognition and iris recognition, said a project in the news “was comparing driver's license photos from different states, where subjects would obtain multiple licenses to avoid tickets. It may be a truck driver who gets a license from several states, for example. There are a lot of reasons you would want to get licenses, from underage students to people who have lost their legal license. It would not necessarily be a security issue, but could be a safety or legal issue.”
“Still others are wanted felons or criminals, such as sex offenders, who are trying to hide their identity by using an alias,” said Jenni Bergal of the Pew Center. “Facial recognition has led to numerous arrests and administrative actions against drivers.”
“Facial recognition technology is helping to protect people not just from identity theft and fraud, but as drivers and neighbors,” said Betty Johnson, an administrator in Nebraska's Department of Motor Vehicles, which has used facial recognition technology since 2009. Since then, about 3,000 people have been prevented from getting a license and over 300 have been arrested for crimes they were sought for, and were identified via the technology.
Yankowski is careful to point out that, “per Michigan State Police policy, facial recognition is not considered to be a form of positive identification. It is considered to be an investigative lead only, requiring the investigator to continue the criminal investigation before making any final conclusion.”
She said that facial recognition search algorithms have advanced greatly over the last five years, “improving our ability to provide in