SEVERE WX : Severe Thunderstorm Watch View Alerts

Face masks are screwing up facial recognition software

As the Covid-19 pandemic continues, companies are plotting what a safe return to the office looks like. To enforce social distancing and mask wearing, AI technology that tracks employees' locations is becoming increasingly popular. But privacy experts warn that may not be the best solution.

Posted: Jul 28, 2020 8:00 PM
Updated: Jul 28, 2020 8:00 PM

Face masks are already known to stop the spread of coronavirus. Apparently, they can also make it much harder for facial-recognition software to identify you, too.

This is the key finding of a new report released Monday from federal researchers at the National Institute of Standards and Technology, or NIST, which is a branch of the US Commerce Department whose functions include measuring the accuracy of facial-recognition algorithms that companies and researchers submit to the lab.

When tasked with matching a picture of a person wearing a digitally added face mask to a different photo of the person without one, the most accurate facial-recognition algorithms failed to make a correct match between 5% and 50% of the time, according to the report. Generally speaking, most of the algorithms tested had failure rates of between 20% and 50%, Mei Ngan, a computer scientist at NIST and an author of the report, told CNN Business.

The identification issues make sense, as facial-recognition systems typically work by comparing measurements between different facial features in one image to those in another. Blocking off part of the face means there is less information for the software to use to make a match.

It highlights a unique challenge the tech industry is already working to confront as the pandemic continues. While the technology is controversial, with a number of companies recently rethinking providing this technology to law enforcement, it's used in a range of products and services, from using your face to unlock your smartphone to passing through a security checkpoint.

For their report, the researchers created nine different black and light blue mask shapes to account for the ways mask shapes vary in the real world and used them to hide part of a person's face in a photo. They then compared a digitally masked photo of each person with another, unmasked photo of the same person. They also conducted a test of the algorithms on both sets of photos with no virtual masks.

In all, they tested 89 algorithms on more than 6 million photos featuring a million different people. The photos came from two sources: applications for US immigration benefits, which were used as the unmasked images, and photos of travelers crossing a border to enter the US, which were given a digital mask.

NIST found that the best of these algorithms -— which were submitted to the lab before mid-March — failed just 0.3% of the time when tested on these same sets of photos without the digital masks. With the digital masks on, however, error rates climbed to 5% among these same algorithms.

An obvious shortcoming of the report is that NIST didn't test the algorithms on images of people who were actually wearing masks — Ngan said the digital approximations of masks were used due to time and resource constraints. On the plus side, it allowed researchers to quickly get a sense of the effect of masks on the algorithms, but real masks fit differently on different people. It's still unknown how texture or patterns might affect accuracy of facial recognition software.

"That's something we want to look at," Ngan said.

Some companies have said their facial-recognition technology can work with masks, and Ngan said she's heard from developers who are working on algorithms meant to make this possible — perhaps by training an algorithm with lots of images of people wearing actual masks, or by targeting the region of the face above the middle of the nose.

That's a focus for Marios Savvides, a professor at Carnegie Mellon University who studies biometric identification. He said a person who's wearing a mask can be essentially invisible to a facial-recognition system, as it doesn't even detect a face in the first place. He thinks the region of the face that includes the eyes and eyebrows tends to change the least over time, which makes it a good part of the face to use when trying to identify a person whose mouth and nose are hidden.

The NIST report is the first of several that the lab plans to issue about how facial-recognition algorithms identify masked faces. In the fall, Ngan said, NIST expects to release a report on the accuracy of algorithms that were specifically created with the intention of spotting people in masks.

West Lafayette
Partly Cloudy
90° wxIcon
Hi: 90° Lo: 72°
Feels Like: 98°
Kokomo
Partly Cloudy
85° wxIcon
Hi: 89° Lo: 73°
Feels Like: 91°
Rensselaer
Mostly Cloudy
82° wxIcon
Hi: 87° Lo: 68°
Feels Like: 86°
Fowler
Partly Cloudy
90° wxIcon
Hi: 89° Lo: 69°
Feels Like: 98°
Williamsport
Partly Cloudy
84° wxIcon
Hi: 88° Lo: 71°
Feels Like: 91°
Crawfordsville
Partly Cloudy
84° wxIcon
Hi: 87° Lo: 71°
Feels Like: 90°
Frankfort
Partly Cloudy
84° wxIcon
Hi: 87° Lo: 72°
Feels Like: 90°
Delphi
Partly Cloudy
83° wxIcon
Hi: 90° Lo: 72°
Feels Like: 88°
Monticello
Partly Cloudy
83° wxIcon
Hi: 92° Lo: 73°
Feels Like: 88°
Logansport
Partly Cloudy
82° wxIcon
Hi: 88° Lo: 70°
Feels Like: 85°
Severe weather risk, then brief, but substantial cool-down...
WLFI Temps
WLFI Planner

Indiana Coronavirus Cases

Data is updated nightly.

Cases: 751242

Reported Deaths: 13795
CountyCasesDeaths
Marion1032931788
Lake556911009
Allen41692692
St. Joseph36990565
Hamilton36588417
Elkhart29398461
Tippecanoe22901226
Vanderburgh22556400
Porter19356325
Johnson18471389
Hendricks17682317
Clark13226195
Madison13149344
Vigo12614253
LaPorte12419221
Monroe12207176
Delaware10966198
Howard10321225
Kosciusko9630121
Hancock8576146
Bartholomew8169157
Warrick7860156
Floyd7811180
Grant7242179
Wayne7162201
Boone6966103
Morgan6761141
Dubois6218118
Marshall6209116
Cass6016110
Henry5900110
Dearborn589878
Noble581488
Jackson509076
Shelby501496
Lawrence4742122
Gibson444894
Clinton442355
Harrison441875
DeKalb439885
Montgomery438090
Whitley406543
Huntington402681
Steuben400159
Miami395269
Jasper388054
Knox375991
Putnam372960
Wabash361983
Ripley347170
Adams345555
Jefferson335886
White331953
Daviess3033100
Wells295281
Decatur289992
Greene286885
Fayette284864
Posey273835
LaGrange273072
Scott270156
Clay267148
Washington246036
Randolph244783
Jennings235349
Spencer234531
Starke228058
Fountain220948
Sullivan214643
Owen211858
Fulton202942
Jay200932
Carroll193620
Orange188255
Perry187237
Rush175926
Vermillion174844
Franklin170335
Tipton166246
Parke149416
Pike138234
Blackford136232
Pulaski120647
Newton113936
Brown104243
Crawford102516
Benton101714
Martin91715
Warren84015
Switzerland8148
Union72810
Ohio57911
Unassigned0420

COVID-19 Important links and resources

As the spread of COVID-19, or as it's more commonly known as the coronavirus continues, this page will serve as your one-stop for the resources you need to stay informed and to keep you and your family safe. CLICK HERE

Closings related to the prevention of the COVID-19 can be found on our Closings page.

Community Events