The research and development unit within the Department Of Homeland Security, Science and Technology Directorate organized a tech rally consisting of tech experts and volunteers to find a Facial Recognition Algorithm that can identify people with masks with the highest possible accuracy.
The event went on for 10 Days. 582 volunteers were tested by the 60 different facial recognition systems. The main acquisition systems were Stone, Vly, West, Dans, Besek, and, Pine which were a mix-and-match of six different image collection systems, paired with ten different matching algorithms which were evaluated according to their face capturing ability and reliability of identifying a face with high accuracy.
The different AI Face Recognition systems gave different results. The average accuracy of recognizing unmasked individuals was 93% and the same for masked individuals was 77% on average. The best performing technology was Dans which was able to identify masked people 96% of the time and the worst technology did it for 4% of the time only. The results were based on Efficiency, Satisfaction, FaceFtAR (Failure To Acquire Rate), and FacemTIR (True Identification Rate)
Efficiency is defined here as the average time required for test volunteers to perform a transaction with each system. In terms of Efficiency, West and Dans were the best as they took only 4 seconds of transaction time. Satisfaction is defined here as the proportion of test volunteers who provided positive responses. West, Dans, and Besek gave the most satisfactory results with a number of 95%.
Failure to Acquire Rate (FtAR) is defined here as the proportion of transactions that failed to capture an image, or that failed to capture an image that could be templated by the MdTF matching algorithm. The MdTF True Identification Rate (mTIR) is defined here as the proportion of transactions where the submitted biometric image matched the correct test volunteer or was unidentified if the probe subject was not in the gallery. In both the results, Vly was at the top. Its FtAR was 1.7% within 20 Seconds and mTIR was 97.8% within 20 seconds.
The full report of the event is not confirmed announced yet. You can check the full report at MDTF which is expected in the coming weeks.
