The growing demand for highly accurate surveillance, intelligence, and forensics systems has propelled the unconstrained recognition problem (1:N matching; also known as "identification") to the forefront of computer vision research. Over the past decade, excellent progress has been made toward the constrained and unconstrained verification (1:1 matching) problems. For controlled face verification, 99.9% accuracy was achieved for the FRGC set. For uncontrolled face verification, 88.13% accuracy has been reported for the once very difficult LFW set. Verification is a fundamentally easier problem than recognition, as it only considers discrete pairs of samples for matching, with a claimed identity choosing a comparison class that is known to the matching system. Recognition is made more difficult by the need to identify an unknown class out of the set of known classes. In open set recognition, it is possible that the unknown class does not correspond to any known class, and thus should not match any class. Compounding these structural considerations is the overall environment of the unconstrained scenario, where any number of effects (pose, illumination, expression, sensor noise, weather, etc.) can impact accuracy. Recognition, in general, is a challenging problem with important consequences for security and forensics applications.

In this work, we have taken a systems level approach to unconstrained face recognition, exploring all facets of the problem from image acquisition to classification. Beginning with a comprehensive study of optics and sensors, we have isolated the key problems that inhibit good recognition with today’s technology. In order to compensate for such problems, new algorithms for image restoration, feature detection and classification have allowed us to achieve reasonable levels of recognition accuracy where many other approaches have failed. We have also placed considerable effort into developing novel approaches for algorithm evaluation, including a controlled semi-synthetic data acquisition platform that injects real environmental effects into standard data sets.

This work was supported by ONR SBIR Award No. N00014-11-C-0243, ONR STTR Award No. N000-14-07-M-0421, ONR MURI Award No. N00014-08-1-0638, Army SBIR Award No. W15P7T-12-C-A210, and NSF PFI Award No. 650251