Human and Machine Perception of Filtered and Synthetic Faces

Keywords: augmented reality, virtual reality, facial recognition, facial analysis, visual perception
Spring 2020 - Present
teaser image

Description

Advances in face synthesis have raised alarms about the deceptive use of synthetic faces. Can synthetic identities be effectively used to fool human observers? In this work, we introduce a study of the human perception of synthetic faces generated using different strategies including a state-of-the-art deep learning-based GAN model. This is the first rigorous study of the effectiveness of synthetic face generation techniques grounded in experimental techniques from psychology. We answer important questions such as how often do GAN-based and more traditional image processing-based techniques confuse human observers, and are there subtle cues within a synthetic face image that cause humans to perceive it as a fake without having to search for obvious clues? To answer these questions, we conducted a series of large-scale crowdsourced behavioral experiments with different sources of face imagery. Results show that humans are unable to distinguish synthetic faces from real faces under several different circumstances. This finding has serious implications for many different applications where face images are presented to human users.

In addition, augmented reality and other photo editing filters are popular methods used to modify faces online. Considering the important role of facial perception in communication, how do we perceive this increasing number of modified faces? In subsequent work we present the results of six surveys that measure familiarity with different styles of facial filters, perceived strangeness of faces edited with different filters, and ability to discern whether images are filtered. Our results demonstrate that faces modified with more traditional face filters are perceived similarly to unmodified faces, and faces filtered with augmented reality filters are perceived differently from unmodified faces. We discuss possible explanations for these results, including a societal adjustment to traditional photo editing techniques or the inherent differences in the different types of filters. We conclude with a discussion of how to build online spaces more responsibly based on our results.

Publications

  • "Has the Virtualization of the Face Changed Facial Perception?
    A Study of the Impact of Augmented Reality on Facial Perception,"
    Louisa Conwill, Samuel Anthony, Walter Scheirer,
    Proceedings of the ACM CHI Conference on Human Factors in Computing Systems,
    May 2024.
  • "A Study of the Human Perception of Synthetic Faces,"
    Bingyu Shen, Brandon RichardWebster, Alice O'Toole, Kevin Bowyer, Walter J. Scheirer,
    Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition (FG),
    December 2021.