Face Synthesis

Keywords: face synthesis, face recognition, face analysis, computer graphics, data augmentation
Fall 2015 - Winter 2023
teaser image

Description

We propose an algorithm to generate realistic face images of both real and synthetic identities (people who do not exist) with different facial yaw, shape and resolution. The synthesized images can be used to augment datasets to train CNNs or as massive distractor sets for biometric verification experiments without any privacy concerns. Additionally, law enforcement and intelligence services can make use of this technique to train forensic experts to recognize faces. Our method samples face components from a pool of multiple face images of real identities to generate the synthetic texture. Then, a real 3D head model compatible to the generated texture is used to render it under different facial yaw transformations. We perform multiple quantitative experiments to assess the effectiveness of our synthesis procedure in CNN training and its potential use to generate distractor face images. Additionally, we compare our method with popular GAN models in terms of visual quality and execution time.

Hardware support was provided by the NVIDIA Corporation.

Publications

  • "Analyzing the Impact of Shape & Context on the Face Recognition Performance
    of Deep Networks ,"
    Sandipan Banerjee, Walter J. Scheirer, Kevin Bowyer, Patrick Flynn,
    Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition (FG),
    January 2023.
    [pdf]
    [bibtex]
    @misc{https://doi.org/10.48550/arxiv.2208.02991,
    doi = {10.48550/ARXIV.2208.02991},
    url = {https://arxiv.org/abs/2208.02991},
    author = {Banerjee, Sandipan and
    Scheirer, Walter and
    Bowyer, Kevin and
    Flynn, Patrick},
    title = {Analyzing the Impact of Shape & Context on the Face Recognition Performance
    of Deep Networks},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
    }
  • "A Study of the Human Perception of Synthetic Faces,"
    Bingyu Shen, Brandon RichardWebster, Alice O'Toole, Kevin Bowyer, Walter J. Scheirer,
    Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition (FG),
    December 2021.
    [pdf]
    [bibtex]
    @inproceedings{Shen_FG2021,
    author = {Bingyu Shen and
    Brandon RichardWebster and
    Alice O'Toole and
    Kevin Bowyer and
    Walter J. Scheirer},
    title = {A Study of the Human Perception of Synthetic Faces},
    booktitle = {Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition (FG),
    year = {2021}
    }
  • "On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs,"
    Sandipan Banerjee, Walter J. Scheirer, Kevin Bowyer, Patrick Flynn,
    IEEE Winter Conference on Applications of Computer Vision (WACV)
    March 2020.
    [pdf]
    [bibtex]
    @inproceedings{BanerjeeWACV2020,
    author = {Sandipan Banerjee and
    Walter J. Scheirer and
    Kevin W. Bowyer and
    Patrick J. Flynn},
    title = {On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs},
    booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
    year = {2020}
    }
  • "Fast Training-free Face Image Synthesis,"
    Sandipan Banerjee, Walter J. Scheirer, Kevin Bowyer, Patrick Flynn,
    IEEE Winter Conference on Applications of Computer Vision (WACV),
    January 2019.
    [pdf]
    [bibtex]
    @InProceedings{Banerjee_2019_WACV,
    author = {Sandipan Banerjee and Walter J. Scheirer and Kevin Bowyer and Patrick Flynn},
    title = {Fast Training-free Face Image Synthesis},
    booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
    month = {January},
    year = {2019}
    }
  • "SREFI: Synthesis of Realistic Example Face Images,"
    Sandipan Banerjee, John S. Bernhard, Walter J. Scheirer, Kevin Bowyer, Patrick Flynn,
    Proceedings of the IAPR/IEEE International Joint Conference on Biometrics (IJCB),
    October 2017.
    [pdf]
    [bibtex]
    @InProceedings{Banerjee_2017_IJCB,
    author = {Sandipan Banerjee and John S. Bernhard and Walter J. Scheirer and Kevin Bowyer and Patrick Flynn},
    title = {SREFI: Synthesis of Realistic Example Face Images},
    booktitle = {The IAPR/IEEE International Joint Conference on Biometrics (IJCB)},
    month = {October},
    year = {2017}
    }