Walter J. Scheirer

walter.scheirer@nd.edu, @wjscheirer, @wjscheirer.bsky.social

Dennis O. Doughty Collegiate Professor of Engineering
Department of Computer Science and Engineering
University of Notre Dame

Research Overview

My technical research is primarily focused around the problem of recognition, including the representations and algorithms supporting solutions to it. I am particularly interested in features and learning-based methods that apply to both vision and language, thus breaking away from the persistent compartmentalization of recognition tasks (something hinted at by David Marr over 30 years ago). This has led to some intriguing and often unconventional approaches that can be applied to a broad set of areas within artificial intelligence including computer vision, machine learning, and human biometrics. Specifically, my work is looking at open world recognition, extreme value theory models for visual recognition, and biologically-inspired learning algorithms. I'm also a cultural critic and historian, commenting on the social context of emerging technologies from the realistic perspective of a computer scientist.

Curriculum Vitae

PDF Format

Postdoctoral Fellows

Steve Cruz (Co-Advised with Jonathan Hauenstein)

Graduate Students

Arturo Russell Bernal (Co-Advised with Jane Cleland-Huang)

Abigail Swenor

Teaching

Spring 2025: CSE 40535 Computer Vision

Fall 2024: CSE 40175 Ethical & Professional Issues

Spring 2024: CSE 20176 The Archeology of Hacking

Books

Virtue in Virtual Spaces (Liturgical Press) with Louisa Conwill and Megan Levis

A History of Fake Things on the Internet (Stanford University Press)

A Unifying Framework for Formal Theories of Novelty (Springer-Nature) edited with Terry Boult

Quantitative Intertextuality: Analyzing the Markers of Information Reuse (Springer-Nature) with Christopher W. Forstall

Extreme Value Theory-based Methods for Visual Recognition (Morgan & Claypool Publishers)

Recent Publications (see all)

"This Probably Looks Exactly Like That: An Invertible Prototypical Network,"
Zachariah Carmichael, Timothy Redgrave, Daniel Gonzalez Cedre, Walter Scheirer,
Proceedings of the European Conference on Computer Vision (ECCV),

October 2024.

[pdf] [code]
[bibtex]
@InProceedings{Carmichael2024,
author = {Carmichael, Zachariah and
Redgrave, Timothy and
Cedre, Daniel Gonzalez and
Scheirer, Walter J},
title = {This Probably Looks Exactly Like That:
An Invertible Prototypical Network},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2024}
}
"Human Activity Recognition in an Open World,"
Derek Prijatelj, Samuel Grieggs, Jin Huang, Dawei Du, Ameya Shringi, Christopher Funk,
Adam Kaufman, Eric Robertson, Walter Scheirer,
Journal of Artificial Intelligence Research,

Accepted for Publication in September 2024.

[pdf] [code]
[bibtex]
@misc{https://doi.org/10.48550/arxiv.2212.12141,
author = {Prijatelj, Derek S. and
Grieggs, Samuel and
Huang, Jin andi
Du, Dawei and
Shringi, Ameya and
Funk, Christopher and
Kaufman, Adam and
Robertson, Eric and
Scheirer, Walter J.},
title = {Human Activity Recognition in an Open World},
publisher = {arXiv},
year = {2022}
}
"Has the Virtualization of the Face Changed Facial Perception?
A Study of the Impact of Augmented Reality on Facial Perception,"
Louisa Conwill, Samuel Anthony, Walter Scheirer,
Proceedings of the ACM CHI Conference on Human Factors in Computing Systems,

May 2024.

[pdf]
[bibtex]
@InProceedings{conwill2024virtualization,
title={Has the Virtualization of the Face Changed Facial Perception?
A Study of the Impact of Augmented Reality on Facial Perception},
author={Louisa Conwill and
Samuel Anthony and
Walter Scheirer},
year={2024},
booktitle={Proceedings of the ACM CHI Conference
on Human Factors in Computing Systems}
}
"Informing Machine Perception With Psychophysics,"
Justin Dulay, Sonia Poltoratski, Till Hartmann, Samuel Anthony,
Walter Scheirer,
Proceedings of the IEEE,

February 2024.

[pdf] [code]
[bibtex]
@article{dulay2024informing,
title={Informing Machine Perception With Psychophysics},
author={Dulay, Justin and
Poltoratski, Sonia and
Hartmann, Till S and
Anthony, Samuel E and
Scheirer, Walter J},
journal={Proceedings of the IEEE},
volume={112},
number={2},
pages={88--96},
year={2024},
publisher={IEEE}
}
"C-CLIP: Contrastive Image-Text Encoders to Close the Descriptive-Commentative
Gap,"
William Theisen, Walter Scheirer,
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
(WACV),

January 2024.

[pdf]
[bibtex]
@inproceedings{theisen2024c,
title={C-CLIP: Contrastive Image-Text Encoders
to Close the Descriptive-Commentative Gap},
author={Theisen, William and Scheirer, Walter J},
booktitle={Proceedings of the IEEE/CVF Winter Conference
on Applications of Computer Vision (WACV)},
pages={7241--7250},
year={2024}
}
"Pixel-Grounded Prototypical Part Networks,"
Zachariah Carmichael, Suhas Lohit, Anoop Cherian, Michael Jones, Walter Scheirer,
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
(WACV),

January 2024.

[pdf] [code]
[bibtex]
@inproceedings{carmichael2024pixel,
title={Pixel-grounded Prototypical Part Networks},
author={Carmichael, Zachariah and
Lohit, Suhas and
Cherian, Anoop and
Jones, Michael J and
Scheirer, Walter J},
booktitle={Proceedings of the IEEE/CVF Winter Conference
on Applications of Computer Vision (WACV)},
pages={4768--4779},
year={2024}
}
"NOMAD: A Natural, Occluded, Multi-scale Aerial Dataset, for Emergency Response
Scenarios,"
Arturo Russell Bernal, Jane Cleland-Huang, Walter Scheirer,
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
(WACV),

January 2024.

[pdf]
[bibtex]
@inproceedings{russell2024nomad,
title={NOMAD: A Natural, Occluded, Multi-scale Aerial Dataset,
for Emergency Response Scenarios},
author={Russell Bernal, Arturo Miguel and
Scheirer, Walter and
Cleland-Huang, Jane},
booktitle={Proceedings of the IEEE/CVF Winter Conference
on Applications of Computer Vision (WACV)},
pages={8584--8595},
year={2024}
}
"The Paleographer’s Eye ex machina:
Using Computer Vision to Assist Humanists in Scribal Hand Identification,"
Samuel Grieggs, Cai Henderson, Sebastian Sobecki, Alexandra Gillespie, Walter Scheirer,
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
(WACV),

January 2024.

[pdf]
[bibtex]
@inproceedings{grieggs2024paleographer,
title={The Paleographer's Eye ex machina:
Using Computer Vision To Assist Humanists in Scribal Hand Identification},
author={Grieggs, Samuel and
Henderson, CEM and
Sobecki, Sebastian and
Gillespie, Alexandra and
Scheirer, Walter},
booktitle={Proceedings of the IEEE/CVF Winter Conference
on Applications of Computer Vision (WACV)},
pages={7177--7186},
year={2024}
}
"Measuring Human Perception to Improve Open Set Recognition,"
Jin Huang, Derek Prijatelj, Justin Dulay, Walter Scheirer,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),

September 2023.

[pdf] [supp. material] [code] [data]
[bibtex]
@article{huang2023measuring,
title={Measuring Human Perception to Improve Open Set Recognition},
author={Huang, Jin and Prijatelj, Derek and Dulay, Justin and Scheirer, Walter},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume={45},
number={9},
pages={11382--11389},
year={2023},
publisher={IEEE}
}
"Automated Transcription of Gə'əz Manuscripts Using Deep Learning,"
Samuel Grieggs, Jessica Lockhart, Alexandra Atiya, Suzanne Akbari,
Eyob Derillo, Jarod Jacobs, Christine Kwon, Michael Gervers, Steve Delamarter,
Alexandra Gillespie, Walter J. Scheirer,
Digital Humanities Quarterly,

August 2023.

[pdf] [code]
[bibtex]
@article{Grieggs_DHQ2023,
author = {Samuel Grieggs and
Jessica Lockhart and
Alexandra Atiya and
Gelila Tilahun and
Suzanne Akbari and
Eyob Derillo and
Jarod Jacobs
Christine Kwon
Michael Gervers
Steve Delamarter
Walter J. Scheirer},
title = {Automated Transcription of Gə'əz Manuscripts Using Deep Learning},
journal = {Digital Humanities Quarterly},
volume = {17},
number = {3},
month = {August},
year = {2023}
}

New Research in arXiv / bioRxiv

"N-Modal Contrastive Losses with Applications to Social Media Data in
Trimodal Space,"
William Theisen, Walter Scheirer,
March 2024.

[pdf]
[bibtex]
@article{theisen2024n,
title={N-Modal Contrastive Losses with Applications to Social Media
Data in Trimodal Space},
author={Theisen, William and Scheirer, Walter},
journal={arXiv preprint arXiv:2403.12747},
year={2024}
}
"HomOpt: A Homotopy-Based Hyperparameter Optimization Method,"
Sophia Abraham, Kehelwala Maduranga, Jeffery Kinnison, Zachariah Carmichael,
Jonathan Hauenstein, Walter Scheirer,
August 2023.

[pdf]
[bibtex]
@article{abraham2023homopt,
title={HomOpt: A Homotopy-Based Hyperparameter Optimization Method},
author={Abraham, Sophia J and
Maduranga, Kehelwala DG and
Kinnison, Jeffery and
Carmichael, Zachariah and
Hauenstein, Jonathan D and
Scheirer, Walter J},
journal={arXiv preprint arXiv:2308.03317},
year={2023}
}
"On the Effectiveness of Image Manipulation Detection in the Age of Social
Media,"
Rosaura G. VidalMata, Priscila Saboia, Daniel Moreira, Grant Jensen,
Jason Schlessman, Walter Scheirer,
April 2023.

[pdf]
[bibtex]
@misc{vidalmata2023effectiveness,
title={On the Effectiveness of Image Manipulation Detection in the Age of
Social Media},
author={Rosaura G. VidalMata and
Priscila Saboia and
Daniel Moreira and
Grant Jensen and
Jason Schlessman and
Walter J. Scheirer},
year={2023},
eprint={2304.09414},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
"Using Human Perception to Regularize Transfer Learning,"
Justin Dulay, Walter Scheirer,
November 2022.

[pdf]
[bibtex]
@misc{Dulay2022b,
doi = {10.48550/ARXIV.2211.07885},
url = {https://arxiv.org/abs/2211.07885},
author = {Dulay, Justin and Scheirer, Walter J.},
title = {Using Human Perception to Regularize Transfer Learning},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution Share Alike 4.0 International}
}

Current and Long-Standing Projects (see all)

Open World Recognition

Spring 2011 - Present

Theory and algorithms that address the difficult problem of training without complete class knowledge

Keywords: novelty, classifiers, architectures, object recognition, activity recognition, handwriting recognition

Psychophysics for Computer Vision

Fall 2012 - Present

Measuring exemplar-by-exemplar difﬁculty and the pattern of errors of humans for supervised learning

Keywords: psychology, citizen science, psychophysics, object recognition, face detection, attributes

Extreme Value Theory for Visual Recognition

Spring 2008 - Present

The theory and practice of recognition score analysis for prediction and fusion

Keywords: calibration, meta-recognition, score analysis, object recognition, face recognition, attributes

Language & Literature

Spring 2009 - Present

Machine learning and related statistical methods to open up new modes of inquiry in the humanities

Keywords: computational linguistics, intertextuality, sound, stylistics, classics, historical document analysis

Digital Image and Video Forensics

Spring 2008 - Present

Approaches to edited content detection, synthetic content detection, and image provenance analysis

Keywords: forensics, image manipulation, image synthesis, provenance analysis, semantic content analysis

Hyperparameter Optimization: Distributed Hardware-Aware and Homotopy-Based Strategies

Spring 2017 - Present

Intelligent mapping of hardware resources to model search tasks making use of homotopy optimization

Keywords: machine learning, hyperparameters, optimization, distributed computing, homotopy

Human-Drone Partnerships for Emergency Response

Fall 2019 - Present

Semi-autonomous UAVs to support emergency response scenarios, such as fire surveillance and search and rescue

Keywords: drones, software engineering, participatory design, object detection, first responders

When Technology Falls Short

Fall 2019 - Present

Constructive criticism of emerging technologies, scientific trends, and policies

Keywords: machine learning, facial analysis, election integrity, scientific integrity, peace studies

Prototypical Part Networks

Summer 2022 - Fall 2024

Learning prototypes of object parts for model explainability by design

Keywords: machine learning, neural networks, object recognition, object parts, XAI

Selected Talks (see all)

A History of Fake Things on the Internet Book Talk: Photoshop Fantasies
Recording of the Hacker History Panel from DEF CON 32
Invited talk at the CVPR 2024 Workshop on Open World Vision: Open Issues in Open World Learning
Invited talk at the Workshop on Humans, Deep Networks & Face Recognition: Restyling Reality: How the Internet Shapes Facial Perception
Invited talk at the CVPR 2022 Workshop Open World Vision: A Unifying Framework for Formal Theories of Novelty in Visual Perception
Invited talk at the WACV 2022 Workshop xAI4Biometrics: Visual Psychophysics for Making Face Recognition Algorithms More Explainable
Invited talk at the Computational Research Integrity Conference (CRI-CONF 21): Understanding the Provenance of Visual Disinformation Targeting Science
Invited talk at the IEEE International Symposium on Technology and Society Conference (ISTAS 20): The 'Criminality from Face' Illusion
Invited talk at the WACV 2020 Workshop Neural Architecture Search for Computer Vision in the Wild: Representational Dissimilarity Analysis as a Tool for Neural Network Model Search
Talk at the 2019 Beijing Forum Panel on Humanity and Legal Governance in the Digital Era: New Perspectives on Digital Media Integrity
Keynote talk at the 18th International Conference of the Biometrics Special Interest Group (BIOSIG 2019): The Limits and Potentials of Deep Learning for Facial Analysis
Talk at the DEF CON 27 AI Village: Backdooring Convolutional Neural Networks with Targeted Weight Perturbations
Invited talk at the CVPR 2018 Workshop "Real-World Challenges and New Benchmarks for Deep Learning in Robotic Vision":
A Psychophysics Driven Evaluation Framework for Visual Recognition
CVPR 2016 Tutorial: Statistical Methods for Open Set Recognition

Activities

CVPR 2025 will be held in Nashville, Tennessee, June 11th - 15th
WACV 2025 will be held in Tucson, Arizona, February 28th - March 4th
View the very hypnagogic AI Art Gallery from CVPR 2024
CVPR 2024 was held in Seattle, Washington, June 17th - 21st (I survived being the lead general chair!)
Join the IEEE Computer Society PAMI-TC (now on LinkedIn) and the Computer Vision Foundation