Machine learning, and artificial intelligence more broadly, presently inhabits an ambiguous space between liberating and anti-social technologies. This work is looking at socially constructive use cases and fixes for existing problems. With respect to new use cases that help society, we're developing an AI early warning system to monitor how manipulated content online such as altered photos in memes leads, in some cases, to violent conflict and societal instability. It can also potentially interfere with democratic elections. Look no further than the 2019 Indonesian election to learn how online disinformation can have an unfortunate impact on the real world. Our system may prove useful to journalists, peacekeepers, election monitors, and others who need to understand how manipulated content is spreading online during elections and in other contexts.

When it comes to existing problems, applied machine learning research has the potential to fuel further advances in data science, but it is greatly hindered by an ad hoc design process, poor data hygiene, and a lack of statistical rigor in model evaluation. Recently, these issues have begun to attract more attention as they have caused public and embarrassing issues in research and development, such as claims that machine learning can predict criminality based on the appearance of a face. Drawing from our experience as machine learning researchers, we follow the applied machine learning process from algorithm design to data collection to model evaluation, drawing attention to common pitfalls and providing practical recommendations for improvements. At each step, case studies are introduced to highlight how these pitfalls occur in practice, and where things could be improved.

This work was supported by USAID under agreement number 7200AA18CA00054, DARPA and Air Force Research Laboratory (AFRL) under agreement number FA8750-16-2-0173, and the NVIDIA Corporation.