CRII: RI: Testing and Interpreting Image-based Computer Vision Models in 3D Space

Auburn University, Auburn AL

Investigators

Abstract

From autonomous vehicles to cancer detection to speech recognition, artificial intelligence (AI) is transforming many economic sectors. While being increasingly ubiquitous, AI algorithms have been shown to easily misbehave when encountering natural, unexpected, never-seen inputs in the real world. For example, when a car on autopilot failed to recognize a white truck against a bright-lit sky, it crashed into the truck, killing the driver. To avoid such costly and unsafe failures, this project develops a framework for rigorously and automatically testing AI algorithms, specifically computer vision systems, in a 3D environment. In addition, via the framework, the project attempts to uncover why an algorithm makes a given decision. Providing explanations understandable by humans for decisions made by machines is crucial in gaining users' trust, advancing AI algorithms, and complying with the current and future legal regulations on the use of AI with sensitive human data. Researchers previously attempted to achieve the two main goals of (1) testing and (2) interpreting computer vision systems by synthesizing a 2D input image that fails a target image recognition model. However, the existing methods operate at the pixel level, generating special patterns that (a) are hard to explain; (b) might not transfer well to the physical world; and (c) may rarely be encountered in reality. Instead of optimizing in the 2D image space, the research objective of this project is to harness 3D graphics engines to create a 3D scene where the factors of variations (e.g. lighting, object geometry and appearances, background images) can be controlled and optimized to cause a target computer vision system to misbehave. This research effort will (1) reveal systematic defects via automatically testing the target model across many controlled, disentangled settings; and (2) improve the existing interpretability methods by incorporating 3D information. The developed methods attempt to provide explanations for the decisions made by computer vision models and create new insights into their inner functions. The project will improve the safety, reliability, and transparency of AI algorithms. This project is jointly funded by the Robust Intelligence (RI) and the Established Program to Stimulate Competitive Research (EPSCoR) programs. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.