CPS: Medium: GOALI: Real-Time Computer Vision in Autonomous Vehicles: Real Fast Isn't Good Enough

University Of North Carolina At Chapel Hill, Chapel Hill NC

Investigators

Abstract

The push towards deploying autonomous-driving capabilities in vehicles is happening at breakneck speed. Semi-autonomous features are becoming increasingly common, and fully autonomous vehicles at mass-market scales are on the horizon. Cameras are cost-effective sensors, so computer-vision techniques have loomed large in implementing autonomous features. In a vehicle, these techniques must function "in real time." Unfortunately, this requirement lies at the heart of a significant disconnect: when computer-vision researchers refer to "real time," they usually mean "real fast"; in contrast, certifiable automotive systems must be "real time" in the sense of being able to predictably react to input information (such as a detected pedestrian) within specified deadlines so that adverse outcomes (such as striking a pedestrian) are provably precluded. The goal of this project is to eliminate this disconnect. It will do so through research on several fronts. First, a real-time computer-vision programming framework will be created by extending OpenVX, which is a recently ratified standard intended for developing computer-vision applications for embedded systems. Second, new computer-vision algorithms that exploit the features of this programming framework will be created, and methods will be developed to transform existing algorithms to make them "real time" in a predictability sense. Third, an experimental evaluation of "real-fast" vs. "real-time" computer vision will be conducted using driving simulators, sub-scale autonomous vehicles, and advanced testing infrastructure at General Motors. While industry is pushing hard in the area of autonomous driving, autonomous vehicles will never become a common mode of transportation unless methods for certifying real-time safety are produced. This project will focus on a key aspect of certification: validating the real-time correctness of computer-vision applications. The results that are produced will be made available to the world at large through open-source software. This software will include the new programming framework to be produced as well as tools for validating the real-time correctness of applications developed using this framework. In this project, a special emphasis will be placed on outreach to girls and women, as three female graduate students will be involved in the project. Such outreach will include: events involving the Graduate Women in Computer Science (GWiCS) group at the University of North Carolina (UNC), which hosts an annual research symposium targeted toward undergraduate women and other under-represented minorities; Tar Heel Hack, a hackathon for local middle and high school girls; the UNC Girls Who Code Club, which provides local girls in grades 6-12 with a community for learning about computer science; and the UNC Computer Science Department's annual Open House and Science Expo. These events will include hackathon projects as well as demos of a driving simulator and a sub-scale autonomous car. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.