RI: Medium: To Sense or Not to Sense: Energy Efficient Adaptive Sensing for Autonomous Systems

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Sensing and computation have been crucial to the significant progress in semi- and fully-autonomous vehicles and robots. Proliferation of many types of sensors (LIDARs, cameras, RADARs, etc.) and the advent of compute-heavy and data-hungry deep-learning approaches have increased the performance of autonomous systems by leaps and bounds. But the wide variety of sensors differ in terms of their performance, cost, and operational difficulty. Thus, specific sets of sensors are chosen for a particular task on a particular robot. This horses-for-courses approach often results in one-off systems that are incapable of adapting to many tasks or robots. Thus, to ensure safety and reliability, multi-tasking systems like autonomous vehicles have resorted to over-engineering, with upwards of 15 sensors and multiple GPUs/CPUs in any car. And, to make matters worse, many of the sensed data is eventually discarded as unwanted background. Thus, while the energy footprint of sensing and computations is increasing at an alarming rate, the flexibility or adaptability of these systems is still lacking. Much of this state of affairs can be attributed to the fact that sensors and algorithms face vastly different hardware and software challenges and are hence designed, developed, and manufactured in separate academic units or industries. This project takes a different approach: adaptively sense mostly (if not only) quantities which help solve the task accurately and within the allotted time. In other words, this project advocates folding adaptive and flexible sensing within a learning framework for autonomous systems. This is achieved by co-design and co-execution of sensing and algorithms to maximize accuracy and flexibility while minimizing expended energy and cost. The approach is motivated by how humans decide what, where, when, and how to sense and apply that to a robot learning framework. Research and education are closely integrated in a diverse and inclusive environment. The project consists of three fundamental research thrusts. Thrust 1: Development of highly novel and fully adaptive design and physical realization of 3D optical sensors. This thrust includes a fundamental mathematical framework that determines the optimal set of emitted and measured rays to achieve a particular task at hand. This is the mathematical foundation for developing a new class of sensors that detect and characterize obstacles---a time critical task of any autonomous system---with maximum energy efficiency, minimal latency (i.e., near-instantly) and with virtually no separate computation. Thrust 2: Novel decision-making framework that efficiently controls the adaptive sensors for the task at hand. This includes determining where and when to sense and adapting behavior policies accordingly. Thrust 3: Support the robot learning framework by learning and interacting with humans. The project will demonstrate the generality of adaptive sensing using three disparate autonomous systems that have broad societal impact: a) autonomous vehicles, b) assistive robots, and c) robots in manufacturing. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.