Collaborative Research: Enabling Machine Learning based Cooperative Perception with mmWave Communication for Autonomous Vehicle Safety

University Of Massachusetts, Dartmouth, North Dartmouth MA

Investigators

Abstract

By understanding what and how data are exchanged among autonomous vehicles, from a machine learning perspective, it is possible to realize precise cooperative perception on autonomous vehicles, enabling massive amounts of sensor information to be shared amongst vehicles. Such an advance can be extremely useful to extend the line of sight and field of view of autonomous vehicles, which otherwise suffers from blind spots and occlusions. The extended field of view on autonomous vehicles will be beneficial at times when there are occlusions preventing a complete perception of the environment. This increase in situational awareness promotes safe driving over a narrow scope and improves traffic flow efficiency over an extended scope. The proposed research work will not only change the way people think about the perception system on autonomous vehicles but could also open up opportunities to design novel systems that were previously inconceivable. This project offers a wide variety of research activities from data collection, algorithm design, system development, and in-the-field evaluation, which will be attractive to students with various backgrounds and goals. Undergraduate and graduate students will be involved directly in the research activities as assistants at different levels. The expected research outcomes from this project will also enhance the current curricula related to machine learning, Internet of things, and wireless communications. The main research objective of this project is to understand the sensing and communication challenges to achieving cooperative perception among autonomous vehicles, and to use the insights thus gained to guide the design of suitable data exchange format, data fusion algorithms, and efficient millimeter wave vehicular communications. Results from this project will include a machine learning based cooperative perception framework, which will shed light on effectively combining feature maps, derived from machine learning models on autonomous vehicles, in a distributed manner. The resulted feature map compression and feature map selection approaches will significantly reduce the amount of data exchanged among vehicles, enabling agile and precise cooperative perception on connected and autonomous vehicles. The proposed scalable feature map transmission mechanism jointly considers the application requirements, link and physical layer characteristics of millimeter wave links, enabling sensor data sharing on a massive scale among autonomous vehicles. The implemented system and evaluation platform will serve as a convincing proof-of-concept for the proposed solution, thus opening the door to widespread adoption of cooperative perception applications via millimeter wave communications in future vehicle networks. The collected dataset from this project will be made publicly available, serving as a catalyst for enabling innovative research on cooperative object detection, vehicular edge computing, and machine learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.