RINGS: l-RIM: Learning based Resilient Immersive Media-Compression, Delivery, and Interaction

Auburn University, Auburn AL

Investigators

Abstract

Augmented Reality/Virtual Reality (AR/VR) has been recognized as a transformative service for the Next Generation (NextG) network systems, while wireless supported AR/VR will offer great flexibility and enhanced immersive experience to users and unleash a plethora of new applications. The primary goal of this project is to explore innovative technologies in NextG and Artificial intelligence (AI) to provide a unified media compression, communication, and computing framework to enable resilient wireless AR/VR. The research agenda comprises five thrusts. Thrusts 1 and 2 are focused on learning-based immersive media compression, i.e., developing high-efficiency Light Field and Point Cloud compression solutions, respectively. The next two thrusts explore the fundamental concepts and techniques that facilitate wireless AR/VR transmission and interaction. Thrust 3 shall develop theoretical models and explore the fundamental performance tradeoffs among compression communication, computation, and caching for wireless AR/VR considering several NextG communications and networking technologies. The work in Thrust 4 shall investigate the application of edge intelligence to develop learning-based algorithms for resilient AR/VR transmissions. The design of resilient AR/VR will be guided by the model-based performance analysis developed in Thrust 3. Thrust 5 will integrate the techniques developed in Thrusts 1-4, and validate their performance with simulation studies using open-source datasets and experimental studies using an AR/VR testbed and publicly available wireless and cloud-related platforms. The proposed research has a high potential to achieve great impacts in our research community and society. The wireless community has been making great efforts on developing learning-based solutions for the NextG wireless and mobile communication, networking, sensing, and computing systems. Findings from this project are expected to facilitate breakthroughs in enabling immersive media applications that require huge amounts of data to train complex deep learning models. The PIs are committed to integration of research and education. This project will offer research and educational opportunities for undergraduate and graduate students at the two collaborating institutions. New training certificate courses will be developed and research outcomes will be incorporated into a forthcoming textbook by PI Mao. The several existing outreach programs at the two institutions will be leveraged to broaden participation from underrepresented groups. Research outcomes will be disseminated through scholarly publications, a project website, open-source repositories, as well as workshops and journal special issues to be organized by the PIs. Industry will also be pursued through the NSF IUCRC Center for Big Learning at The University of Missouri, Kansas City. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.