Collaborative Research: NeTS: Small: A Privacy-Aware Human-Centered QoE Assessment Framework for Immersive Videos

University Of Texas At Arlington, Arlington TX

Investigators

Abstract

Immersive videos, also known as 360-degree videos, provide viewers with a complete visual perspective of their environment. The global market for immersive video is projected to reach 22 billion U.S. dollars by 2025. With the rising popularity of 360-degree videos, network operators and service providers are increasingly keen on gaining insights into users’ Quality of Experience (QoE) perception. However, the conventional QoE models designed for two dimensional videos fall short of capturing users’ unique subjective feelings in viewing this new type of video. To bridge this gap, the project focuses on constructing human-centered models that utilize “human factors” via a variety of virtual reality (VR) onboard sensors. Leveraging multi-modal sensory readings, the researchers will tap into a direct representation of users’ perceptual experiences. This approach captures subjective feelings often missed by the objective system parameters commonly used in existing QoE models. The success of this project will provide an in-depth understanding of users’ nuanced perceptual experience while engaging with 360-degree videos. It will create opportunities for stakeholders to implement user-specific network resource optimization and video streaming strategies, ultimately enhancing customer satisfaction on an individual basis. The research outcomes will significantly contribute to the human-centered sensing and networking research communities and benefit numerous VR applications beyond 360-degree videos. This project involves three closely related research thrusts. The first thrust investigates how to extract salient features from multi-modal sensory readings for effective QoE assessment. To combat bias arising from datasets with unevenly distributed labels, novel approaches will be developed to augment underrepresented data by exploring useful information across sensing modalities. The second thrust seeks to optimize system resource utilization for QoE assessment. Multiple personalized models will share common neural network layers at edge servers for resource-efficient model hosting. Adaptive sampling will be applied at VR terminals during sensory data acquisition to maintain data utility. The third thrust focuses on devising data privacy protection mechanisms without sacrificing QoE assessment accuracy. Under the differential privacy framework, the approach features formal quantification of data correlation of multi-modality sensory data, which has been largely overlooked by prior work. The proposed research is inter-disciplinary, which spans data-driven modeling, sensing, and privacy-preserving computing. The proposed mechanisms and designs will be thoroughly evaluated via a combination of measurement campaigns, simulations, and experimental studies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.