RUI: CCSS: Collaborative Research: Cooperative Unmanned Aerial Vehicles Enabled Scalable Mobile Panoramic Video Surveillance

University Of Missouri-Columbia, Columbia MO

Investigators

Abstract

This project investigates a cyber-physical system (CPS) in which cooperative unmanned aerial vehicles (UAVs) are used to enable scalable mobile panoramic video surveillance. This system generates mobile real-time video panorama by flying a fleet of cooperative UAVs. It is an interdisciplinary collaborative effort that synthesizes the expertise of Computer Science and Aerospace Engineering. This project demonstrates a convergence of sensing, control and communication. The project will generate outcomes including distributed video stitching algorithms, scalable distributed systems, and cooperative optimal flight control algorithms. This project significantly improves the performance such as delay and error in video panorama results. It advances the frontiers of both video processing and control theory. This interdisciplinary effort will promote and contribute to crosscutting collaborations between two research communities: computer science and aerospace engineering. The project aims to improve the education of underrepresented student populations by involving them in the proposed research activities and enhancing course curriculum. Results of this project have broad applications to areas such as remote sensing, environmental sampling and monitoring, homeland security, etc. The project has three research thrusts: 1) distributed hierarchical video stitching to generate mobile video panorama across cooperative UAVs with high scalability, 2) integrated optimal cooperative control of UAV formation flying to support the distributed video stitching, and 3) integration, interfacing and communication to unite the distributed hierarchical video stitching and the cooperative control of formation flying. A prototype system will be developed for evaluation. It has four salient merits. First, the distributed hierarchical architecture is scalable to mobile cameras and fault-tolerant. Second, it exploits the temporal and spatial features of video frames for efficient computation. Third, the integrated optimal cooperative flight control addresses the multiple cooperative objectives in one unified framework, and provides a computationally efficient, distributed and control. Forth, the unification of distributed video stitching and cooperative control of UAV facilitates the generation of mobile video panorama. In particular, the distributed video stitching minimizes the stress of intensive computation required in stitching. It maintains high synchronization among video frames by pushing the stitching to the front close to the cameras. The proposed hierarchical video stitching significantly contributes to distributed system theory and algorithms. In addition, the stitching algorithms exploit the spatial and temporal correlation among UAV videos, yielding a novel contribution to computer graphics/vision. The proposed optimal cooperative control method will significantly advance the cooperative control of multi-vehicle or multi-agent systems. It integrates many challenging cooperative problems into one unified optimization framework. This method enables a number of desired capabilities: closed-form, distributed and local information based control law, synchronous formation, cooperative tracking, and obstacle/collision avoidance. These integrated features not only make precise and real-time cooperative surveillance possible, but also move forward the cooperative control theory to a new horizon for a wide range of cooperative missions.