NRI: FND: Scene Understanding and Predictive Monitoring for Safe Human-Robot Collaboration in Unstructured and Dynamic Construction Environments

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

The construction industry has the highest number of fatalities and injuries due to hazardous working conditions. The introduction of robots on construction sites has the potential to relieve human workers from dangerous and repetitive tasks by making machines intelligent and autonomous. However, robotic solutions for construction face significant challenges. This project will develop technologies of automated monitoring and intervention through computer vision to provide a means to dramatically improve the perception of construction safety in the presence of co-robots. The new methods developed in this project will impact computer vision, machine learning, and effective human-robot collaboration in unstructured environments, while significantly contributing to safety. Further, the developed methodologies can be broadly applicable in situations where robots are deployed in human-centered environments (hospitals, airports, shipyards, etc.) and have other priorities such as productivity and efficiency as their objective. This project will engage a diverse group of individuals by training graduate and undergraduate students (including women and underrepresented minorities), reaching out to K-12 students, and interacting with industry professionals for broad dissemination of the research results. This research will investigate new computer vision based methods that can be coupled with other sensing modalities for holistic understanding and predictive analysis of jobsite safety on co-robotic construction sites. The project will consist of two main research thrusts. First, holistic scene understanding will be pursued on construction sites using graphical models to enable joint reasoning of various scene components. This holistic understanding in turn will help evaluate compliance with established safety rules expressed as formal statements. Second, predictive analysis will be investigated by exploiting the fact that, for safety intervention, the complex dynamics of a construction scene make it necessary to simulate what will happen next. In particular, Recurrent Neural Networks will be leveraged to predict future events and prevent impending accidents. Finally, an integrated demonstration system will be built and tested on real construction sites.