PFI:BIC: Multimodal-Sensor-Enabled Environments with Advanced Cognitive Computing Enabling Smart Group Meeting Facilitation Services.

Rensselaer Polytechnic Institute, Troy NY

Investigators

Abstract

Millions of meetings take place every day in the US, incurring a tremendous cost in terms of managers' and employees' precious time and salary. Unfortunately, group meetings suffer from serious problems that undermine productivity and collegiality, including overt or unconscious bias, "groupthink", fear of speaking, and unfocused discussion. Few automatic tools exist for keeping meetings on track, accurately recording who said what, and making group discussions more productive. The goal of this research is to design intelligent rooms that provide facilitation services by identifying meeting participants, understand their conversations, summarize discussions, and help the group efficiently get through an agenda, all without requiring the participants to wear microphones or other sensors. The research will have broader impacts in several aspects. Any steps to make group meetings for complex, long-term projects more productive and easier to control would result in immediate economic impact. The success of a service system that facilitates long-term group interactions will result in a major opportunity for technology transfer and a highly marketable hardware/software platform for collaboration in domains including business, education, and finance. The project will result in new group meeting data to be used by researchers in different fields such as organizational psychologists, and computer scientists. Finally, the project will produce highly visible infrastructure for research and education that has the potential for greater public engagement with science and technology. The research will be realized at different scales in two existing physical testbeds: the Smart Conference Room (SCR) at the Engineering Research Center for Lighting Enabled Systems and Applications (LESA) and the Collaborative Research Augmented Immersive Virtual Environment Lab (CRAIVE-Lab), a much larger space with a tall, panoramic screen. Both testbeds will be expanded as part of this project to integrate sensing and computing technologies developed by the partners. The major technology modules include: (1) advanced time-of-flight sensors for robust occupant tracking, resulting in centimeter-accurate, real-time locations of all participants in the meeting without requiring video cameras or wearable sensors; (2) custom beamforming microphone arrays for acoustical tracking and sound source separation, allowing highly-directional beams to be directed at each participant's instantaneous location, clearly isolating their speech without requiring lapel or headset microphones; (3) natural language understanding algorithms for extracting knowledge from the speech, enabling the system to learn meeting-specific terminology, link concepts, assess speaker roles, and summarize discussion; and (4) cognitive computing tools for meeting facilitation, including assessments of participation and productivity, ideation interventions for brainstorming, and active decision support. The resulting service systems will be cognitive physical environments that understand their occupants' locations, movement, speech, vocabulary, and intentions. A key aspect of the research is a multi-year study that tracks technical research groups that hold regular, unscripted meetings in the testbeds, assessing the effectiveness of the service system. The lead institution for the project is Rensselaer Polytechnic Institute, with investigators from the departments of Electrical, Computer, and Systems Engineering, Computer Science, and Architecture. The industrial partners in the effort are IBM Research (large business, Yorktown Heights, NY) and Heptagon Advanced MicroOptics (large business, Zurich, Switzerland). This award is partially supported by funds from the Directorate for Computer and Information Science and Engineering (CISE), Divisions of Information and Intelligent Systems (IIS) and Computer and Network Systems (CNS).