Improved Human-Computer Interaction for Design of Complex Systems

Cornell University, Ithaca NY

Investigators

Abstract

Future design of complex systems, for example, the design of aircraft, buildings, and transportation systems, will be done collaboratively by mixed teams of humans and artificial intelligence (AI) systems. The success of this collaboration depends on the degree to which the humans and the AI systems can effectively communicate with each other their goals, intentions and plans of actions. This award seeks to improve how humans and AI systems communicate when designing such systems. In particular, the investigators will give intelligent design agents the ability to provide explanations of their actions and suggestions to humans through both verbal and non-verbal communication channels. It also investigates how the AI system can use robotic body language to improve the collaborative design process. This has potential to have an impact on many aspects of our society which engage in system design, including architecture, medicine, urban planning, industrial design, and business management. Additionally, the research project provides opportunities for education and outreach during its execution. The research objective of this award is to enable mixed-initiative human-computer design of complex systems by giving intelligent design agents self-explaining abilities, modeling human-computer joint design as a collaborative activity, and leveraging the use of non-verbal channels and embodied interaction to improve human-machine communication in design. Starting from a model of design tools as intelligent agents, and based on the knowledge generated by the Human-Robot Collaboration literature, we will lay out the foundations of a new paradigm for engineering design based on mixed-initiative human-agent collaboration. The four pillars of this new paradigm are: (a) the coordination and meshing of shared plans and intentions between humans and design agents, and their resolution into individual agent plans and actions; (b) the dynamic allocation of roles and gradation of autonomy; (c) the reasoning about, generation, and maintenance of shared attention and common ground; and (d) the integration of verbal and nonverbal channels in communication about agent beliefs, intentions, and goals. Controlled experiments with human subjects will be used to test the effectiveness of the new framework and algorithms. If successful, this research can radically improve the quality of the designs and reduce the resources spent during the design process of complex engineering systems of national importance and high value to society such as systems-of-systems for weather forecasting, climate monitoring, disaster relief, or intelligence, surveillance, and reconnaissance. This research will also have broader impacts into areas outside of engineering that engage in system design activities, including architecture, medicine, urban planning, industrial design, and business management.