EAPSI: Identifying Relations between Computer-Generated and Manually Annotated Interpretations of Activities for Planning and Plan Recognition Tasks

Freedman Richard G, Amherst MA

Investigators

Abstract

While humans perceive activities using words and cluster similar activities by some properties, computers using unsupervised learning algorithms do not necessarily identify them the same way and thus generate different interpretations. The goal of this research is to develop an analogy between human and machine definitions of activities so that artificial intelligence planning and plan recognition methods do not need to be adjusted for each set of definitions. Usually, either humans define the activities in a way which is too vague for machines to make accurate computations or computers define activities such that people cannot understand the underlying reasoning. Developing an interpreter for each entity will not only smooth the interaction between users and devices when solving problems together, but also contribute to bridging the human-computer gap. This project will use techniques in the research areas of interest to Dr. Alex Fukunaga at the University of Tokyo who studies many facets of artificial intelligence, especially those regarding autonomous planning, search, and optimization. Extending prior research on unsupervised activity recognition using topic models, this work proposes a two-step approach consisting of constraint optimization and heuristic search. The first phase uses constraint optimization to align a computer's recognized activity sequence with a given human's annotation of the same sequence. Then the second phase similarly aligns a computer's recognized activity sequence with an annotation of the sequence derived by heuristic search over a human-defined hierarchical task network. To develop these methods, the project will include formalizing the constraint and search problems, coding their formulations, and testing results of the identified mappings between the two definition sets. This test will be performed by combining an unsupervised activity recognition method previously developed by the PI for machine-interpreted actions with a commonly used plan recognition method that uses human-perceived action representations. This NSF EAPSI award is funded in collaboration with the Japan Society for the Promotion of Science.