CCRI: Planning-C: Developing a Minecraft-based Testbed for Evaluating Human-AI Teaming Research

Arizona State University, Scottsdale AZ

Investigators

Abstract

Human AI Teaming (HAT) is an emerging and rapidly evolving field where physical and virtual spaces require rapid strategic and tactical decision-making. In the context of national security, it is imperative to develop tools that can accelerate what is known as the observation to action loop, enabling decision makers to take swift and effective action that can reduce the impacts of our adversarial threats. It is common for teams to operate in dynamic and uncertain environments and they will require not only cooperation between human and artificial agents, but the use of artificial intelligence (AI) to help structure those teams, create plans of actions, execute those plans, and adapt plans as the environment and goals change. This integration between human and artificial agents can be used to enhance the speed and accuracy of human decision makers; automate the generation, evaluation, and execution of mission plans; and improve training effectiveness. This CISE Computing Research Infrastructure (CCRI) planning grant will undertake an exhaustive community-centric requirements gathering, testing and finalization process that would result in a comprehensive set of features necessary to build a software suite comprising of tools that would improve the effectiveness of human AI teams. Specifically, the proposed software testbed would be flexible and customizable to meet the needs of a broad community of AI researchers and helps them accelerate the development of AI agents that are designed to work in Human AI teams. By organizing the AI community into multiple working groups and disseminating the results via a workshop, the project aims to develop the requirements for modifying the popular Minecraft gaming platform including the data engineering, testing and evaluation requirements for tools that would achieve the desired goals within the HAT community of researchers. The team builds upon prior success in using Minecraft for conducting HAT experiments in Urban Search and Rescue (USR) based settings and broaden its use among AI researchers who would be interested in developing and testing HAT experiments. The resulting community feedback would be integrated into a CCRI full proposal that aims to build the software suite based on the requirements arising from the activities conducted via the planning grant. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.