User Interface Softbots

North Carolina State University, Raleigh NC

Investigators

Abstract

This project focuses on ibots, interface agents that interact with software applications through the graphical user interface, through the same medium as human users. The project will have two main thrusts. First, the PI will define a taxonomy of tool use in the user interface, based on descriptions of user behavior in a variety of common applications, and he will build a computational model of agent tool use that integrates work on visual routines and cost-based user modeling. An environment for agent exploration and evaluation will then be developed, to include sets of tools in three distinct task areas: support for the construction of domain descriptions and the incremental build/test cycle for agent development; support for the exploration of different control strategies in carrying out these tasks; and support for the evaluation of interface agent performance. As this environment is completed, the PI will evaluate it along two parallel tracks, as a mechanism allowing cognitive models to interact with off-the-shelf software, focusing on models in the ACT-R, Soar, and LICAI frameworks, and as a testbed for developing new problem domains for planning algorithms. This work will have an impact in several areas. It will allow cognitive modeling researchers to evaluate their models in actual (not simulated) environments, a long-standing goal for researchers in this area. It will provide a comparable benefit to planning researchers, who also need domains that are both real and tractable for their planning algorithms. It will produce novel interface agents and domain descriptions, with opportunities to test the adequacy of existing agent designs against real-world problems at a relatively low development cost; the project also has the potential to increase the visibility of interface agents research, by allowing agents to work in previously inaccessible areas. Finally, it will result in a model and implementation of agents specialized for tool use, an area of ecological research that remains unexamined to date, and which should contribute significantly to our understanding of complex software environments.