HCC-Medium: End-user debugging of machine-learned programs

Oregon State University, Corvallis OR

Investigators

Abstract

This is a project to give the end user some ability to debug programs that were written by a machine instead of a person, especially when the users are not expert programmers. This is the problem faced by users of a new sort of program, namely, one generated by a machine learning system. For example, intelligent user interfaces, categorizers of email and web sites, and recommender systems use machine learning to learn how to behave. This learned set of behaviors is a program. Learned programs do not come into existence until the learning environment has left the hands of the machine learning specialist, because they learn from the user's ongoing data. Thus, if these programs make a mistake, the only one present to debug them is the user. Giving end users the ability to debug such programs can improve the speed and accuracy of these systems. Specifically, the project envisions a fine-grained, iterative, interactive debugging process. First, a user notices an erroneous classification (with the system's help, based on reasoning about its own competence), such as an email message that might be misfiled. Second, the user asks for an explanation. Third, using the system's explanation, the user provides reasoning constraints, declaring, for example, that "today" is not an important word, and that anything from the company president should go into the "company" folder. The learned program reevaluates competence models and redoes its reasoning, giving the user an opportunity to immediately see the result of the change. The loop then begins again. Thus, the goals of this project are the following: 1. To help users identify reasoning problems, and to provide explanations of the behavior of machine-learned programs suitable for end users. 2. To elicit rich feedback from the user, incorporating it into the reasoning of the learned program. 3. To improve the speed and accuracy of machine learning by integrating this rich feedback into learning. In addition to the potential speed and accuracy improvement in the machine learner, users may become more productive and make fewer errors. Providing disclosure of the learned programs' reasoning engenders trust, and with it, increased willingness to use the system. Thus, this project has the potential to make significant advances in the user acceptance of machine learning in a variety of new, real-world applications. Combining human constraints and guidance with statistical learning could enable highly accurate learning from small data sets, which is critical to creating successful intelligent user interfaces. The project will also result in learning systems whose data sources and input features are easy to change and whose behavior is easy to control. In combining human-computer interaction principles with machine learning, this project opens opportunities for novel perspectives, especially in the realm of interdisciplinary education. Graduate students will be trained in this blended research area, and aspects of it will be incorporated in classes in both human-computer interaction and machine learning, and in other educational experiences for undergraduates and high school students.