Doctoral Dissertation Research in DRMS: Judgments of Answerers and the Answers They Give

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

A basic feature of human social life is asking and answering questions, a complex task that requires one to decide whom to ask, how to phrase the question, and how to interpret the answer and judge its worth. Despite the centrality of the activity, however, little is understood about the general principles involved in each step of the process, making it difficult to design better ways to help people ask questions and find the information they need. The problem becomes even more pressing in the information age: people ask more questions than ever before, on a wider range of topics, and they take those questions not only to friends and family, but also to strangers and acquaintances on the wider world of online bulletin boards and social media. This dissertation work addresses gaps in our understanding, using a combination of methods from psychology and computer science. Tools from machine learning and artificial intelligence will be used to analyze large-scale collections of questions and answers that people produce on online bulletin board systems and results will be used to build general theories that explain and predict how and when people find good answers to their questions. In laboratory experiments to test these theories, participants will be shown questions and answers with varying underlying properties, and the results will identify features that are particularly crucial for information gathering. The large-scale nature of the data science investigation affords the development of theories based on subtle patterns in the relationship between question and answer, while the laboratory work allows investigation into the causal nature of the problem: what, for the question-asker, makes an answer good? One of the most common ways people explore and learn about the world is by seeking information from others through asking questions. This behavior requires two kinds of judgments: (1) from whom one should seek answers and (2) how should one judge whether the answer satisfies the question? These two judgments are inter-dependent and require that people make decisions under uncertainty. This two-pronged investigation into how such judgments are made includes not only the lab-based experiments that are a traditional strength of social and decision sciences, but also a data science component that looks at how these decisions are made in the wild. This mix of methodologies will be used to examine the defining features of good answers and those who give them. The research focuses on questions and answers that are given in human-to-human linguistic dialogue. Techniques from data science and computational linguistics will be used to analyze collections of questions and answers produced on online discussion forums, ranging from requests for technical help on Stackexchange to more nuanced, social questions asked by parents on Mumsnet. Two key properties of the relationship between questions and answers—the extent to which they overlap, and the degree to which answers focus the solution space posed by the question—are hypothesized to correspond to how answers are evaluated. Controlled laboratory experiments will be used to elucidate preferences for those who answer questions. The types of preferences a questioner has for potential answerers has direct consequences for the relevance and quality of information received, and in turn affects the utility of subsequent answers and further decisions. A Bayesian framework is used to formalize the task facing the questioner: to infer what type of answerer an individual is, given their past answering behavior. Behavioral experiments will test key predictions of the model. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.