EAGER: Using Social Media and Crowdsourcing to Create a New Affect Dictionary

Columbia University, New York NY

Investigators

Abstract

Research on sentiment and emotion in textual and spoken language relies heavily on human annotation of corpora and human judgments of words in context to provide gold standard data and useful features for prediction. Current resources such as Whissell's Dictionary of Affect in Language (DAL) are of limited utility, due to their limited coverage. To provide richer gold standards, in recent years computational linguists have been making use se of crowd-sourcing websites such as Amazon Mechanical Turk to accomplish annotation tasks that previously would have been done by trained annotators or human subjects in laboratory experiments. Recently there has been some research on using social media websites for similar purposes (mining existing social exchanges for information and creating games to elicit new information). In this project the PI will explore the use of all three of these approaches to augment existing data on human judgments of the emotional content of lexical items to support her ongoing investigation into the classification of emotional text and speech (a major objective of which is to move beyond simple valence judgments relying upon acoustic and prosodic features by taking into account more nuanced aspects of affective text). She will examine the value of crowd-sourcing, social media mining and games implemented in social websites to ascertain what lessons can be learned about acquiring reliable annotations and judgments of emotional content. Broader Impacts: This exploratory research has the potential to open up new sources of information about the affective connotations of lexical items that will be invaluable to researchers working in text and in speech affect. Advances in the automatic identification of affect in text and in speech would have major applications in fields such as business and medicine, inter alia. Business interests in assessing consumer opinion is rapidly moving from focus groups to analyses of product reviews, while medical informatics researchers similarly try to learn patient attitudes to diagnoses, drugs, and treatments by mining online forums. A major component of such endeavors is the use of affect dictionaries. New and better sources of affective connotations of lexical items should prove enormously helpful in these efforts, increasing our ability to learn useful, practical information from the data individuals provide freely online. The lessons learned through these experiments and results of subsequent data collection will be made available to the larger research community in the form of new affect lexicons.