CHS: Small: Collaborative Research: Measuring and Promoting the Quality of Online News Discussions

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

This project will amplify the efforts of people to bring out the best in other people in online conversations, and will make it easier for people to find high quality online conversations. There are numerous concerns about the tone and content of online conversations on public affairs at the present time. At its best, everyday online debate can lead people to consider alternative perspectives and even change their minds. This happens in environments where people may disagree, but where they try to inform and convince each other rather than simply yell at each other. The first goal of the research is to create automated classifiers to measure the quality of everyday online political talk. Classifiers will estimate the quality of online conversations about news articles in public venues such as Twitter, Facebook, Reddit, and the comments sections of news pages. A Conversation Finder tool (a website and a browser extension) will use the automated classifiers to recommend, in real time, venues where particular news articles are being discussed and where the quality scores are high. The second goal of the research is to create a Conversation Coach that helps the general public to improve the quality of conversation spaces they participate in, by helping them craft messages that directly contribute to quality and that indirectly inspire others. It will include a Message Assistant that extracts elements from conversations in order to help people craft messages and a Message Impact Assessor that predicts the likely impact of a draft message on the quality metrics for subsequent conversations. Quality of online conversations will be measured in terms of a variety of dimensions that communication scholars have articulated as desirable. Training data for the classifiers will be collected from conversation participants in addition to trained coders, and experiments will be conducted to determine the most effective sequence of requests to make of conversation participants in order to maximize motivation to contribute. Creation of the Conversation Recommender will lead to several intellectual contributions, including: (1) developing computational assists that help human raters achieve high inter-rater reliability; (2) identifying methods to motivate conversation participants to act as raters; (3) architecting neural-network based classifiers that achieve high prediction accuracy when trained using the collected ratings as training data; (4) developing techniques to make the classifiers produce interpretable results (explanations). Creation of the Conversation Coach will lead to two intellectual contributions: (1) identifying parts of conversations that can be automatically extracted and that writers find relevant and useful when composing messages; (2) architecting a predictive model that accurately estimates the impact of messages on subsequent conversation quality.