CAREER: Machine Learning-Based Approaches Toward Combatting Abusive Behavior in Online Communities

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

This research aims to computationally model abusive online behavior to build tools that help counter it, with the goal of making the Internet a more welcoming place. Since its earliest days, flaming, trolling, harassment and abuse have plagued the Internet. This project will lay bare the structure of online abuse over many types of online conversations, a major step forward for the study of computer-mediated communication. This will result from modeling abuse with statistical machine learning algorithms as a function of theoretically inspired, sociolinguistic variables, and will entail new technical and methodological advances. This work will enable a transformative new class of automated and semi-automated applications that depend on computationally generated abuse predictions. The education and outreach plan is deeply tied to the research activities, and focuses on scaling-up the research's broader impacts. A public application programming interface (API) will enable developers and online community managers around the world to integrate into their own sites the defenses against abuse developed by this research. The work will consist of two major phases. In the first, the research will develop a deep understanding of abusive online behavior via statistical machine learning techniques. Specifically, the work will appropriate theories from social science and linguistics to inform the creation of features for robust statistical machine learning algorithms to predict abuse. These proposed abuse models will enable a brand new, transformative class of mixed-initiative artifacts capable of intervening in social media and online communities. In the second phase, this project will explore this newly enabled class of artifacts by building, deploying and evaluating sociotechnical tools for combatting abuse. Specifically, it will explore two classes of tools that use the abuse predictions: shields and moderator tools. The first, shields, will proactively block inbound abuse from reaching people. The second class of tools, moderator tools, will flag and triage abuse for community moderators.