Forecasting and Political Discourse

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

IARPA (Intelligence Advanced Research Projects Activity) recently sponsored a series of geopolitical forecasting tournaments to discover the most accurate possible methods for crowd-sourcing and aggregating probabilistic predictions of geopolitical events. Our group, the Good Judgment Project, competed and won the tournament each year from 2011 to 2015. In a 2015 study based on data from this project, Mellers, Arkes, and Chen examined changes in actively open-minded thinking (i.e., the tendency to look for evidence that contradicts one's favored views) between the time participants entered and exited the tournament. Forecasters were significantly more open-minded at the end of the tournament. Even opinionated forecasters appeared to become more circumspect when making judgments that were publicly scored for accuracy. These incentives may have encouraged self-critical and flexible patterns of thinking because the quest for accuracy may have trumped the desire to maintain partisan positions. The boost in open-mindedness (perhaps driven by the self-discipline needed to be accurate) may well be the most important discovery of the IARPA tournaments. This project moves this research forward with designs for new tournaments to pry open closed minds still further. The research team examines three behavioral interventions that might increase open-mindedness and reduce polarization that characterizes many political debates. The discovery of methods to change opinions and cognitive styles may reduce divisiveness and ultimately improve decision making in a world that seems more fractured and divided than ever. This project explores whether forecasting tournaments can be designed to increase open-mindedness. The research team uses an existing public tournament to explore three factors. The first is accountability to a heterogeneous forecasting team with variability in political orientations. The previous IARPA tournaments showed that team forecasts were more accurate than independent forecasts. The researchers, however, never explored team composition, and this factor may be critical. The second factor is group pre-commitment to cognitive flexibility. Before the outcomes of events are known, forecasters will state whether they would be willing to change their minds about various debates depending on the outcomes. In "before" surveys, participants will be asked: If outcome A occurs (or does not occur), how much should your political party (or the other party) change its mind about issue X, "After" surveys will measure changes in cognitive styles and political views. The third factor is perspective taking. When forecasting questions are released, the researchers will ask liberals to write a paragraph to express the conservative position, and vice versa. Participants will be told that their work will be evaluated based on how well it captures the essence of the other side's position. The research team will use Pre and post measures of open-mindedness and political viewpoints to evaluate the effectiveness and changes that occur from the interventions.