Collaborative Reseach: Model-based Methods for Debiasing Individual Probability Assessments: Theory, Experiments, and Application to Mississippi River Delta Restoration

Duke University, Durham NC

Investigators

Abstract

Experts are often asked to provide judgments to inform both private sector and public policy decisions. Such judgments may be used alone or with scientific models to estimate the probability of events such as changes in energy markets, levels of future carbon dioxide emissions, global temperature change, or the number of hurricanes to make landfall in the United States. Expert judgments are essential because past data may either be unavailable or not directly relevant due to changing conditions. From psychological research, however, we know that when making such probability judgments, people use mental short-cuts, or heuristics. The heuristics skew how people express judgments, resulting in unintentional biases in probabilities that systematically distort an individual's stated probabilities. These cognitive biases are the focus of the research, rather than intentional biases in which expressed probabilities are deliberately distorted in order to game the system. Minimizing cognitive biases in expert probabilities is essential when the probabilities are inputs to scientific models or to decisions that must be made without waiting for perfect information. The objective of this research is to develop mathematical models and statistical procedures with which an analyst can estimate the degree of bias for an individual and thereby quantify adjustments that would eliminate those biases. The research focuses on three cognitive biases: overprecision, the tendency to be too sure that a particular event will occur; partition dependence, in which judged probabilities depend inappropriately on how the range of the uncertain variable is divided; and carryover, an ordering effect in which an individual?s stated probabilities may be affected by previous judgments. The bias measurement and debiasing methods are to be developed and tested in experimental settings using a large group of participants. The experimental results will show the extent of the biases under various circumstances, and the effectiveness of the method for removing the bias. This research will enable experts to provide probabilities that better represent their beliefs and knowledge, undistorted by bias, when engaged in public- and private-sector risk analyses. Potential applications include decisions in which data scarcity, coupled with high stakes, make the use of expert judgments essential. These include many areas of business decision making, as well as high-stakes policy decisions concerning, for instance, climate change and terrorism risk.