Prescriptive Models for Improving Subjective Probability Judgments

Duke University, Durham NC

Investigators

Abstract

Decision making in public policy as well as commercial settings is often complicated by disagreement among experts regarding estimates, forecasts, and even the nature of the problem under discussion. Moreover, it is often necessary to make a decision in the face of such disagreements; collecting the required "hard evidence" may simply be infeasible within a reasonable period of time. Thus, it is important to find ways to use the valuable knowledge that experts possess. Expert knowledge often is best characterized as "opinions" that can be expressed as probability judgments, the expert's assessment of the chance that something is true. Typical public-policy settings where expert probability judgments are used include energy (especially nuclear power and engineering), military, intelligence and security, environment and health policy, and economic forecasting. In commercial settings, expert judgments play large roles in overall strategic decision making and situations such as oil and gas exploration and pharmaceutical research and development. No simple answer exists for improving experts' probability judgments. Psychological research has documented many ways in which such judgments are biased. Because different psychological processes may be at work in different situations, methods to counteract the corresponding biases must be adapted to the specific situation. The proposed research offers three practical methods for improving experts' probabilities. All three methods are grounded in psychological theory and are designed to be applicable to a variety of probability-assessment situations. Each method will be subjected to rigorous evaluation to demonstrate its effectiveness. The three methods further provide models for how a scientist can use psychological results and theory for developing additional adjustment methods for improving expert probabilities.