Resistances to Using Linear Models in Decision Making

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Parole boards must ask themselves a crucial question: should the prisoner before them be released, or is he or she likely to commit another crime? While statistical prediction rules (SPRs) have been shown to be the best way of combining information about such prisoners to predict recidivism, parole officers tend to rely on their own intuition for combining the information to make their decisions. College admissions officers make similar sorts of errors each year, by basing admissions on criteria such as one-on-one interviews, instead of statistical models that can predict success in college based on past performance, often incremented by tests that have been specifically designed to predict performance. Nothing predicts without any inaccuracy, but the conclusion that SPRs are more accurate than intuition is well established, by over 150 studies comparing the two prediction modes. The proposed project asks a related question, one with important public policy implications: Why do people believe that they can make better decisions than can statistical models? One hypothesis to be investigated is that while SPRs automatically specify how poorly they predict when they specify how well, people overestimate their own ability to predict outcomes and hence reject the statistical models. Moreover, such overestimation may be based in part at least on the cognitive factor that our belief in our ability to predict is partially based on our ability to "explain" ("fit") situations in retrospect ("I could have told you he'd kill again, given his behavior as a child"). By understanding these biases of overestimation and retrospection, we can begin to devise effective methods for convincing people to rely more on statistical prediction rules and less on their own judgment, thereby befitting clients to whom accuracy of prediction is important, often vital.