Collaborative Proposal: Model Knowledge and Scientific Judgment

Loyola University Of Chicago, Chicago IL

Investigators

Abstract

A central issue in science studies is the relationship between evidence and hypothesis. But science studies is not the only discipline that investigates this relationship. Science studies has too long neglected the half-century's worth of fascinating and important empirical literature we call Ameliorative Psychology. A central finding of Ameliorative Psychology is that statistical prediction rules (or SPRs) are generally more reliable (and usually significantly more reliable) than human experts when it comes to making predictions about practically important matters (Meehl 1954, Sawyer 1966, Dawes 1994). This literature offers effective reasoning strategies that have undergone repeated and successful empirical tests. This proposal is to develop and articulate a view of human knowledge and reasoning that takes Ameliorative Psychology seriously as an exemplification of reasoning excellence. It intends to extract from the literature what science has learned about good reasoning, and then apply those lessons back to individual reasoners and social institutions, including the institutions of science. The new position is defined as strategic reliabilism: Reasoning excellence involves the efficient allocation of cognitive resources to robustly reliable reasoning strategies. Strategic reliabilism is an explicitly cost-benefit-based epistemology for real, bounded knowers. Epistemology is the branch of philosophy concerned with knowledge and good reasoning. The proposal will liberate epistemology from the scholastic debates of analytic philosophy and treat it as a branch of the philosophy of science: Normative epistemology is to be properly understood and investigated as an essential, though often implicit, feature of cognitive science. The PIs will bring the skills and interests of the philosopher of science to epistemology by unearthing and clarifying the philosophical (in this case, epistemological) presuppositions of a particular branch of science (in this case, psychology). A novel theory about how best to reason about empirical matters, especially one that comes with a half-century's worth of empirical support, is bound to have broad and significant impact. Here are four such impacts: (1) The view of reasoning excellence we extract from Ameliorative Psychology naturally suggests a number of concrete useable lessons about how people can improve their reasoning about causation, diagnosis, and cost-benefit analysis (among other issues). The proposal's aim is to begin to employ the underutilized power of science to improve people's reasoning about important issues. (2) SPRs are currently being used by institutions to make some very important decisions that deeply affect people's lives (from medical diagnoses to credit decisions). The use of SPRs by social institutions is a significant (and largely ignored) way in which science is affecting society. The project will offer a clear articulation of the principles and assumptions underlying Ameliorative Psychology. (3) The view of reasoning excellence we extract from the Ameliorative Psychology can suggest alternative decision-procedures for social institutions. However, there are still many social institutions that have yet to heed the advice of the SPR literature, including diagnostics for mental illness, predicting violent behavior, and admissions to graduate and professional school. (4) The predictive models developed by Ameliorative Psychology are especially valuable for reasoning about complex phenomena about which one has many different lines of evidence. This is a perfectly accurate description of the task facing those who are charged with assessing science at various levels. Philosophers, historians and sociologists of science take the huge mass of data that is the history of science and try to draw general lessons about the methods and theories of science. Those scientists on grant review boards also make assessments about the promise of various projects by considering and weighing a number of different lines of evidence. It is ironic that those who assess science have not used methods that have the strongest scientific support. This is true for individuals extracting lessons from the history of science (Faust & Meehl 1992), and for scientific institutions rating scientific grant proposals (NIH Panel Report, 1996, especially section 5).