CAREER: Statistical Solutions for Survey Measurement Errors

William Marsh Rice University, Houston TX

Investigators

Abstract

This research develops new statistical theory, methodology, algorithms, and software to produce more robust survey estimates. The new methods address a central challenge in the field of survey methods: the widespread problem of survey‐taker satisficing. Satisficing respondents provide low-effort responses without sufficiently considering the survey questions. Improving survey estimates benefits a range of disciplines that rely heavily on surveys or experiments with survey-based measures, including the social sciences, public health, and education. The research assists the countless decision makers and organizations that rely on survey data to inform their decision making. The project’s education component invests in the next generation of survey researchers through short courses, software development, and summer research experiences for undergraduates. By forging methodological innovations and training the next generation of survey researchers, the project expands survey methods expertise and produces widely disseminated tools. This research draws on techniques from causal inference, econometrics, and convex optimization to develop new statistical estimators that are robust to measurement error caused by satisficing. While survey researchers have developed ad hoc techniques, such as attention checks, to try and exclude satisficing respondents from survey samples, these techniques are often detached from statistical theory. This research demonstrates the inadequacy of existing solutions and the necessity of integrating statistical theory and survey design to produce more robust survey estimates. In particular, the investigator works on three main objectives. First, the investigator develops new statistical theory and estimators for common survey data collection techniques that address satisficing though attention checks, panel data, and the like. Second, the investigator extends the new statistical framework to question experiments—where response options are permuted—and by doing so relaxes the strong assumptions needed for screeners or panel data to detect satisficing. Third, the investigator develops new tools to make experiments with survey-based measures more precise despite the presence of measurement error. The output of the research includes theory, methodology, algorithms, and software that can be deployed to estimate population-level attitudes and beliefs and to estimate causal effects on survey-based outcomes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.