Anticipatory Survey Design: Identification and Reduction of Nonresponse Bias Dur

Research Triangle Institute, Durham NC

Investigators

Abstract

DESCRIPTION (provided by applicant): This project investigates the utility of a new strategy to address one of the most serious problems in inference from survey data: bias and variance in survey estimates resulting from low rates of survey participation. Much of health policy and health research relies on survey data from studies, such as the National Longitudinal Study of Adolescent Health (Add Health) funded by the National Institute of Child Health and Human Development (NICHD), yet a nonrandom part of the survey sample fail to respond. Such longitudinal studies suffer from attrition nonresponse while they also dispose of substantial amount of information on sample members. The potential for biased survey estimates presents an enormous problem for health researchers and policy makers alike. The novel approach to the nonresponse problem proposed in this project uses information available on all sample members, to focus on those who are most likely to contribute to greater nonresponse bias and greater standard errors for key survey estimates if they are not interviewed. Our method contrasts markedly with the typical nonresponse approach many survey managers adopt, which seeks to maximize overall survey response rates. Increasing overall response rates does not always decrease response bias in survey estimates. The method outlined here focuses on critical survey variables (y) and uses multiple sources of information to produce models that predict the likelihood of a case contributing disproportionately to nonresponse bias as well as to weighting variance that results from disproportionate nonresponse. After we empirically identify those cases most likely to contribute to nonresponse bias in key (y) variables, we introduce a more effective protocol to encourage these sample members'participation. We believe that this theoretically and empirically driven approach can improve the quality of survey estimates by dramatically reducing bias and variance due to nonresponse in key survey estimates, using existing data during data collection. This project will evaluate this new approach to survey nonresponse via an experiment in an upcoming data collection. A random sample of cases will be assigned to the new approach to addressing nonresponse, while the other half will be assigned to a typical approach of response rate maximization. The results from this experiment could provide analysts and methodologists in health and social sciences with a new method to address bias and variance impacts on estimates stemming from nonresponse in surveys. The proposed approach can help improve the estimates data users derive from NICHD survey data and could have benefits for the broader scientific community, as a method to reduce nonresponse bias in surveys at a time when survey nonresponse poses a serious, growing threat to information used for policy making and studying changes in populations. PUBLIC HEALTH RELEVANCE: Accurate survey data are critical to health research and the subsequent formation of sound health policy. Biased survey estimates present a complex problem for health researchers and policy makers requiring accurate data. This R21 project investigates the utility of a new solution aimed at ameliorating increased nonresponse bias and total nonresponse error due to declining participation in sample surveys.