Doctoral Dissertation Research: Addressing Survey Nonresponse and Attrition in Probability-based Online Panels and Online Longitudinal Data Collections

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

This doctoral dissertation research project will address issues of survey nonresponse and attrition in probability-based online panels and online longitudinal data collections. Survey data are a major source of policy information in the United States. However, traditional methods of data collection, such as face-to-face and telephone interviews, face challenges in declining response rates and rising cost. Probability-based online panels have emerged as an attractive option due to their flexibility, convenience, speed, cost-effectiveness, and statistical efficiency. Despite these advantages, survey nonresponse and attrition in online panels remain significant challenges. Given the increasing use of online data for policy decision-making, it is essential to collect valid and accurate online population data. This study will rigorously evaluate the effectiveness of two interventions aimed at improving response rates, reducing attrition, and enhancing data quality in online panels across disciplines. All de-identified micro data, together with replication files and project documentation, will be made publicly available to the broader research community with minimum delay to encourage further analyses and utilization of the study's research products. The project will provide research opportunities for undergraduate students through a partnership with the Undergraduate Research Opportunity Program at the University of Michigan, Ann Arbor. As a Doctoral Dissertation Research Improvement award, support is provided to enable a promising student to establish a strong, independent research career. This project will conduct two online experiments to analyze the causal effects of survey burden and different incentive payment plans. The first experiment will be conducted within the Understanding American Study (UAS), a probability-based online panel representative of US adults aged 18 and above. A sample of 2,000 new respondents will be used to analyze the impact of survey burden on survey nonresponse and attrition in probability-based online panels during the first year of panel membership. UAS provides tablets and free broadband internet access to new respondents without Internet access. Respondents will be recruited via address-based sampling by drawing from post office delivery sequence files provided by a vendor. Participants in this survey burden experiment will be randomly assigned to either a low or high survey burden condition, and the intervention period will span 12 months. The low and high burden groups will receive 1 and 2 survey invitations per month, respectively. The second experiment will be conducted among a random sample of about 500 master’s students at the University of Michigan (U-M), Ann Arbor, campus. Participants in the U-M study will be randomly assigned to one of three incentive payment plans that vary in the timing of receipt of incentive payment. Participants will complete three online longitudinal well-being surveys over the course of the academic year 2023-2024. The project’s use of experimental data will address selectivity issues present in previous studies that used observational data. Additionally, the project will leverage available administrative records and longitudinal survey data for all respondents to explore treatment heterogeneous effects and assess selection bias. Findings from this project will provide empirical evidence to enhance the effectiveness of online panels in social science research and policy­making and inform future research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.