Effects of Interviewers, Respondents, and Questions on Survey Measurement

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

This research project will investigate how the quality of survey data is influenced by features of the survey interview. Survey data remains one of the most important ways that researchers learn about the behaviors and attitudes of populations. Using data and transcripts from a health survey, the investigators will examine the interplay between the characteristics of the survey questions, the behaviors of interviewers and respondents when questions are administered, and the socio-demographic attributes (for example, age, ethnicity, and gender) of both interviewers and respondents. The project will improve understanding of the issues involved in participation in medical research, questionnaire design, interviewing methods, and interviewer training. The results of this project will be of value to survey practitioners, federal data collection efforts, and social, behavioral, and medical researchers. Improving the quality of survey data benefits society by elevating the contribution that social science surveys make to the nation's data infrastructure. Results from this project will be disseminated in journals and at conferences. Materials produced by the project will be made available to practitioners and other researchers. Students will be trained as part of this project. The context for this research project is survey measurement, particularly as it applies in a diverse and multi-cultural setting like a health-related survey. The research will address the question: How do characteristics of interviewers, respondents, and survey questions independently and in combination impact survey quality? Building upon prior research, the investigators will examine how features of the survey interview may introduce systematic measurement error, bias comparisons across social groups of respondents, and lead to increased unreliability. The research will concentrate on two areas simultaneously: 1) a qualitative analysis identifying behaviors of interviewers and respondents and the interplay between them that produces a survey answer; and 2) a quantitative enumeration of these features of the interaction and their association with characteristics of interviewers, respondents, and survey questions. To facilitate the latter analyses, the investigators will develop and apply systematic coding to transcripts of standardized survey interviews. This coding will inform an examination of whether the behaviors of interviewers and respondents - especially behaviors other research finds are associated with lower validity or reliability - vary for respondents across groups (for example, racial/ethnic groups) and vary across characteristics of the survey questions. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.