Cultural Variability in Survey Question Processing and Response Behaviors

University Of Illinois At Chicago, Chicago IL

Investigators

Abstract

Although considerable evidence suggests that cultural, ethnic, and racial background may have important effects on survey measurement error, the processes by which culture influences the survey respondents' behavior remain largely unexplored. To address this important problem, the study has four objectives. These include (1) investigating the effects of respondent race/ethnicity and culture on respondent verbal and nonverbal behaviors as indicators of response processing difficulties in survey interviews, (2) the effects of race/ethnicity on survey response styles, (3) the degree to which the effects of race/ethnicity on cognitive processing is moderated by question design features such as topic and format, and (4) the degree to which race/ethnicity effects can be accounted for by individual differences in cultural value orientations. Questions from major NSF-funded surveys, including the General Social Survey, the National Election Survey, and other major national social surveys, will be used to examine these objectives. Survey interviews will be conducted in the Chicago metropolitan area with a total sample of 800 adults from four distinct cultural groups: African Americans, Korean Americans, Mexican Americans and non-Hispanic Whites. Interviews will be conducted in English, Korean, and Spanish. Interviews will be audio and video recorded and subsequently behavior coded to identify verbal and nonverbal markers of cognitive processing difficulties, including problems with question comprehension, memory retrieval, response mapping, and socially desirable responding. The response latencies associated with each answer also will be electronically recorded. Hierarchical linear modeling will be used to analyze these data and address each study objective. As the U.S. ethnic and racial structure continues to diversify, understanding how this demographic transition may complicate the interpretation of survey results will become an increasing priority. This research will contribute to the understanding of the mental process respondents engage in as they answer survey questions. The study also will provide crucial insight into racial/ethnic differences in these processes. It will offer insights about the nature of racial/ ethnic variability in question comprehension across a variety of common types of survey questions, as well as racial/ethnic variability in response styles. It will address the role of race/ethnicity through the lens of cultural values, unpacking known racial/ethnic differences in extreme response style, acquiescence, socially desirable responding, and other such differences by linking them to cultural value orientations. The study also may lead to recommendations regarding best practices for the design of survey questions that minimize cultural differences in comprehension and response styles. The research is supported by the Methodology, Measurement, and Statistics Program, the Political Science Program, the Sociology Program, and a consortium of federal statistical agencies as part of a joint activity to support research on survey and statistical methodology.