Cleaning ICPSR's State Legislative Election Returns Data and an Application to the Study of Repeat Challengers

Florida State University, Tallahassee FL

Investigators

Abstract

Research on American legislative elections focuses predominantly on Congress, with far less attention given to state legislatures. This is unfortunate because the large number of state legislative elections, the variation across states in the character of legislative bodies and election rules, and the increased importance of states and state legislatures in policy making in the U.S. make state legislative elections especially well suited for testing theories about the nature of elections. Probably the greatest impediment to such research is the lack of readily accessible data about elections in the fifty states. The ICPSR data set, State Legislative Election Returns in the United States, 1968-1989 (Study #8907) would seem to overcome this obstacle, as it contains information about all state legislative general elections in the United States over a two decade period, as well as primary elections in sixteen southern and border states, and runoff elections in eight states. Part I of this data set includes an observation (N = 173,811) for each candidate running in any election during the period, and shows among other variables, the candidate's party affiliation, the number of votes (s)he -and other candidates -receive, whether (s)he is an incumbent, and the type of district in which the election is held. However, scholars have been unable to fully exploit these enormously valuable data because of significant problems in the way candidate names are recorded. For candidates for whom there are multiple observations, the name recorded frequently varies across observations (e.g., with and without a middle initial, with and without nicknames), a problem compounded by typographical errors. These errors make it impossible to reliably trace a particular legislator from one election to the next, thereby greatly limiting the utility of the information in the data set. Berry, Berkman and Schneiderman (2000) estimated that among incumbents running in general elections for whom there were more than one observation included in this data set, only 65% were identified by an identical name across all observations; 35% had at least one case in which their name was recorded differently. The authors developed various computer algorithms for identifying when two similar but different names likely indicate the same person. Relying on these algorithms and visual inspection of a listing of legislators' names, names were adjusted to achieve greater consistency, reducing the error rate from 35% to 6%. This project to extends the "cleaning" of names in Part I of the State Legislative Election Returns data set in several ways. The initial cleaning by Berry, Berkman and Schneiderman was restricted to incumbents running in general elections, which constitute only 25 percent of the observations in the data set; the new project includes all observations in the data set, i.e., candidates (winners and losers) competing in general, primary and runoff elections. Also, more sophisticated error detection algorithms -expected to lower the error rate from 6% to less than 2% -are employed. The ICPSR has agreed to release a new version of Part I of the Election Returns data set adding both an "adjusted name" variable and a dummy variable that identifies those observations for which the revised name differs from the original. To enhance the value of this data set to scholars, a companion data set reporting the values of numerous state-level variables for 1968-1989 is also created; it includes institutional characteristics of legislatures, election returns, and other political, socio-economic and demographic characteristics of states. Several election and state politics scholars have already expressed interest in doing research using the enhanced State Legislative Election Returns data set. Indeed, when this data set is successfully cleaned, it will permit students of state legislative elections to conduct research on a diverse range of topics, such the consequences of redistricting, partisan realignment in the South, the behavior and impact of minor party candidates, the determinants of electoral success of incumbents over the course of their careers, and the linkage between elections and the public policy choices of politicians. This project promises to enhance substantially our understanding of the topic and produce a dataset of value to numerous other scholars.