Robust Experimental Designs for Measuring Spillover Effects in Large Networked Populations

University Of Chicago, Chicago IL

Investigators

Abstract

This research project will investigate the role of networks in influencing the outcomes of interventions across various domains, such as education, marketing, and tax decisions. Traditional analytic approaches have depended heavily on statistical modeling assumptions. These models have limitations due to the complexity of relationships among individuals. This project will focus on randomized experiments that incorporate network considerations into the experimental design. New methodologies will be developed to study network effects and to design more effective interventions. The experimental methods will be helpful to stakeholders in economics, public practice, and social sciences, who are regularly faced with complex decision questions in networked populations. Open-source software will be developed to implement the new methods. The investigators will integrate the research program with graduate and undergraduate educational initiatives and outreach programs in STEM, particularly targeting underrepresented minorities. Given that the new methods will not rely on complex modeling assumptions, they should be a good resource for classroom use, enabling educators to effectively integrate the methods into courses related to economics and public practice. This research project will develop methods and software that enable robust randomization-based inference for various problems in estimation of network effects. Traditional methods tend to employ complex models that make strong assumptions about the network data dependencies and their asymptotics. Other experimental methods have employed clustering algorithms, but these are not easily scalable and often cannot generate clean and interpretable practical results. Randomization tests are finite-sample valid under minimal assumptions and can thus offer robust recommendations. Additionally, they can scale effectively to large-scale problems through simple permutation procedures. This project will employ such procedures both to understand the role of networks in shaping optimal decision design and to quantify the magnitude of spillover effects compared to the direct effects of decisions. Such analysis is critical because decisions might benefit certain individuals while inadvertently harming others. The new methods will aim to identify the key characteristics of networks that influence these spillover effects, such as whether they are positive or negative, localized or widespread across the network, or disproportionately affecting individuals from certain groups. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.