Standard: Ethics-in-the-Making: Changing Practices in Data Science

Cornell University, Ithaca NY

Investigators

Abstract

A nontechnical description of the project, which explains the project's significance and importance. This award supports a research project that studies the phenomenon of ethics-in-the-making in data science and machine learning. Specifically, the project will focus on both research and development contexts where prediction and classification models are used to study risks, especially health and financial risks. The research team will provide an account of how ethics emerges in practice in these contexts, and it will identify points where different ways of envisioning and doing ethics diverge and converge. The analysis will shed light on the production, reception, and influence of the prediction and classification systems and research practices that express specific understandings of what it means to do ethical data science and machine learning projects. Understanding the dynamics of ethical change in prediction and classification research is of utmost importance given how fast these models are being woven into the fabric of society. Changes introduced via ethics-in-the-making can strengthen, weaken, or otherwise affect the moral and structural integrity of science and the political legitimacy of scientific institutions. The researchers propose to use qualitative research methods to investigate empirically how ethical standards and practices are being reconfigured in prediction and classification research and development. The research team will study the process of building ethically significant forms of accountability and control into prediction and classification research practices; and the team will examine the ways that prediction and classification researchers conceptualize ethical issues in their work. Of special interest is what practices and systems researchers consider ethically problematic or unproblematic, and why. The study will provide a timely basis for cultivating ethical practice in contemporary data science. Specifically, the project will document existing and potentially uncover new ethical challenges and opportunities; disseminate findings in academic journals; contribute to societal debate with perspective pieces in scientific journals; produce a white paper for data scientists and practitioners in academia and industry; and develop a set of pedagogical case studies intended to help students and professionals reflect on data science ethics. Finally, the findings will be incorporated into a curriculum development project, the 'Data Science & Society Lab,' which will be made available on an open access basis. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.