CAREER: Improving the Reliability of Human-Centered Secure-Development Research

University Of Maryland, College Park, College Park MD

Investigators

Abstract

Improving software security is a critical need for the U.S. and the world. Despite significant technical advances in software security, insecure software remains a common problem, sometimes with disastrous results. Solving this problem will require understanding how human decision-making interacts with technology in the process of secure software development. However, studying these human factors is typically expensive, time-consuming and difficult, for several reasons: professional developers are a small and hard-to-reach study population, professional software development is a complex task that can be hard to mimic in a study environment, and security is, despite its criticality, often a secondary goal that can be difficult to study directly. Researchers attempting to conduct such studies must make many choices about experimental design while balancing complexity, time and cost constraints, and validity or usefulness of expected results. Unfortunately, there is little evidence-based guidance as to how best to make these choices. By conducting a variety of experiments directly comparing the effects of different experimental design choices on studies of human-centered secure development, this project will help future researchers design better experiments and deploy their resources as effectively as possible. This project will improve the validity and reliability of developer-centered security research by empirically establishing best practices and tradeoffs, building on best practices from the usable security and empirical software engineering communities. Researchers will undertake a series of methodological studies and experiments in three key areas: (a) how to design appropriate programming tasks; (b) how to choose a study environment that effectively balances experimental control with ecological validity; and (c) how to measure relevant outcomes such as developer self-efficacy and API usability. Investigators will test these critical questions of study design across multiple underlying research questions, such as comparisons of APIs, documentation resources, and security tools. The results will provide deep insights into how tradeoffs among design decisions play out in different kinds of studies, allowing researchers to make informed choices that fit best in their context. The results will be synthesized into comprehensive guidelines to help researchers conduct better studies, acquire stronger evidence, and therefore improve the process of secure development. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.