CAREER: Scientist-in-the-loop-software testing

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Many scientists write programs every day for their work, for example, to analyze their data, automate their labs, and support collaboration. If these programs have faults, then resulting scientific results can be invalid. For example, a program fault may cause scientists to think something is true when it is not. This can lead to major theoretical or policy shifts. Scientists often learn to write software on the job, but they are rarely taught to write tests. Therefore, scientists do not follow common testing practices in professional software development. This project aims to combat faults in scientific programs by empowering scientists to efficiently create effective suites of test cases. The main thrust of the project is to conduct a set of studies to determine the underlying issues for testing in the realm of scientific programming. The new knowledge will be used to create tools that assist scientific programmers to generate suitable test suites for their software. The project integrates the research and education activities by creating new courses/curriculum directed at non-Computer Science graduate students as well as programming courses for scientific programmers that incorporate the software testing tools into the courses and continue to evaluate research hypotheses. The goal of this proposal is to help scientists write and use tests by automating away many of the challenges they currently face. The project will design and implement tailored test generation tools for scientific programmers. These tools will build on the latest innovations in program synthesis, program analysis, and testing research to support scientists in their construction of test suites. The project will design effective interaction methods for scientists to use the tools naturally and productively. The testing tool innovations will be used in education and training. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.