SHF: SMALL: Effective and Equitable Technical Interviews in Software Engineering

North Carolina State University, Raleigh NC

Investigators

Abstract

In the software industry, a software engineer must pass a series of technical evaluations, called technical interviews, when applying for a job. A popular form of the technical interview requires software engineers to write computer code on the spot in front of interviewers. There are industry-wide concerns that such interviews may lack real-world relevance and be ineffective at evaluating technical competencies, while introducing systemic bias and unnecessary stress. This project aims to develop technical evaluations of software-engineering proficiency that are more equitable. The main objective is to enable broadening participation in the software-engineering workforce. To achieve this objective, the research investigates software-engineering processes involving human judgment of code, such as during code reviews. This project will develop techniques to measure the technical performance of a programming task and identify factors that negatively impact the ability to write code, such as high cognitive load, poor affective state, and limited affordances. Three main research tasks will be investigated for this project. The first task will involve data mining technical-interview experiences and engagement with stakeholders in order to determine a comprehensive list of factors concerning technical evaluation of code. The second task will involve a series of randomized, controlled trials to establish the effects of various conditions, such as problem-solving complexity, use of concurrent think-aloud, and presence of stereotype threat on technical performance. The third task will involve the development of interventions that reduce unwanted effects on technical performance, such as retrospective think-aloud or private interviews, and will include evaluating trials in industry. Overall, this project connects to the stream of research on identifying opportunities for increasing equity in software engineering and information work: improving diversity and tenure of teams and providing resources for disadvantaged and low-resource job seekers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.