Development and Validation of Assessments for Industry-Valued Professional and Technical Learning Outcomes in Engineering Education

Ohio State University, The, Columbus OH

Investigators

Abstract

To educate engineering graduates of higher value to industry upon graduation, curriculum reform requires effective assessment of knowledge, skills and abilities (KSA) such as project management, performance on multidisciplinary teams, critical thinking, addressing open-ended problems, thinking entrepreneurially, communicating with diverse audiences, taking calculated risks, and evaluating business and technical trade-offs. This project focuses on KSA assessment in senior capstone design courses where students gain technical and professional skills in realistic project experiences. The project team is defining and prioritizing required professional skills; developing, testing, and validating assessments that measure individual student achievement in diverse settings; and disseminating proven assessments through workshops, conference presentations, and focused communication vehicles including the Capstone Design Conference and an Internet-based Capstone Design Hub. Validated assessment tools will apply to engineering capstone design courses of all disciplines, project types, team configurations, and student backgrounds. Instruments are being developed with input from and testing with diverse student populations, giving specific attention to female and minority populations. Psychometric analysis of the instruments will involve the use of Classical Test Theory (CTT) and Item Response Theory (IRT) psychometric analysis. Specifically, the Classical Test Theory will be used to analyze item properties (item difficulty and discrimination indices) as well as test properties (test reliability and discrimination indices). Results from the CTT analysis will allow the team to verify (a) whether items can collectively function well in testing what they are purported to test; and (b) if not, which items may warrant further inspection. A diverse community of practitioners is contributing to the ongoing refinement and implementation of the assessments in order for them to gain broad adoption, guide curriculum change, and benefit society through better educated engineers entering the global workforce.