Semi-Parametric Factor Analysis for Item Responses and Response Times

University Of Maryland, College Park, College Park MD

Investigators

Abstract

This research project will develop a generic modeling framework for measuring latent traits, such as language proficiency and cognitive skills, from item response and response-time data. Response time has been identified as a useful indicator for test performance. The project will increase understanding the role of response time in complex test-taking behaviors, such as problem-solving strategies and the practice/fatigue effect. Research on the role of response time will benefit the practice of test/survey development and administration. The measurement model to be developed will rely on very few parametric assumptions, which will make it appropriate for analyzing a wide variety of computerized assessment/online survey data. The project will develop efficient algorithms for estimating the new semi-parametric model. Graduate students will be mentored, and open-source software will be developed to facilitate the dissemination of the research product. This project will formulate a general factor analysis model for categorical item responses and continuous response times. To achieve maximal generalizability, the distributions of item responses and response times conditional on the latent traits will be characterized by smooth probability mass/density functions and approximated by tensor-product regression splines with a functional analysis of variance parameterization. The investigators will develop and implement a stochastic expectation-maximization algorithm for the penalized maximum likelihood estimation of the proposed semi-parametric model. To avoid overfitting, they will examine the empirical selection of smoothing parameters by effective degrees of freedom and a mixed-effects model representation. The performance of these methods will be assessed via extensive Monte Carlo simulations. The 2015 Programme for International Student Assessment data and the Test of Relational Reasoning-junior data will be analyzed to demonstrate the flexibility of the proposed method. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.