SBIR Phase II: Applying Semantic Paradata to Outcomes-aligned Assessment

Eduworks Corporation, Vancouver WA

Investigators

Abstract

This SBIR Phase II project will result in a cloud-based web application that uses natural language processing to automatically generate online assessment questions for any digital textual material and to align questions with topics and learning outcomes. These questions will be available for instructors to review, edit and use in online quizzes that students can take on desktops, laptops, tablets or smartphones. The application will grade the quizzes and analyze both individual and aggregate results. Instructors can enhance the analysis by tying errors to common student misconceptions. The application will provide instructors with near real-time updates on student learning and provide students with personalized and immediate feedback. The application will maintain a bank of questions that can be customized to an academic unit or institution and that will grow with use. The question bank and data collected by the system will be mined to improve performance over time. A beta release will be evaluated at multiple universities during Phase II. The broader/commercial impact of this work includes its potential to improve instruction for over 21 million American college and university students. Assessments play critical roles in higher education, but it currently takes considerable instructor time and effort to produce, grade and maintain them. Using the application developed in this Phase II SBIR, instructors will be able to generate and deliver quizzes quickly enough for ad-hoc in-class use and easily enough to integrate frequent practice and diagnostic tests into daily instruction. This will enable instructors to better evaluate students, keep students engaged, and tailor their instruction to student needs. Students will be able to take more practice tests, which evidence shows helps retain and recall class content, and administrators will obtain more data tied directly to student outcomes. Expected benefits include increased student success rates and richer data that universities and accreditation agencies can analyze to evaluate and improve programs. In addition, this Phase II SBIR will implement, test and improve techniques for automated question generation and outcomes alignment that can be used in other applications.