Assessing Cognitive Components of Design for Course-based Improvement

University Of Massachusetts, Dartmouth, North Dartmouth MA

Investigators

Abstract

This project is developing, implementing, and evaluating assessment tools and methods that support continuous improvement in student preparation in electrical and computer engineering design and related areas. Educational research has developed findings which suggest that engineering education would be improved by deliberately and carefully assessing important educational outcomes and building continuous improvement loops around courses and curricula. One of the most important goals of engineering education is preparing students to succeed in developing solutions to large-scale, ill-structured design problems that are typical of senior capstone design project courses and the professional world of engineering practice. The project is using the knowledge of expert functioning provided by cognitive science and instructional research to identify crucial aspects of the design process. It is developing multiple measures and gathering data to verify that these aspects are significant components of design skill. In broad terms, the components of expertise that are being addressed are declarative knowledge, procedural knowledge, and metacognitive processing knowledge. These are being illuminated by the development of quantitative ways to measure quality of design. Declarative knowledge (including "facts") tends to build and become refined with experience, and its structure tends to become organized in more abstract schemas with experience, allowing it to be applied confidently to a greater range of problems. Procedural knowledge is understanding how to address problems efficiently. Metacognitive processing knowledge is particularly significant for developing lifelong learning skills. It is developed as students learn to think broadly about how they are solving particular problems and seek better or more efficient approaches for future use. This is knowledge that requires planning, self-monitoring, and reflection. Its growth responds to deliberate efforts to develop it by instructors. The approach is being class tested in sophomore design courses and senior project design courses in two engineering areas, electronic and computer software design. Participating instructors are evaluating the assessments, writing improvement plans, and implementing these over a period of three years. The project is producing three distinctive products. First, it is producing assessments of various components of design skill, with reliability and validity findings. Second, it is evaluating the effectiveness of these assessment measures for use in course-based continuous improvement of student learning. Finally, the assessments of sophomore knowledge in the first year of the project is being used to predict their performance in their senior design courses.