Development of the Biology Lab Inventory of Critical Thinking for Ecology (Eco-BLIC)

Cornell University, Ithaca NY

Investigators

Abstract

With support from the NSF Improving Undergraduate STEM Education Program: Education and Human Resources (IUSE: EHR), this project aims to serve the national interest by developing a new assessment instrument, titled the Biology Lab Inventory of Critical thinking for Ecology or Eco-BLIC, that biology instructors can use to measure undergraduate students' critical thinking skills in the context of ecology data collected from field settings. Critical thinking is defined here as the evidence-based ways in which individuals make decisions about what to trust and what to do. While education research to date has largely focused on student learning from lectures and recitations, there is far less research on the critical thinking learning outcomes that are often stated goals of instructional laboratory and field courses. This unequal balance leaves many open research questions regarding what students are currently learning from these courses, what students could be learning, and how to measure such learning. Furthermore, ecology lab and field courses that help students engage in critical thinking through participation in research projects can be some of the most resource intensive components of a departmental curriculum. They often require specialized facilities, extensive coordination, and occupy a significant amount of student and instructional staff time. These logistical challenges and resources, along with little research to evaluate what critical thinking skills students are learning in these courses, have caused a decline in offerings of authentic laboratory and field ecology courses at the undergraduate level. With the Eco-BLIC, departments will be able to measure potential growth in students' critical thinking skills, which in turn can help them determine how to best argue for and use resources to enhance their ecology laboratory and field-based offerings. While there are related instruments, no single instrument effectively meets the needs of assessing experimentation-focused critical thinking in undergraduate ecology lab and field courses. The proposed Eco-BLIC instrument will allow the biology education research community to respond to calls to build evidence about laboratory and field course instruction, and to generate new knowledge about student core competencies and difficulties in critical thinking. The design of the Eco-BLIC will build from recent work to develop a Physics Lab Inventory of Critical thinking (PLIC). Based on the design of the PLIC, we will create experimental contexts and suites of questions that probe students' critical thinking skills about ecology field research data. The contexts and questions will be intended to assess a range of students from multiple institutions and will be research-validated following standard procedures, including comparing open- and closed-response versions, interviewing students, and getting feedback from experts. During the question development process, the expert feedback will check the clarity and accuracy of the Eco-BLIC questions. Expert answers will also be used to: 1) develop the scoring scheme, 2) evaluate discrimination of the instrument by comparing differences in undergraduate and expert critical thinking skills, and 3) measure pre/post changes in students' agreement with the expert responses. Pilot versions of Eco-BLIC will be given to students at multiple institution types and we will use a suite of statistical techniques (e.g., reliability coefficients for test-retest stability, item analysis, test analysis, factor and network analysis) to collect evidence of assessment validity and reliability, and to determine the extent to which the questions are able to measure outcomes in various student populations. We will also compare student answers on the PLIC versus the Eco-BLIC to identify and study themes about critical thinking that span multiple disciplines. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.