Scaling the Peer Review Process for National Stem Education Digital Library Collections

California State University, Trustees, Long Beach CA

Investigators

Abstract

One of the primary goals of STEM digital libraries, as well as the NSDL itself, is to enable users to identify potentially useful digital learning materials and information from a wide range of collections. Faculty, instructors and student users of STEM digital libraries are often presented with numerous results from a search query where they then face the challenge of searching for only those that serve their educational purposes. STEM undergraduate educators expect and need digital libraries to provide them with the information and tools to evaluate the quality of content of learning materials, their potential efficacy in teaching and learning, their usability and stability, and the adaptability of these materials for their educational needs. How best to develop, implement and scale up effective, efficient and reliable peer review systems plagues both mature and newly developing STEM digital libraries. Differences in what is reviewed (journal articles vs. learning materials) and the purpose of the materials (advancing knowledge vs. promoting learning) help compound the situation. Another factor affecting peer review processes for digital libraries stems from users. From their standpoint, the ultimate goal of peer review is to help them select the learning materials. In this case, they are not interested in advancing knowledge in a particular field or helping authors improve their articles (the purpose of peer review for online journals). Effective systems require STEM digital collections to expend significant resources to recruit, train and support large numbers of reviewers who evaluate materials by applying a set of criteria that requires expertise in the integration of technology with pedagogy, cognitive science and content. In collaboration with its partners, NEEDS (a Digital Library for Engineering Education, and HEAL) the Health Education Access Library, this project is developing one solution to the problems associated with implementing and scaling peer review systems for NSDL collections, namely, the high costs associated with face-to-face training, retention of reviewers, and the inability of reviewers to keep pace with the rapidly increasing size of the collections. This work builds on the foundation of the MERLOT peer review process. To encourage adaptability by other NSDL collections there is an adaptable online professional development processes for implementing both peer and Premier reviews that is designed as a channel adaptable to the framework uPortal (the software selected by the NSDL-Core Integration development team to create the initial NSDL Main Portal). The main outcome of this collaborative project is a professional development module for training peer reviewers, designed for NSDL collections (and others) to effectively and efficiently implement and sustain peer review of their collections. The adaptable module, training and mentoring processes save NSDL collections significant time, effort and funds in their implementation of valid and reliable peer review processes.