Complex Adaptive Systems as a Model for Network Evaluations (CASNET)

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

Complexity theory as a theoretical framework for the evaluation of the Nanoscale Information Science Education Network (NISE Net) is the focus of this evaluation research project. The researchers include evaluation scholars at the University of Minnesota and research directors at the Science Museum of Minnesota, The Museum of Science in Boston, and the Oregon Museum of Science and Industry. The intent of the project is to study how complexity theory supports evaluation capacity building within the network of informal science organizations that makes up a national community of researchers and informal science educators whose goal is to improve public awareness of, engagement in and understanding of nanoscale science, engineering and technology. Components of complexity theory that are being studied include internal diversity and redundancy, the interactions among different individuals and organizations with the network, the extent to which the control of the network is distributed, the extent of a shared goal or identity operates throughout the system established by the network, and how the system responds to change in the external environment. This study employs a tiered case study of the way that primary actors within NISE Net are designing and implementing a new evaluation system to examine the programs that the institutions within the network are conducting. The first tier includes those individuals who are responsible for designing and implementing the resources used in the network to improve the public understanding of nanoscale science. The second tier are the individuals who use the resources developed by the network. The study utilizes a multiple case study methodology that examines individual cases that can be collected together to describe the operation of a complex network of individuals and organizations. NISE Net operates as a heterarchy whereby individuals are organized hierarchically within institutions and communicate horizontally across organizations. The case study methodology results in a framework of the evaluation efforts that will be studied iteratively using a design-based research approach to inform stakeholders and support changes in the framework as it evolves over the course of the study. Observations of the planning and implementation of evaluation efforts, document review and interviews with key stakeholders comprise the data collected and analyzed in this study. Complex evaluation efforts such as those being conducted by NISE Net require more than simple evaluation frameworks that examine inputs, activities, outputs and outcomes. Networks by their very nature are a system of stakeholders whose interactions quickly become complex and difficult to study. Complexity theory offers the conceptual organization by which complex systems can be studied and their evaluation efforts characterized. This project informs scholars in the evaluation field about models of evaluation that support examining the quality of work being conducted by complex systems such as NISE Net. The publications intended by this study have the potential to inform the evaluation community about how complexity theory can be brought to bear to study network systems in education.