EAGER: Collaborative Research: A Computational Model for Evaluating the Quality of Citizen Science Contributions

University Of Maryland, College Park, College Park MD

Investigators

Abstract

Citizen science is a form of research collaboration that involves members of the public in scientific projects, bringing multiple voices and ideas to problem solving and community participation. Citizen science can be powerful because while specific individuals may lack formal expertise and be limited in their ability to contribute high-quality data and new directions, a crowd of individuals may collectively possess the expertise and creativity necessary for identifying and solving difficult problems. However, a major concern for collecting scientific data from the crowd is the varied quality of the contributed data and their relevance to scientific hypotheses. The PIs will explore the potential for deriving metrics from research on computational creativity to automatically assess the quality of citizen science data as a complement to existing research on human assessment of data quality. The project will also explore how the automated assessment of quality can be incorporated into an agent that makes suggestions to individuals in the crowd about the quality of their data, resulting in a prototype for a computational agent that measures the novelty and value of a cizen science contribution. This project will inform future research in computational agents that learn from and contribute to the crowd in order to address challenges associated with the quality of the data and ideas from crowdsourcing in citizen science. More specifically, the project includes a) development of a model of citizen-science-data quality based on the notion that good contributions are not just reliable and accurate but also novel and surprising; b) an evaluation of the model against citizen-science data that has been labeled by humans for quality; and c) initial studies of the effect of computational quality feedback on the behavior and perception of members of the crowd. Extending quality assessment to include creativity and being able to make such assessments automatically is potentially tranformative for citizen science projects. The PIs will demonstrate their agent-based model of quality in citizen science projects including their own NatureNet project, which involves crowd participants in data collection in nature preserves and also in the design of scientific challenges and interaction experience that facilitate data collection.