CAREER: Signals for evaluating the credibility of web sources and advancing web literacy

Wellesley College, Wellesley Hills MA

Investigators

Abstract

This project will experimentally test the hypothesis that when users are presented with search results that are augmented with signals for evaluating the credibility of web sources, they will strengthen their critical thinking skills and become better consumers of online information. Evaluating the credibility of web sources is a crucial critical thinking skill in our digitally mediated world. We have learned to rely on search engine platforms to show us the right results, while forgetting that algorithms are biased or that they can be manipulated. This research aims to discover whether and how humans can reclaim their agency when deciding what information sources to trust. Additionally, it will: (1) identify a set of human-understandable signals that are deemed helpful for evaluating the credibility of web sources and validate them through users studies; (2) propose and implement algorithmic techniques for computing some of these signals, providing a trail of transparency about how they work, and (3) develop a novel web platform for the interactive exploration of signals, modeled upon nutrition fact labels, that will contribute in advancing web literacy skills in the broad public. Experimental evidence on whether labeling information sources with credibility signals affects the critical thinking skills of young people will inform the emerging public debate on how cyber-human systems such as search engines, which are shaping our minds and societies, could support a basic consumer right, that of being informed about the source of information that is being consumed. Other broad impacts are: (1) advancing web literacy through an interactive web platform that explains and gives information about signals to evaluate credibility; (2) monitoring search engine result pages (SERP), by establishing a SERP Observatory on results about vulnerable populations to discover attempts to manipulate the ranking, and (3) educating the public through writings on algorithmic decision making on the web. This research will also involve the entire community of the college in the production of digital content through a yearly Wikipedia Edit-a-thon, which will enrich and expand the results shown by search engines on topics about women, gender, and vulnerable populations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.