III: Small: Models and Measures for Novel and Diverse Search Results

University Of Delaware, Newark DE

Investigators

Abstract

Traditional models of information retrieval assume documents are independently relevant. However, users often want to see the retrieved documents cover several subtopics, or different possible interpretations, or nuggets of relevant information for a query. In these cases, modeling documents as independently relevant does not provide the optimal user experience. This research project attempts to remedy this with new models of document interdependence and new evaluation measures. There are three threads running through this work: (1) the models of diversity, novelty, and redundancy that will be needed to implement ranking algorithms; (2) measurements of diversity, novelty, and redundancy in a ranking of documents; and (3) optimizing model structures and parameters to the measures. The main challenges addressed in this project include: how to model relevance along with interdependencies among search results for different tasks and domains; how to predict the degree of diversity; and how to evaluate the new models. The new models include set-based and pattern-based retrieval models, and the new measures include set-based labeling, list-based preference etc. The models will be evaluated with the existing measures as well as the new measures, and the new measures will be evaluated with in-depth statistical analysis. The result of this project includes a suite of models and measures that combine novelty and diversity with relevance for different domains, including biomedical search, legal search and Web search. The far-reaching potential impact of this project is an improvement in search engine utility across the board. The project addresses specific applications to biomedical search, legal or patent search, and web search reflect major domains in which search engines are important and in which improved models for novelty and diversity would directly improve the user experience. Project results, including publications and evaluation datasets, will be disseminated via the project website (http://ir.cis.udel.edu/monads).