CRII: IIS: III: Towards Conversational Search Systems for Math

University Of Southern Maine, Portland ME

Investigators

Abstract

This project aims to improve the effectiveness of math information retrieval by designing and developing MathMex, a conversational math search engine. Search Engines can often return extraneous or irrelevant information when queries for mathematical concepts are posed. Inherently, mathematical concepts usually have many aspects, layers, and nuances that search engine users fail to qualify appropriately. For instance, a search for 'complete graphs' may have various intentions, such as seeking definitions or exploring applications. This project aims to address this issue by developing MathMex, a conversational math search engine. With MathMex if a user's query is ambiguous, the system will engage in a dialogue, asking clarifying questions to better understand the user's intent and deliver more accurate search results. Additionally, MathMex will offer suggestions for related information, drawing from a diverse dataset comprising annotated mathematical resources, including proofs, applications, and definitions. New language processing approaches and algorithms for resolving ambiguities and making suggestion will be designed and tested as part of this research. Furthermore, the project will introduce innovative methods for automatically extracting relevant information and generating clarifying questions. The potential societal benefits of MathMex are particularly high for educators and students, who will gain access to a more efficient tool for finding information and solving math problems. Importantly, MathMex's approach will differ from existing models like ChatGPT, focusing on facilitating information retrieval rather than providing direct problem solutions, thus removing the risk of misinformation. Moreover, this project will contribute to the establishment of an Artificial Intelligence and Information Retrieval lab at the University of Southern Maine, helping further advancements in these critical fields. The research will involve four key steps: 1) developing a labeled mathematical dataset, 2) linking different information regarding mathematical concepts, 3) developing language processing techniques for resolving ambiguity for math, and 4) evaluating the proposed techniques. Initially, the team of researchers will focus on training BERT-based classifiers to effectively extract various types of information about mathematical concepts from extensive textual and formulaic sources. These classifiers will enable annotating text based on the category of information about the mathematical concept, such as definition, proof, or application. Subsequently, efforts will be directed towards developing models that establish connections between different mathematical concepts, drawing upon linked data sources such as Wikipedia and Math Stack Exchange. This phase will involve exploring techniques to quantify and measure the relatedness of mathematical concepts to enhance the comprehensiveness of search results. Additionally, the project will investigate the generation and utilization of clarifying questions in response to user queries, particularly when ambiguity is detected. This will entail the development of algorithms and strategies for dynamically generating relevant clarifying questions to refine the search process. Finally, comprehensive evaluations of the approach will be conducted using established math search test collections and a newly created dataset for conversational math search. Through these activities, the project aims to advance the state-of-the-art in mathematical information retrieval, with potential applications across various scientific disciplines, such as chemistry and physics. Moreover, the research will pave the way for further exploration into alternative data types, such as images and videos, in mathematical search tasks, as well as innovations in math problem-solving methodologies. The expected outcomes include improved retrieval performance, enhanced user satisfaction, and advancements in natural language processing and information retrieval. This research will answer questions on design considerations for conversational math search and will raise new research questions on using other data types such as images and video searches and improved math problem-solving. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.