III: Medium: Collaborative Research: Extracting and Linking AI Artifacts

Temple University, Philadelphia PA

Investigators

Abstract

The goal of this project is to create a framework for linking all salient aspects of an artificial intelligence (AI) workflow, including data, AI models, AI tools, tasks, and training methodology. The investigators seek to create a framework that takes a holistic view of the AI workflow, and thus, will provide a solution to one of the three key problems identified in the Report of the Office of Science Roundtable on Data for AI: “Address open questions in AI with frameworks for relating data, models, and tasks.” One of the key provisions of federal funding agencies is the creation and open dissemination of research artifacts (e.g., data, models). Although publication-based knowledge is easily reused, data and models are not. Data are the key ingredients to generate AI models. However, the relation between an AI model and the data used to generate it or the task it solves, and the data on which the AI model is tested on, is captured by neither the model nor the data or task. Thus, the investigators seek to create a unified approach to construct this relationship and annotate it. This project will contribute to the broad field of information retrieval and, in particular, to the field of named entity recognition. In this project, the named entities are the datasets, AI models, developing tools, and the names of various methods, such as those employed in training. The investigators will employ a holistic approach to the management of AI research artifacts, i.e., paper-task-data-model-tool, which in turn will produce an innovative way to conceptualize and execute data-AI model search and aggregation. The technical innovation of this project is the creation of novel techniques for entity and relation extraction as well as for entity linking. The project will also contribute to the field of scientific literature mining. The investigators will create novel technology to automatically identify and catalog public AI data and models that increase their reusability. The key insight is that, without the research papers themselves, the research AI artifacts lack the necessary context for reuse. For example, papers describe the role of a dataset (e.g., training or testing) and tell if a model is original or used as a baseline. By automatically inferring task-data-model relations, this project will increase the ability of suggesting artifacts to a new undertaking, thus shortening the time for relevant artifact search. Educationally, this work will involve training of graduate and undergraduate students, particularly encouraging the participation of women and underrepresented groups in the research efforts, and curriculum development. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.