EAGER: Unified categories for describing and quantifying scientific research

University Of California-Irvine, Irvine CA

Investigators

Abstract

This project develops state-of-the-art machine learning methods to describe and quantify scientific research. The particular approach taken is to develop new topic models that can learn underlying research categories across a wide variety of text data sources including NSF and NIH grant awards, scientific publications, and US patents. The important innovation is the building of the technology to permit feedback from domain experts and end users. The approach is potentially transformative in that it can potentially overcomes some of the known limitations of current topic modeling approaches by improving the quality and utility of topics across diverse data sources. The web-based tool displays and manipulates learned topics so that users can apply it to create comprehensive overviews of scientific funding, research, and production, including the answers to questions like: - What types of science are funded by NSF, NIH and other agencies? - What types of science are produced by funded investigators? - What types of science are described in US patents? The tool also provide users with answers to more complex questions about the science of science and the relationship between funding and scientific achievements, tracking trends, describing funding programs, and identifying funding overlap (across agencies, or even within agencies). Intellectual Merit: The proposed research advances techniques and methods for tracking scientific research in several ways. First, it advances the development of unsupervised statistical topic models to categorize, describe, and measure scientific research. Second, it addresses known problems with topic modeling for this type of application, such as improving the coherence of all topics, and making topics transcend different types of document collections (grants, publications, patents). Users can more directly measure impacts of funding as a result of being able to make use of unified topics from grants, publications, and patents. Third, the research develops evaluation frameworks that shift the focus from machine learning metrics to the needs of domain experts and end users. Broader Impacts: This work has an array of broader impacts. It creates useful data for funding agency staff, researchers, interested public, government bodies, media and other stakeholders. The web based tool allows users to create custom-based data sets, tailored to their particular needs. Such data sets allow users to answer an array of science of science policy questions. The knowledge created in this work supports initiatives such as STAR METRICS to document the value of investments in scientific research.