Collaborative Research: Scaling Insight into Science: Assessing the value and effectiveness of machine assisted classification within a statistical system

University Of Chicago, Chicago IL

Investigators

Abstract

The project develops and compares cutting edge methods for the classification and analysis of American scientific research and compares them to existing manually generated approaches. The project examines the strengths and weaknesses of four computational approaches: topic models, network-based partitioning methods, Wikipedia-based labeling, and an active-learning approach. In particular, the research examines whether or not the different approaches can correctly classify established research areas and discover emerging fields across a broad range of disciplines. Each approach is evaluated based on a set of metrics measuring effectiveness, computational costs, human oversight costs, and the need to retain consistency with existing classification frameworks. The work directly responds to a number of National Academies recommendations and National Center for Science and Engineering Statistics (NCSES) reports that suggest using computational approaches to classify scientific research fields. A longer-term impact is improvement in data collection, processing, and reporting of key national statistics on science and engineering.