III: Small: Active Learning of Language Models for Information Extraction

Northwestern University, Evanston IL

Investigators

Abstract

This project studies methods for extracting accurate knowledge bases from the Web. Fully-automated Web information extraction techniques are massively scalable, but have accuracy and coverage limitations. This proposal investigates how to improve automated extraction techniques by introducing carefully-selected human guidance. The proposed system continually extracts knowledge from the Web, along the way dynamically synthesizing and issuing queries to humans to increase the accuracy of the system's knowledge base and extractors. The approach extends the PI's previous work utilizing statistical language models (SLMs) for information extraction. Novel SLMs are investigated for unifying the extraction of relational data expressed in Web tables with extraction from free text. New active learning techniques utilize the models to identify "high-leverage" queries -- requesting, for example, textual extraction patterns that when retrieved from the Web yield thousands of novel extractions. The queries investigated are mostly amenable to non-experts, meaning that much of the human input can be acquired at scale via online mass-collaboration. The broader impact of this project lies in the potential for accurate Web extraction to radically improve Web search, allowing users to answer complicated questions by synthesizing information across multiple Web pages. In domains like medicine and biology, mining extracted knowledge bases could lead to important discoveries and novel therapies. Further information may be found at the project web page: http://wail.eecs.northwestern.edu/projects/activelms/index.html