SBE-UKRI: From Manual to Automatic: Scaling Syntactic Annotation for Historical Language Change Studies

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

Annotated corpora -- texts marked up with information about grammatical structures -- are transforming how researchers study language change over time. However, the high cost of manual annotation has limited the size of these corpora and the research questions that can be asked. This project addresses this problem by building on and extending recent major advances in natural language processing (NLP) (a branch of artificial intelligence (AI)) to efficiently create very large collections (hundreds of millions of words spanning several centuries) of grammatically-analyzed text for two languages with rich written traditions. The availability of these new collections at scales previously not possible will enable linguists to make new discoveries about how languages change over time. It will also benefit researchers in other fields such as history, literature, and heritage and cultural studies, which in many cases currently rely on simple searches for individual words or sequences of words. Moreover, the AI techniques developed to carry out this work can also be applied to other languages with large amounts of unannotated text, with similar benefits to researchers in linguistics, history, and literature. The manually annotated resources developed for the project will be of great interest to NLP researchers. This project addresses a critical limitation in historical linguistics research: while many hypotheses about language change require tens or hundreds of millions of words to test rigorously, the high cost of manual annotation has restricted existing syntactically-annotated corpora (treebanks) to only 1-2 million words. The research team combines expertise in linguistics and AI (specifically, NLP) to create comprehensive treebanks for heritage languages through a two-stage process. First, smaller manually-corrected treebanks provide the foundation for training models for language modeling (masked language models) and constituency parsing (neural parsers) to be robust to the challenges of historical texts, such as orthographic variation, Optical Character Recognition errors, and inconsistent use of punctuation. Second, these models are used to automatically parse much larger, unseen historical corpora. The resulting annotated corpora enable investigations of language variation and the persistence of linguistic features during periods of language contact and change. This project develops novel NLP methods to recover empty categories and antecedent co-indexing, necessary for the desired use of the large treebanks. In addition to the benefits for linguistics and other fields, this project advances NLP work in two ways. First, the advances on modeling the challenging aspects of historical texts generalizes to other languages with large amounts of unannotated text. Second, the manually annotated treebanks developed for the project are new testbeds for work on historical texts, which have rarely been used for evaluation. This award is made possible through the NSF-UKRI lead agency opportunity. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.