Collaborative Research: EAGER: Visual Prosody Annotation in a Sign Language Corpus

Rochester Institute Of Tech, Rochester NY

Investigators

Abstract

Linguists studying sign languages experience an immense resource gap. Resources for studying visual prosody in sign languages, and its grammatical and emotional functions, are scarce. This project contributes towards closing this gap and promotes data-driven sign language research. Housed in ideal research environments, the project aims to create a large sign language corpus, inclusive of dialogues, with annotations. The project plans to release this resource for linguistic and sign language technology research and provide open access teaching modules and assignments with instructor guides for use with the corpus. This project focuses on understudied characteristics in sign languages, whose study necessitates a new corpus resource, and on their reproducible annotation representations, using an iterative process of quality measurement of inter-annotator and intra-annotator agreement. The anticipated project outcomes include: (1) a sign language corpus that captures currently understudied characteristics, (2) a tested method for representing those characteristics in the corpus, (3) best practice guidelines for continued use, and (4) research dissemination in written manuscripts and video-recorded research products. Additionally, the team aims to train students and open pathways to increase the study of sign languages in the research workforce, preparing deaf scientists with linguistic research skills, and also to release a learning module for researchers. The new annotated corpus can help develop predictive models to reduce the time and resources required to carry out annotation and accelerate scientific insights, while promoting improvements to the state of the art in sign language analysis technology. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.