NIST Assistance with NTP SR Automation

National Institute Of Environmental Health Sciences

Investigators

Abstract

NIEHS seeks advice from NIST in the areas of human language technology and natural language processing component evaluations that support the measurement of systems that automatically extract toxicology information from publications to support the complex human task of systematic review of literature. NIST is positioned to assist NIEHS building upon existing test and evaluation infrastructure through its Text Analysis Conference (TAC) program. NIST is coordinating the 2019 Systematic Review Information Extraction evaluation (SRIE 2019) task for NIEHS as part of the Retrieval Group?s Text Analysis Conference (TAC) program. This coordination includes advising NIEHS on developing annotation guidelines; advising NIEHS on dataset construction and distribution; writing guidelines for the evaluation task; developing scoring methods and supporting software; including the evaluation task as part of the TAC program and call for participation; accepting participant submissions in the evaluation; evaluating those submissions; and reporting results of the evaluation. NIST and NIH will design an evaluation task in this domain.