RI-Small: Learning to Generate High Quality Paraphrases with a Broad Coverage Lexicalized Grammar

Ohio State University Research Foundation -Do Not Use, Columbus OH

Investigators

Abstract

Automatic paraphrasing is considered vital to applications as diverse as machine translation (MT), question answering, summarization, and dialogue systems. Paraphrasing has also been shown recently to hold promise for automatic methods of evaluating MT, when the paraphrases are of sufficiently high quality. This project investigates novel methods for acquiring and generating such high quality paraphrases in order to automatically approximate the human translation error rate (HTER) metric for MT evaluation, where human annotators post-edit MT outputs into acceptable paraphrases of the reference translations. The project emphasizes the use of a linguistically informed, grammar-based parser and realizer for acquiring and generating paraphrases using disjunctive logical forms (DLFs), in sharp contrast to most recent work that relies entirely on shallow methods. Specifically, the project investigates methods of (1) engineering a broad coverage English grammar from the CCGbank, with semantic roles integrated from Propbank; (2) scaling up OpenCCG for efficient parsing and realization with this grammar, adapting supertagging and parse ranking methods for generation; (3) adapting and extending previous methods of acquiring paraphrases to work on DLFs; (4) generating high quality n-best paraphrases of one or more reference sentences; and (5) experimentally evaluating whether the automatically generated paraphrases can be used with current MT metrics to yield improved correlations with human judgments of translation quality. By providing a way to automatically approximate the HTER metric, the project will help drive future MT research. Additionally, by dramatically extending the realization capacity of OpenCCG, the project promises to benefit a wide range of NLP tasks where the breadth of target texts is of crucial importance.