CAREER: An Integrated Framework for Controllable Text Generation

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). This CAREER project focuses on creating writing assistant systems that take text as input and generate a revision, that is improved while retaining the original meaning. For example, replacing complex words and grammar with simpler ones to make a sentence easier to understand. This will advance state-of-the-art technologies for analyzing the human editing process and automating document editing using machine learning methods. The proposed framework could be adapted to support many useful applications, including helping K-12 teachers prepare educational material for their students at an appropriate reading level, assisting STEM students to improve their scientific writing, and helping social media users to rewrite biased language into a neutral tone. It can also directly provide reading aids for children and people with low literacy or disabilities, as well as help the general public better understand complex policies and medical documents. Empowering students to read and write about science is important for stimulating interest in science careers and for supporting U.S. economic growth. This project will address long-standing challenges in the field of natural language generation, including the lack of interpretability and controllability in neural generation models, and the lack of task-specific training data and reliable evaluation methods. The new framework will consist of four major components, which include: (1) high-quality data construction for a novel application of scientific writing; (2) controllable neural generation models; (3) interactive annotation interfaces; and (4) a redesigned and more reliable evaluation methodology. We emphasize the closely integrated design of all four components. The generation model will learn fine-grained control over edits at the individual word and sentence levels from static text corpora, as well as from human feedback data collected through an interactive interface. To ensure data quality and analyze the complexities in the human editing process, we will exploit a non-trivial methodology that combines manual annotations with automatic models for aligning correspondent text fragments based on semantics, and for classifying the intents of the edits to make the text generation model more explainable. This framework will not only allow better interpretability when suggesting edits to users with an explanation but also support better personalization for varied user preferences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.