Collaborative Research: SHF: Small: Architecture Innovations for Enabling Simultaneous Translation at the Edge

Oregon State University, Corvallis OR

Investigators

Abstract

Simultaneous translation, which begins translating after just a few words are spoken, has significant real-world value that may disrupt and benefit a wide range of domains, such as military personnel deployed in foreign countries, businesspeople participating in multilingual meetings, medical service providers, law enforcement, customer support services, diplomats and political representatives, and countless international tourists. Currently, accurate real-time simultaneous translation is only possible by expensive, specially trained human interpreters or by running machine learning-based algorithms on server-grade graphics processing units (GPUs); both options are impractical for extensive and ubiquitous deployment of simultaneous translation in edge devices. Innovative domain-specific architecture needs to be designed that can reduce computation requirements by orders of magnitude while maintaining translation accuracy, thus enabling simultaneous translation at the edge. This research investigates the challenges and opportunities in designing hardware accelerators for transformer-based simultaneous translation. The objective is to utilize the unique characteristics of transformer models and the distinctive behaviors of simultaneous translation to develop cross-cutting solutions that meet the accuracy, latency, power, and resource efficiency goals. Among some of the specific lines of research that are explored include input-centric dataflow and data reuse that take advantage of extensive input data sharing, compute-proportional architecture that aims to achieve linearly scalable attention calculation, utility-driven linear transformation architecture that efficiently reduce dimensionality based on their actual utility, and customized routerless on-chip interconnects that provide scalable, flexible and ultra-low cost interconnects for simultaneous translation accelerators. Beyond specific technical contributions that advance the fields of computer architecture and natural language processing, this project also impacts more broadly on research, education, and outreach. Findings from this research are incorporated into graduate curricula, courses, and undergraduate research experiences. Various outreach activities have been planned to broaden inclusion and participation of diverse populations in the educational and societal impacts of this project. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.