CIF: Small: MoDL: Interpreting Deep-Learned Error-Correcting Codes

University Of Illinois At Chicago, Chicago IL

Investigators

Abstract

The theory and development of error-correcting codes for channel coding is a field with a long and distinguished history. Enormous progress has been made through mathematical and algorithmic breakthroughs toward developing codes that approach capacity. Powerful channel codes are crucial in ensuring reliable communication for our connected society and its technological innovations. Recently, a new paradigm has emerged in the development of error-correcting codes: learn, using machine learning, the encoders or decoders in an end-to-end fashion. While it has not been easy to outperform decades of work in coding theory, some of these codes excel at practical blocklengths, some are able to achieve similar performance while offering robustness benefits, and some are able to dramatically outperform existing codes in channels with feedback. Such advances are part of a larger picture: machine learning is playing an increasingly important role in various pieces of the communication stack, particularly in the wireless domain. While deep-learned codes are explicitly given by the specific trained neural networks (often with a very large number of trained parameters), they have been met with skepticism, some of which at least stems from these deep-learned error-correcting codes being viewed as "black boxes," with little so far beyond limited experiments to demonstrate when or why they perform well. Our research will help engineers understand, interpret, and trust the performance of deep-learned error-correcting codes. This project will ultimately help intelligently and carefully integrate them into the extensive body of theoretical research as well as practical systems. Further, it will advance the use of deep learning in other scientific fields by providing tools and algorithms that will enable the new machine-learned artifacts to be interpreted and related to existing formalisms. It is important to train our next generation of data-science engineers to critically interpret and understand what their systems learn: the proposed research will do so from a new, rigorous angle enabled by the quantitative nature of error-correcting codes, with precise metrics and accurate modeling of distributions and statistics. We present a transformative research program that builds upon promising preliminary work that will lay the foundation for interpreting deep-learned error-correcting codes in a structured, formal manner. This interdisciplinary line of work merges aspects of coding and information theory, explainable artificial intelligence, deep learning, computational learning theory, optimization, system identification, and Boolean function analysis towards the following ambitious goals: 1) developing algorithmic tools for the interpretation of deep-learned error-correcting codes, 2) using these tools to understand deep-learned error-correcting codes, 3) providing a formal characterization of deep-learned error-correcting codes based on the experimental results, 4) using this to design improved deep-learned error-correcting codes, and 5) using the insights to improve traditional codes. Results obtained are expected to provide important new tools for explainable artificial intelligence in general and will allow us to broaden the research program to deep-learned codes for the network and joint source-channel coding settings. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.