CAREER: Foundations of Information Theory: Information Inequalities and Dimension-Free Phenomena

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Information inequalities stemming from entropy and mutual information form the basis for Shannon's mathematical theory of communication, data transmission and storage, the vast consequences of which have ushered in the modern information age. While originally bounding theoretically the rate at which data may be transmitted over an imperfect channel, or how far an information source may be compressed, the foundational nature of these inequalities has led to their widespread applications in quantitative fields ranging from computer science to physics. This may be attributed, in part, to these inequalities being dimension-free, that is, the sharpness of the estimate does not degrade with the data dimension, thus making them suitable for analysis and inference in high-dimensional settings that are characteristic of modern problems in statistics, optimization and data science. Broadly speaking, this project seeks to further elucidate these foundational underpinnings of information theory, and to extend their applicability to modern problems in statistics and data analysis that seek to uncover information from large data sets. The research is coupled with a plan to integrate research and education at multiple levels: the project will train researchers at the interface of statistics, information theory and mathematics, preparing them to enter academic and industrial careers in data science, and skillfully adapt to new fields as national priorities change. Other aims are to promote collaboration within the broader research community through development of thematic workshops and tutorials. This project will undertake a systematic investigation of information inequalities. This goal will be achieved through an integrated research agenda that seeks to: (i) quantify high-dimensional statistical phenomena; (ii) characterize extremal properties of information inequalities; and (iii) discover new concentration and isoperimetric phenomena through linking information- and transportation-based quantities on graphs and other discrete spaces. Through these aims, this project will promote the influence of information theory across traditional boundaries and enrich the set of intellectual questions addressed. Given the intrinsic importance of quantifying statistical and probabilistic phenomena throughout science and engineering, the results will have broad and lasting impact. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.