AF: EAGER: Data Streaming with a View towards Cloud Computing

Dartmouth College, Hanover NH

Investigators

Abstract

The ability to efficiently and effectively process big data is becoming central to modern life. Increasingly, much of this processing takes place "in the cloud", on large clusters of powerful servers, which provide computational power far exceeding those available on a weak client, e.g., an end-user's personal computer. Many big data problems require time-and-space-efficient processing of massive data streams generated almost continuously: examples include financial transactions, medical records, and scientific experimental data. Access to a cloud computing service makes tractable a number of data streaming problems that would otherwise be intractable for a weak client. Existing cloud computing services do not give clients a guarantee that their computations will be executed error-free. In the course of this project, the PI will work to design and study the theoretical foundations of techniques that would enable a weak client to trust results arrived at when working with such a service. New algorithms designed during this project could have an impact on the practice of cloud computing and data streaming systems. The project will enable the training of graduate and undergraduate students in theoretical computer science research and exposition, and in experimentally studying algorithmic ideas relevant to such trustworthy cloud computing. The project has two major threads. The first is about algorithm design, wherein the PI will seek new or improved algorithms (in terms of space usage and communication costs) in the above weak-client/powerful-server setting. In particular, the PI will revisit such fundamental problems as norm estimation and clustering. The second thread is complexity-theoretic: the PI will seek to understand the limitations of this computational setting, most likely through proving new lower bounds in communication complexity. This thread naturally leads to open questions about Arthur-Merlin communication, a topic long known to be difficult enough that even small advances could be significant breakthroughs.