CCF: Medium: Enabling Real-Time Quantitative Decision Making over Streaming Data

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

A key component of an effective Internet of Things (IoT) system is the ability to make decisions in real-time in response to data it receives. While the exact logic for making decisions in different applications requires domain-specific insights, it typically relies on computing quantitative summaries of large data streams in an efficient and incremental manner. Programming the desired logic is challenging due to the enormous volume of data and hard constraints on available memory and response time. This project aims to assist IoT programmers in meeting this challenge by designing a query language with natural and high-level constructs suitable for processing streaming data, supported by a compiler and run-time system to facilitate deployment while meeting the constraints of desired accuracy, memory footprint, and real-time response. The design of the query language draws upon insights from two distinct paradigms: extensions of relational query languages for handling streaming data, and state-based languages for runtime monitoring and synchronous programming. The novel integration of linguistic constructs allows the programmer to impart input data stream a logical hierarchical structure and also employ relational constructs to partition the input data by keys and to merge data streams from different sensors. While theory of approximation and streaming algorithms allows exploration of trade-offs among language features, accuracy, and processing time during compilation, distributed platforms such as Apache Storm are used for high performance processing. Two application domains, device monitoring for physiological patient data and dynamic monitoring of network traffic for anomalies, are used for experimental evaluation. A course module on predictable real-time decision making is being developed, and can be used in an undergraduate course on data science, in graduate courses on database systems and cyber-physical systems, in summer schools, and as tutorials. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.