III-COR-Small: Capturing Data Uncertainty in High-Volume Stream Processing

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

The goal of this project is to develop a stream processing system that captures data uncertainty from data collection to query processing to final result generation. This project focuses on data that is naturally modeled as continuous random variables. For such data, it employs a principled approach grounded in probability and statistical theory to capture data uncertainty and integrates this approach into high-volume stream processing. The first contribution of the project is to capture uncertainty of raw data streams from sensing devices. Since the raw streams may not present data in a format suitable for query processing and can be highly noisy, this project employs probabilistic models of the underlying data generation process and machine learning techniques to efficiently transform raw data into a desired representation with an uncertainty metric. The second contribution is to capture uncertainty as data propagates through various query operators. To efficiently quantify result uncertainty of a query operator, this project explores various techniques based on probability and statistical theory to reduce statistics that input streams need to carry and to expedite the computation of result distributions. This project integrates research and education through curriculum development and enables broader participation of women in research through college outreach and CRA's distributed mentor program. This project also includes software release and real-world deployments in domains such as object tracking and monitoring and hazardous weather monitoring, resulting in significant scientific and social impacts. Results of this project are disseminated at the project web site (http://avid.cs.umass.edu/projects/uncertain-streams/).