Survivable Continual Data Streams

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Continual data streams are a generalization of continuous data streams due to two major factors: (1) irregular data bursts and (2) the need to integrate all kinds of data and metadata, instead of pure time series or multimedia. Continual data streams naturally have a trade-off between performance scalability properties and system survivability properties. On the one hand, survivability requires increased redundancy since node and network instability inevitably renders parts of the system unavailable. On the other hand, scalability requires a decrease in data redundancy, to reduce update and propagation costs. This inherent trade-off between survivability and scalability is a major research challenge due to the irregular arrival and integration of continual data streams. The project will investigate the approaches that span the spectrum between absolute consistency guarantees for replicas in traditional replication on one extreme and by-chance consistency/zero guarantees for cached copies in traditional proxy caches on the other extreme. Formally, this approach is based on the notion of bounded inconsistency such as Epsilon Serializability. The main technical idea is to keep the distance between a replica and the original to within the specified threshold (to handle bursts) while optimizing the performance scalability and integrating heterogeneous data streams.