III-COR: Exploiting History in Continuous Monitoring Systems

University Of Washington, Seattle WA

Investigators

Abstract

The goal of this research project, called Moirae, is to investigate the benefits and challenges of integrating history into a near-real-time monitoring system and to build a new continuous query engine that supports this integration. To achieve this goal, the project takes the following steps: (1) develop new query models for integrated queries over live and historical data; (2) develop new algorithms that effectively match new events with similar past observations; (3) develop a new continuous query engine that effectively supports the new query model (the engine includes a partitioned stream data store, a scheduler for fair and incremental historical query execution, new operators for merging historical data with data streams, and mechanisms for user-feedback); and (4) evaluate the practicality and performance of the system on data traces from real application domains. Project funds support the training of PhD students. However, the project includes a large system development component that serves to train both graduate and undergraduate students in research and systems building. The result of this project will provide a new type of continuous processing engine that will better support monitoring applications in domains ranging from computer system monitoring, to network monitoring, and sensor-based environment monitoring. Such large-scale monitoring applications are critical for enterprises that operate at large scales. These enterprises need to carefully monitor their infrastructures to effectively handle and diagnose failures and deliver high-quality services to their customers. The software and technical papers resulting from this project will be disseminated through the project website(http://data.cs.washington.edu/moirae/moirae.shtml).