III: Small: Scalable Event Trend Analytics For Data Stream Inquiry

Worcester Polytechnic Institute, Worcester MA

Investigators

Abstract

Data streams have grown in unprecedented scale and velocity in recent years. The real-time discovery of emerging event trends in data streams is essential for time-critical applications from computing infection spread patterns across major medical facilities to detecting frequent stock trends. This project overcomes the shortcomings of state-of-the-art systems and provides practical solutions for this important class of analytics. The invention of transformative strategies to provide these Event Trend Analytics services is game changing. Event stream applications affecting our daily lives include health care and financial fraud. The integration of this project's activities with the training of a future STEM workforce will occur within a new interdisciplinary degree program in Data Science. This project develops both a sound theoretical foundation and practical processing strategies that empower applications to pursue interactive event trend analysis with high responsiveness. This project enables Kleene-closure based analytics in modern stream processing systems. Scalable Event Trend Analytics allows applications to gain trend-related insights from high-velocity stream data. Project activities include:(a) expressive event trend modeling and matching semantics, (b) compact event trend encoding techniques, (c) optimizaton methodology for aggregation push-down into the Kleene closure computation avoiding expensive trend construction, (d) aggregation computation at multiple granularity levels, (e) principled foundation of correctness and completeness of the query processing paradigm, and (f) distributed and shared execution of trend analytics for scale-up. Principles from database systems, such as query rewriting, optimization, execution sharing, and scalable cloud computing, are applied to this problem. Innovations include trend aggregation strategies that selectively skip expensive trend construction, and compression methods for aggregation state models. These methods will be validated for utility and effectiveness using real-world workloads in partnerships with industry collaborators in healthcare and financial applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.