Active Sequential Change-Point Detection for High-dimensional Streaming Data

New York University, New York NY

Investigators

Abstract

This research project focuses on active sequential change-point detection for high-dimensional streaming data under sampling or resource constraints, with numerous important real-world applications, including biosurveillance, environmental monitoring, epidemiology, disaster management, homeland security, quality control in manufacturing engineering, and threat detection. The project aims to develop simple yet effective algorithms that are able to quickly detect undesired anomalies or events, subject to false alarm rates and sampling control constraints, when monitoring large-scale streaming data from complex systems. The results of the research are expected to advance the understanding of real-time anomaly detection and online monitoring of high-dimensional streaming data. Graduate students will also receive training through their involvement in the project's research. This project aims to develop new mathematical, computational, and statistical theories and tools for active sequential change-point detection for high-dimensional streaming data under sampling or resource constraints. Our specific research aims are to develop computationally scalable and statistically efficient algorithms to detect sparse changes in the high-dimensions under two settings of sample control constraints: (i) the sequential design setting where sampling matrices can be sequentially or adaptively chose based on past observed data, and (ii) the random design setting where the sampling matrices are random as in the modern statistics or machine learning literature. Moreover, real-world case studies in anomaly detection or online monitoring will be investigated. Results of the project are expected to significantly advance the state of the art in sequential analysis, change-point detection, multi-armed bandit problems, streaming data analysis, and large-scale inference. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.