Collaborative Research: ATD: Rapid Structure Recovery and Outlier Detection in Multidimensional Data

Colorado State University, Fort Collins CO

Investigators

Abstract

The project aims to develop and apply tools coming from algebra to identify attacks embedded within legitimate communications streams and online fora. The research will enhance statistical methods and Artificial Intelligence to detect and combat threats. Specifically, the research will work with unlabeled data and will be applicable in environments that cannot support learning on training data, such as new domains or previously unseen threats. The project will provide training opportunities for both undergraduate and graduate students, preparing them for their future careers in STEM fields. The research on detecting outliers in data, recovering missing data, and detecting hidden constraints will have many applications across the sciences. The project aims to design a self-adaptive linear-time algorithm to separate signals, find hidden constraint equations, and detect similarities in high-dimensional data (tensors). This collaborative research of the three investigators and student participants will focus on three independent tasks. The first will extend signal separation and outlier prediction to a continuous spectrum. The second will refactor algebraic structures into tensor networks for uniform algorithms. The third will devise faster (linear-time) solutions to matrix systems to enhance practical range. Analysis of high-dimensional data often runs afoul of the curse of dimensionality: as the number of independent parameters increases, the time needed to search neighbors grows exponentially. Also, the meaning of outlier becomes blurred as notions of far apart and close together are less distinguishable, and traditional statistics tend to identify large subspaces. The new algebraic markers will detect structure in any dimension and be quickly computable. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.