Collaborative Research: ATD: Hawkes Process-Based Causal Relationship Discovery For Complex Threat Detection and Forecasting

University Of California-San Diego, La Jolla CA

Investigators

Abstract

This project develops novel causality-guided approaches for reliable threat detection and forecasting in complex event streams. Understanding causality is crucial because it allows us to identify the true drivers behind anomalies and pinpoint critical events that will significantly impact future event streams. For instance, to swiftly adapt to extreme climate shifts, it is essential to detect unusual earth movements or severe weather patterns that causally induce these shifts. Recognizing these causal relationships enables the implementation of preemptive countermeasures and enhances long-term forecasting. Similarly, in the context of information hazards, identifying latent patterns in social media posts that causally drive the spread of misinformation is vital. Understanding these causal drivers allows for quicker assessment and recognition of future threats, making it possible to take timely and effective action to ensure public safety. Moreover, the benefits of such methods extend far beyond security applications. They can unlock mechanistic insights into scientific event streams like neural activities, enriching the collection of techniques for scientific discovery. This project opens new lines of research, expanding the domain and scope of algorithmic threat detection. Specifically, it focuses on three key research topics: (1) causal inference for observed event streams with latent confounders and nonstationarity, (2) causal representation learning for latent event streams, and (3) causal anomaly detection and long-term forecasting. Leveraging the Hawkes process model—a self-exciting point process model—the investigators will establish a formal framework to determine when and how causal links can be inferred from partially observed and potentially non-stationary event sequences. The identified causal relationships will enable comprehensive situational awareness while pinpointing anomalies and providing long-term forecasts. The mathematical theory, algorithms, and software produced through this research will be transformational. This project aims to establish a foundational understanding of causality for algorithmic threat detection, provide principled algorithms for analyzing complex event streams, and broaden the application of these methods to diverse social and scientific domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.