BIGDATA: Collaborative Research: F: Discovering Context-Sensitive Impact in Complex Systems

Arizona State University, Scottsdale AZ

Investigators

Abstract

Successfully tackling many urgent challenges in socio-economically critical domains (such as sustainability, public health, and biology) requires obtaining a deeper understanding of complex relationships and interactions among a diverse spectrum of entities in different contexts. In complex systems, (a) it is critical to discover how one object influences others within specific contexts, rather than seeking an overall measure of impact, and (b) the context-aware understanding of impact has the potential to transform the way people explore, search, and make decisions in complex systems. This project establishes the foundations of big data driven Context-Sensitive Impact Discovery (CSID) in complex systems and fills an important hole in big data driven decision making in many critical application domains, including epidemic preparedness, biological pathway analysis, climate, and resilient water/energy infrastructures. Thus, it enables applications and services with significant economic and health impact. The educational impacts of this project include the mentoring of graduate and undergraduate students, and the enhancement of graduate and undergraduate Computer Science curricula at both Arizona State University (ASU) and New Mexico State University (NMSU) through the incorporation of research challenges and outcomes into existing classes. The technical goal of this project is to establish the theoretical, algorithmic, and computational foundations of big data driven context-sensitive impact discovery in complex systems. This project develops probabilistic and tensor-based models to capture context-sensitive impact from complex systems, often modeled as graphs, and designs efficient learning algorithms that can capture both the contexts and the impact scores among entities within these different contexts. The modeling of the context sensitive impact considers dynamic nature of relevant contexts and the diverse applications. This requires addressing several major challenges, including latent contexts of impact, heterogeneous networks of entities, dynamicity of impact in varying contexts, and high computational and I/O costs of context-sensitive impact discovery. Therefore, this project designs novel scalable probabilistic and tensor-based algorithms to capture and represent context-sensitive impact. These algorithms and the novel data platforms they are deployed in are efficient and scalable in terms of off-line and on-line running times and their space requirements. To achieve necessary scalabilities, the developed platforms employ novel multi-resolution data partitioning and resource allocation strategies and the research enables massive parallelism and efficient data access through new non-volatile memory based data management techniques.