Frameworks: StreamCI: an AI-ready streaming data platform for a wide range of scientific applications

Purdue University, West Lafayette IN

Investigators

Abstract

This project creates StreamCI, an open-source streaming data platform designed to help researchers and users across industries to more effectively harness massive sensor data streams using modern data analysis methods such as artificial intelligence (AI) and machine learning (ML). StreamCI simplifies the collection, management, and analysis of sensor data streams in a user-friendly, cloud-based system accessible to the broad research community. By lowering technical barriers and making data AI-ready, StreamCI empowers domain experts to build intelligent and responsive applications that drive faster discoveries and more effective solutions. The platform also serves as an open educational resource, training the next generation of data- and AI-savvy researchers to create data-driven solutions for critical societal needs. StreamCI is designed to streamline the entire workflow for sensor data streams, from capturing raw sensor data to processing it at various levels of fidelity, anonymization, and transformation; and to apply suitable ML methods across different modalities and enable Findable, Accessible, Interoperable, and Reusable (FAIR) data sharing for research reproducibility and cross-domain science. The core innovations of the project include high-level data abstractions that allow researchers to combine data streams across sources; AI-readiness through novel tools for data preparation and canonical and customized data pipelines; low-code application development and integration of powerful tools; and scalable, seamless service deployment and delivery. StreamCI is used across a wide range of domains, including energy, manufacturing, transportation, agriculture, ecology, audiology, and education, demonstrating its broad applicability and versatility. StreamCI is deployed as an open-source, cloud-based software-as-a-service. Its open-source release allows researchers to set up and operate StreamCI on their local computers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.