CiC (RDDC): Continuous Bulk Processing in the Cloud

University Of California-San Diego, La Jolla CA

Investigators

Abstract

This research explores an alternative data processing architecture that fundamentally improves computing efficiency to reduce costs and provide enhanced data mining capabilities for cloud computing. The next major advancements in IT, medical, and science informatics will largely be dictated by our ability to store, manage, and analyze large amounts of information. For many important problems, advances in data acquisition tools are rapidly increasing data generation rates, exceeding our ability to manage and process the data they produce. This proposal investigates novel architectures for continuous bulk data processing, which support data-intensive applications that perform complex multi-step computations over successive batches of input data (e.g., from scientific or medical instruments), allowing analytics to simply be updated, not recomputed, when new data arrives. The research seeks to raise incremental processing as a first-class abstraction, significantly lowering the barrier of data-intensive projects to take advantage of cloud computing. Towards that end, the work will develop data analysis portal architectures to increase the reach of on-demand data analytics across various application domains. As a case study, it will develop a portal for fatty-liver disease, allowing care givers on-demand access to medical analytics to improve care, reduce cost, and improve clinical outcomes for this important disease.