CSR-EHS: High-throughput Real-time Stream Processing in Java

Purdue University, West Lafayette IN

Investigators

Abstract

Stream processing is a programming paradigm suitable for the class of data driven applications that must manipulate high-volumes of data in a timely and responsive fashion. Example applications include video processing, digital signal processing, and monitoring of business processes. The appeal of stream programming comes from its conceptual simplicity. A program is a set of independent filters that communicate exclusively by the means of unidirectional data channels. The output behavior of any filter is completely determined by the value on its input channels, thus reducing the chances of data races. Interestingly, the abstractions provided by stream processing languages are surprisingly close to event-driven real-time systems. Previous research has shown that stream programs can be implemented very efficiently on modern multiprocessors. The goal of this project is to integrate stream-based programming with real-time systems. The project extends Java to combine streams with objects and allow programmers to integrate stream computations with traditional object-oriented components in the same Java virtual machine. The work relies on the facilities provided by the Real-time Specification for Java to ensure that streaming codes meet real-time constraints of their target application. The broader impact of this project is to introduce a new programming paradigm that can be used by real-time and traditional developers. The choice of Java is expected to ease adoption, as programmers can use mainstream development environments, libraries and development methodologies for writing stream processing code.