DCS: Ganga: Removing the Semantic Gap between Stream-based Applications and Stream Accelerators

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Project Description The PI plans to develop a novel distributed programming framework, Ganga, which addresses the increasing gap between the emerging application domains and the target platforms. At the heart of the framework is a new abstraction capsule that embodies computation and state. The application program is a dataflow graph of capsules connected by streams of data. The programming model does not require the application programmer to provide explicit mapping and distribution of the capsules to the nodes of the target platform (comprising of clusters and sensor networks). Intellectual Merit A semantically clean and simple model for programming complex distributed applications is one of the intellectual merits of the proposed research. The semantics of the capsule abstraction emulates many of the known paradigmsfor distributed computing such as RPC, active messages, continuations, and futures. The runtime automatically decides the appropriate semantics commensurate with the dynamic needs of the application and the dynamic resource availability of the target platform. Mechanisms in the runtime system include the ability to partition the capsule into finer capsules connected by streams, coalescing finer capsules into a coarser grain one internalizing the streams that connected them, migration of capsules among the computational nodes, restarting failed capsules, and garbage collection of memory resources. The sense-process-actuate characterization of the application is also applied at the system level to orchestrate the resource management decisions. The system state is sensed and the decision-making entity re-evaluates (and if need be, alters) the use of the resources. Developing such a characterization is another intellectual merit of the proposed research. Broader Impact As the discipline matures, it is clear that computing has to interface with domain experts to increase the relevance and importance of computing in the societal landscape. In particular, it is important to develop simple and intuitive interfaces for the end user that hide the complexities of the emerging distributed hardware platforms. By targeting application domains that have the potential for societal benefits, the importance of the proposed research is enhanced. Th PIs plan to do technology transfer of the research artifacts from this project through their partnership with industry. Ganga will be an interesting tool for teaching about parallelism. It cleanly incorporates known styles of parallelism in a single clean model. It exposes all the big issues while hiding the uninteresting details. It is the PIs intent to put the system into use for educational purposes. There is an active educational program at Georgia Tech in pervasive computing. An appropriate usability test of these technologies is to make the system available for class projects by novice users of distributed systems in pervasive computing classes both at the undergraduate and graduate levels.