NGS: An Integrated Middleware and Language/Compiler Framework for Data Intensive Applications in a Grid Environment

Ohio State University Research Foundation -Do Not Use, Columbus OH

Investigators

Abstract

EIA-0203846 Gagan Agrawal Ohio State University An Integrated Middleware and Language/Compiler Framework for Data Intensive Applications in a Grid Environment We propose to develop methods that will enable data intensive applications to be developed or specified using high-level interfaces, and yet effectively utilize grid resources. This will be accomplished by developing runtime techniques, a middleware based upon filter-stream programming, and Just in Time (JIT) compilation techniques for decomposing high-level programs into a set of filters. Our tools and techniques will exploit important commonalities of the data intensive applications we target. Our target class of applications include both scientific and commercial data intensive applications. We are developing a middleware framework called DataCutter. DataCutter provides support for processing of datasets stored in archival storage systems in a wide-area network. DataCutter exports an interface for specifying processing as a set of coordination filters. While DataCutter offers a framework suitable for developing scientific and commercial data intensive applications in a grid environment, its programming interface is a relatively low-level one. The programmers need to decompose their application into a set of filters and specify the filters and their interaction. In this project, we propose a series of high-level language interfaces that can be used for expressing data-intensive applications. We target data parallel Java, XML query language (XQL), and mining operators as interfaces that will allow processing to be specified assuming that all data and compute cycles are available at a single site. We propose research in compiler techniques that will decompose such applications into a set of filters, based upon the availability of data, computing, storage, and networking resources.