MRI: Development of a System for Interactive Analysis and Visualization of Multi-Terabyte Datasets

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

This project, creating a new cluster computer design targeted at interactive exploration of datasets tens of terabytes in scale, supports analysis and visualization of massive datasets. Research efforts concentrate mainly on genomics and bioinformatics, cosmology and geophysics, and computational fluid dynamic (turbulence and stellar convections). For the purpose of rendering, in near real-time, datasets whose render information requires up to 2TB of simultaneously active storage, a large system consisting of 80 dual processor nodes with a massive amount of disk space will be used for the visualization task. The visualization system will allow slightly more leeway for exploratory viewing of interactions among multiple variables at multiple time slices. Motivated by the need to interactively explore voluminous datasets, the system supports interactive analysis and visualization that follows any interactively specified and/or modified path through the 4 dimensions examining all the data along that path by streaming this data, potentially replicated at each of 80 network nodes, into the memories at a rate that matches the node's ability to perform significant analysis and/or visualization operations on the data and to display the results for the user at high resolution on viewing screens as large as the LCSE's 13 Mpixels Power Wall. Volume visualization and various kinds of pattern matching and cross correlation target this data. The design goals require significant research in scientific visualization, data management, data storage and cluster data sharing, as well as prority-based job scheduling. Broader Impacts: The development of this system involves many students and postdocs in the collaborating disciplines. The system will be used in instruction as well as in research leading to advance degrees. The computer science department at Fond du Lac Tribal and community College in Cloquet Minnesota will also benefit from this project due to the existing collaboration of the Laboratory for Computational Science and Engineering (LCSE) with this Native American serving college. LCSE has also shared its technology developments and software. Once again, the developed software will be made available over the Web in Open source format.