Flash Gordon: A Data Intensive Computer

University Of California-San Diego, La Jolla CA

Investigators

Abstract

ABSTRACT UCSD 0910847 Norman, Michael L. This project supports the acquisition, deployment and operation of a new supercomputing system suitable for data-intensive applications. The system, to be known as Flash Gordon, will be deployed by the University of California at San Diego at the San Diego Supercomputer Center and integrated into the TeraGrid. The system, which has been designed by Appro International Incorporated, with partners Intel and ScaleMP, seeks to bridge the widening latency gap between main memory and rotating disk storage in modern computing systems. It uses flash memory to provide a level of dense, affordable, low-latency storage that can be configured as either extended swap space or a very fast file system. The system will consist of very large shared virtual-memory, cache-coherent "super-nodes" to support a versatile set of programming paradigms. Peak performance will exceed 200 teraflops/s in double precision. Flash Gordon's large addressable virtual memory, low-latency flash memory, and user-friendly programming environment will provide a step-up in capability for data-intensive applications that scale poorly on current large-scale architectures, providing a resource that will enable transformative research in many research domains. Even sequential codes will be able to address up to terabytes of fast virtual memory. Examples of scientific challenges, as described in the proposal, that this resource will allow researchers to tackle, include the following. De Novo Genome Assembly: Gene sequencers produce information about many small fragments of a genome. Some recent assembly algorithms use a graph-based approach, much more readily executed on a shared-memory system. Using Flash Gordon, researchers will be able to rapidly assemble complex genomes such as mammalian genomes. Astronomy: Modern astronomy databases can be large; for example, the Sloan Digital Sky Survey is approximately six terabytes in size. Typically, the analysis algorithms that researchers use to perform complex searches for astronomical phenomena can be implemented more easily on shared-memory systems. Flash Gordon will enable researchers to load a copy of the Sloan Digital Sky Survey into the flash memory associated with a super-node, greatly extending the types of analyses astronomers can make. Astrophysics: Cosmological simulations produce many terabytes of output describing the simulated universe. Detailed analysis of the results of these simulations, to find features such as collapsed halos, galaxy mergers, dwarf galaxies, and galaxy clusters, often requires density-based cluster analysis that does not parallelize well. With Flash Gordon, these analyses can be accelerated by exploiting the large SMP partitions and fast flash memory. Interaction Networks: Interaction networks, graphs representing the relationships between objects, are used in research in areas such as epidemiology, phylogenetics, systems biology, and population biology. These interaction networks can represent relationships between types of data stored in different databases; for example, the combination of social network databases with medical records and genomic profiles to explore questions such as genetic resistance to disease. Flash Gordon will speed analysis of large interaction networks because the databases can be stored on the solid-state disks, greatly reducing access time and permitting more complex types of analysis. The project team will leverage a number of ongoing educational activities at UCSD to expand and diversify the community of users that can utilize this computational resource, including successful outreach programs for women and minorities from underrepresented groups in science and engineering. The project will also create a summer training program for undergraduates.