Deterministic Parallel Programming for High Performance Computing

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

Hardware for High-Performance Computing is advancing at a relentless pace: In the not too distant future we can expect to see systems with over a million of concurrently executing threads, with hardware support for global memory access. On the other hand, we continue to use today the same low-level parallel message passing libraries that we have used in the last 15 years. This causes lower user productivity and does not leverage well modern communication hardware. We propose to explore new language designs that address both problems. It is generally accepted that programming in a shared memory model is easier (at least for initial program development): the ability of each thread to access each variable, using a common name space, reduces much of the burden of distributed memory programming. On the other hand, shared memory programming languages generally allow users to write nondeterministic code (where different outcomes are possible) and do not protect the user from memory races (where accesses to shared variables are not synchronized). This results in subtle bugs that are not reproducible and hard to detect. Furthermore, shared memory languages provide limited support for locality control ? resulting in lack of scalability. Nondeterminism is rarely needed in scientific computing, and scalability is essential. The PIs believe it is possible to develop languages that will support the large majority of programming patterns used in high-performance-computing; will provide the convenience of a shared-memory model; will prevent, by design, nondeterminism and detect races; and will provide user control of locality. The proposed research will explore the design for such a language and the required support technologies.