ITR: Massively Convergent Distributed Computing

Cornell University, Ithaca NY

Investigators

Abstract

Computer networks have become more pervasive in scope and mission-critical to businesses, scientific endeavors and other computing applications. But at the same time, network instability and unreliability has become an increasing problem. This research tackles this issue by laying the groundwork for a new system infrastructure and an associated generation of algorithms capable of overcoming the unreliable nature of the network using probabilistic techniques for ensuring consistent cooperative behavior, coordination, and coherency. The central property of the infrastructure is that it uses probabilistic techniques (instead of absolute guarantees) but, by so doing, obtains outstanding stability under stress, scalability, and robustness to even extreme disruptions. This research will proceed in steps, starting with a quantitative study of existing distributed system architectures, especially focusing on (1) gossip-style data dissemination, (2) anonymous communication, and (3) peer-to-peer networks. The next step will explore the design of new and enhanced architectures for large-scale distributed analysis systems. It is the ultimate goal to implement new designs in ways compatible with prevailing platforms (Windows, Java, Unix), and release public-domain versions of these experimental solutions, and to measure their behavior on a large scale.