CSR-AES: Troubleshooting Large Scale Computing Grids with Machine Learning Techniques

University Of Notre Dame, Notre Dame IN

Investigators

Abstract

Both users and administrators of computing grids are presented with enormous challenges in debugging and troubleshooting. Diagnosing a problem with one application on one machine is hard enough, but diagnosing problems in workloads of millions of jobs running on thousands of machines is a problem of a new order of magnitude. Suppose that a user submits one million jobs to a grid, only to discover some time later that half of them have failed. Each individual failure could be the manifestation of one of many kinds of error: a job specification error, a machine configuration error, a transient system state, and many others. It does little use to investigate any one instance of failure. Rather, users of large scale systems need tools that describe the overall situation, indicating what problems are commonplace versus occasional, and which are deterministic versus random. Machine learning techniques can be used to debug these kinds of problems in large scale systems. The proposal poses the following research questions: What sort of failures is most common in grids? What data must be collected to identify these failures? What innovations in machine learning algorithms are required to be successful in this domain? This project will generate new understanding by collecting large amounts of production data from TeraGrid and OSG, developing novel analysis techniques, and working closely with end users to produce useful diagnoses. The results will provide both new understanding of complex large scale computer systems, as well as innovations in machine learning algorithms to tackle such scenarios.