Complexity Regularized Signal Processing for Networking Applications

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

Abstract 0310889 Robert D. Nowak William Marsh Rice U Today's private and public communications networks are critical systems of data terminals, routers, and switches that provide the backbone of our economic, scientific, and education systems. Consequently, signal processing methods for estimating performance characteristics and detecting key component failures and malevolent behavior are crucial for insuring the reliability and robustness for this vital infrastructure. Moreover, networks of sensors and actuators will soon be part of our information systems. These networks will provide critical links between our environment and our information-based society. Distributed and hierarchical algorithms for data analysis and signal processing will be central to the operation of such networks. This research targets these important, emerging signal processing applications in networking. The proposed research blends the fields of adaptive and distributed signal processing, nonparametric estimation and classification, network traffic measurement and modeling, and network traffic analysis and tomography. The project involves the development of theories and methodologies for data analysis, estimation, and classification in networking applications. Most inference problems arising in networking are extremely challenging; the amount and nature of the data that is easily collected is very limited, and the estimation or classification tasks are often severely ill-posed. Common inference methodologies such as maximum likelihood are of limited utility in large-scale networking problems due to these confounding factors. Therefore, complexity regularization methods for estimation and classification provide the core computational signal processing tools in the project. These methods temper the trade-off between fitting to the data and model complexity (and hence variability). Two core problems in network analysis and inference are the focus of this proposal. (1) Network Tomography: Network tomography involves estimating performance characteristics and traffic flow patterns from measured traffic at a limited number of points in the network. This problem is very ill-posed and existing schemes do not perform well in large-scale implementations. We propose novel complexity regularized approaches to network tomography that aim to capitalize on certain, naturally occurring, sparsity characteristics of the network tomography problems. (2) Multi-channel Network Traffic Analysis: Much work has been done in the area of network traffic analysis, but the focus has been mostly on signal point traces. Analysis and understanding of the relationships between traffic flows at different points in networks is crucial to overall performance. We propose to investigate and develop analysis methods capable of revealing important traffic interrelationships, dependencies, and coincidences at multiple measurement points. The research investigates: 1) fundamental limits in network inference methods for estimating and detecting conditions critical to network performance; 2) integrated and flexible approaches to spatio-temporal analysis of internetwork traffic patterns; 3) scalable, distributed, and decentralized algorithms for network data analysis and inference; 4) basic theory of complexity regularization and distributed signal processing.