NeTS-NBD: Accurate Estimation of Network Measurement Matrices Using Multiple Data Sources

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Traffic and loss matrices are the most important network performance statistics that need to be accurately measured and carefully monitored in an Internet service provider (ISP) network. They are essential for network management functions such as traffic engineering, capacity planning, traffic anomaly detection, and fault diagnosis. Existing techniques for estimating traffic matrix are to infer it indirectly from some other network statistics that can be directly measured such as SNMP link counts or Cisco NetFlow records. The accuracy of these inference techniques, however, are generally not very high because such statistics can be noisy, sparse, and dirty. We propose to develop novel statistical signal processing techniques that allow us to infer traffic and loss matrices as accurately as possible from the data we already have in hand that can be noisy, sparse, and dirty. The general methodology correlates multiple sources of data, which by exploiting the statistically orthogonal nature of noises in independent observations, achieves much better accuracy than obtainable from a single data source and identifies and removes dirty data. Broader Impacts: This project will offer undergraduate and graduate students research and learning experience across multiple disciplines. The ongoing collaborations with industry through this project will facilitate application of scientific discoveries to the application domains. The results will be broadly disseminated through papers, talks, workshops, and software releases. The PIs will continue to work hard to actively engage underrepresented groups in research and education.