CCRI: Medium: FANTAIL -- facilitating advances in network topology analysis

University Of California-San Diego, La Jolla CA

Investigators

Abstract

Internet cartography--the scientific measurement, analysis, and annotation of Internet topological structure--has emerged as a new field in computer and network science. In support of this field, several global Internet measurement infrastructures continuously execute comprehensive topology mapping measurement experiments. For years the Center for Applied Internet Data Analysis (CAIDA) has operated the longest-running of these measurement infrastructure platforms (Archipelago), which has supported scientific measurement experiments of the global Internet since September 2007. These data sets have already yielded impacts across a broad range of computer science sub-disciplines. Yet the biggest remaining obstacle to even more productive scientific use of this type of data is infrastructural: the lack of an easy-to-use and analytically powerful exploratory interface. This project will develop the FANTAIL system--Facilitating Advances in Network Topology Analysis--to enable the full potential value of massive raw Internet end-to-end path measurement data sets. The proposed system will enable new research directions and increase our understanding of the Internet as critical infrastructure. CAIDA's solutions will lower barriers for entering the field of network science and computing research thus leading to increased diversity of the user base. FANTAIL will have four components: (1) an interactive web interface; (2) an application programming interface (API) built on web standards; (3) a full-text search system; and (4) a big-data processing system. This project will be driven by specific community-requested use cases, use open-source components where possible, and create analysis modules that will serve as primitives for constructing more complex data-processing pipelines. Users will specify, via the web interface or API, a sequence of analysis modules to execute on the set of traceroute paths matched by their queries. The FANTAIL system will then perform the queries, run the analysis modules, and provide the output for download for further analysis or processing by researchers on their own systems. Different analysis modules will (1) perform data reduction (to minimize the amount of data users need to download and process), (2) enhance raw topology data with various annotations available publicly or created by CAIDA, and (3) offload commonly-needed analysis/data processing tasks from users. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.