NeTS: Small: Mashup Content Harvesting for an Open Internet

Northwestern University, Evanston IL

Investigators

Abstract

The Internet has transformed from a small network into a gigantic infrastructure of enormous scale. The amount of publicly-accessible data on the Web, hosted at millions of servers and billions of Web pages around the world, is approaching the Exabyte limit. Given the enormous size of this data, which continues to grow at a rapid pace, this project asks if it is possible to mirror content, i.e., represent it as a function of the massive data already available online. Consequently, a related question is if the entire Internet can be treated as a single Content Distribution Network. That is, can one harvest the desired content, which may not exist at any one particular location on the Internet, by downloading pieces of mirrored content from the publicly available Web services? The PI proposes mashup content harvesting as a comprehensive approach that aims to accomplish the above goals. In this approach, users have the ability and means, which the PI will develop, to create or replicate content by representing it in terms of the significant amount of data publicly available on the Web. By enabling users to effectively mirror new data as a function of existing data, it becomes feasible to achieve mass-scale content distribution, yet without hosting any new server- or peer-to-peer infrastructure. Contrary to steganography methods, which explicitly embed information by writing it to a carrier medium, the prosed system effectively mirrors data onto existing publicly-available content without any explicit writing. Intellectual Merit: The proposed research will address fundamental questions that are key to developing and deploying mashup content harvesting. The major challenges are (i) how to embed both data and hopping information via information available on publicly accessible sites, (ii) how to select a set of core data segments to maximize mirroring efficiency, (iii) how to devise and deploy methods for scalable and accurate characterization of mashup content harvesting data carriers, (iv) which fundamental Web carrier properties enable effective mashup content mirroring and harvesting, (v) what are the lower-bounds in terms of the Web page diversity and connectivity that affect system performance, (vi) how mashup content harvesting performs in closed national-level Web environments, and (vii) how mashup content harvesting performs when content carriers are limited to Web pages associated with specific world languages. Broader Impact: This project has the capacity to make a significant impact by facilitating the development of a free and open Internet. Indeed, free, i.e., non-commercial, Web services have created enormous value to numerous individuals and organizations on the Internet. Common for such services is to leverage community contributions from individuals around the world. Unfortunately, despite enthusiastic collaborative contributions, existing free Web services aren't really free. For example, Wikipedia must raise large amounts of money to support its hosting costs. Mashup content harvesting provides a feasible approach for online communities to sustain, no matter how small they may be. This is possible to achieve neither by threatening nor competing with commercial mainstream Web services, nor affecting their performance. The PI plans to design and disseminate mashup content harvesting as easy-to-use browser plug-ins.