EAGER: Studying Decentralized Searches in Large-scale Agent Networks

Drexel University, Philadelphia PA

Investigators

Abstract

Many digital information networks such as the Web and the Internet operate in a rather decentralized manner with little global control. Searching in these networks is a great challenge because data are increasingly large and distributed. The proposed research envisions a fully decentralized architecture in which individual systems (agents) can communicate and work together to assist people in finding relevant information from growing distributed sources. The proposed search model will utilize collective intelligence of software agents in large networks in order to help individuals find access to information of an unmatched magnitude. The outcome of the research will revolutionize the way in which large-scale search engines operate and significantly improve the coverage of the world's information that can be discovered by ordinary people. Decentralization is the nature of many naturally, socially, and technologically grown structures that scale. The research aims to address the problem of finding relevant information in large-scale agent networks with distributed contents. The project will study decentralized search methods in growing agent networks and focus on their capacity to scale up (out). The primary goal is to understand how a very large number of search agents (many of which have limited computing capacity) can interconnect and collaborate to enable efficient query routing and information discovery. The research team will experiment with very large agent networks (of at least a million agents) and study how information indexing and searching can function efficiently in a fully decentralized manner. The plan is to examine the influences of important factors such as semantic network overlay (inter-connectivity) and distributed neighbor sampling (learning) on search efficiency and adaptation. The project will also investigate the capacity of the distributed agent architecture for greater search coverage as well as better freshness and timeliness in the search results.