SBIR Phase I: Customized Metasearch Engine

Webscalers L.L.C., Lafayette LA

Investigators

Abstract

This Small Business Innovation Research (SBIR) Phase I research project proposes to demonstrate the feasibility of the company's metasearch technology by developing a prototype system for automatically creating customized metasearch engines based on the search engines specified by a user (i.e., the URLs of the interface pages of the search engines are provided by the user). A metasearch engine is a system that provides unified access to multiple existing search engines and it is an effective mechanism to combine the coverage of multiple search engines and to reach the portion of the Web that cannot be publicly crawled. Several issues must be addressed. First, different form tags that contain search engine interfaces need to be automatically analyzed so that information useful for connecting to the search engines (i.e., submitting queries and receiving results) through a program can be correctly extracted. Another issue is that various result pages returned by different search engines in response to user queries need to be automatically parsed so that useful information such as the URLs and snippets of retrieved documents can be correctly extracted. A third issue is that retrieved documents from multiple search engines need to be automatically merged into a single ranked list with good retrieval effectiveness while minimizing the need to downloading/analyzing the actual documents. One final issue is that created metasearch engines need to be automatically maintained so they can continue to work correctly when independently managed search engines change. The outcome of this project will contribute to research in several related areas such as distributed information retrieval, autonomous systems, information extraction and self-maintainable systems. Customized metasearch engines can benefit individual users and companies/organizations in their information acquisition. This solution saves users' time and increases productivity