Spectral Analysis for Data Mining

University Of Washington, Seattle WA

Investigators

Abstract

This research is concerned with the development, analysis and empirical evaluation of algorithms for information retrieval and data mining, primarily using spectral analysis. A partial list of the fundamental questions being addressed are: How should data be stored, organized and processed so as to allow the most effective retrieval of information? How can ``important'' structure and ``meaningful'' patterns be found within a large data-set? How and when can this hidden structure be used to facilitate determination of missing data or to ``clean'' data that is imprecise or partially incorrect? What are appropriate models for data generation, and how can these models be used to improve the design of data mining algorithms? The researchers are studying applications that have already received a great deal of attention and on which empirical success has been achieved, including object clustering and web site ranking. In addition, they are designing and analyzing algorithms, and developing and analyzing models for newer data mining problems, including collaborative filtering, topic distillation, spam detection and prevention, and hierarchical clustering. Matrix perturbation theory is the foundation for the development of theoretical results. The researchers are also studying new techniques for speeding up the computation of the SVD. The theoretical research is being complemented by experimental evaluation on real data sets.