III: Small: BigSolver: Data-Intensive Solver Support for Big Data Exploration and Mining

Brown University, Providence RI

Investigators

Abstract

Exploratory data analysis plays a key role in data-driven discovery in a wide range of domains including science, engineering and business. This project will enable data scientists from many domains to search and explore their large data sets far easier and faster than they do today. Rather than spending a lot time to set up exploration pipelines by combining multiple software tools, users will work with a single, general purpose and more usable system. Overall, this project will enable fundamentally richer means for data exploration and lead to significant productivity improvements; it will accelerate discovery and breakthroughs in many domains such as e-commerce, finance and science. This research will be incorporated in undergraduate and graduate coursework. The outreach activities include special research and education-focused programs that are geared towards undergraduates and high-school girls. This research proposes to design and build a new prototype database system, called BigSolver, that will uniquely integrate constraint solving and data management techniques. The result will enable rich, highly-efficient means for generic ad hoc search, exploration and mining over large multidimensional data collections. BigSolver will allow Constraint Programming (CP) machinery to run efficiently inside a DBMS without the need to extract, transform and move the data. This marriage will offer the rich expressiveness and efficiency of constraint-based search and optimization provided by modern CP solvers with the ability of Database Management Systems (DBMSs) to store and query data at scale. This work will be an early yet transformative step in enriching the functionality of database systems towards new data- and search-intensive applications. The proposed effort will develop novel approaches for synopsis-based in-memory processing, speculative solving, search query optimization, parallel processing and load balancing, as well as architectural innovations, which will collectively yield performance and usability levels that far improve those of the state of the art. For further information, see the project website at: http://database.cs.brown.edu/projects/SearchLight/