BIGDATA: F: Bringing Interactive Data Management to Scientists, Analysts, and the Masses: A Holistic Unification of Spreadsheets and Databases

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

With the proliferation of datasets in many spheres of our everyday life, including science, finance, and commerce, managing tabular data, i.e., data represented by a set of tables or relations, has become increasingly essential. The state of the art in tabular data management spans two distinct paradigms with a sharp divide--spreadsheets and databases--with drastically different strengths and weaknesses; both enjoying tremendous successes but facing fundamental limitations. To date, interactive ad-hoc management of large data remains cumbersome and difficult. This project aims to holistically unify database and spreadsheet technology, and thus bring to scientists, analysts, and lay users, the ease of use and interactivity of spreadsheets with the scalability, expressiveness, and collaboration capabilities of databases. The project develops DataSpread, a system to holistically unify databases with spreadsheets--to marry the traditional strengths of scalability, expressiveness, and consistency of databases with the interactivity of spreadsheets that billions of end-users are so familiar with, and in the process rethink user-facing interactions beyond spreadsheets which the project will develop and evaluate via extensive user studies. The research team studies the development of new models, algorithms, structures, and interaction primitives to make the unification practical. Specifically, this research develops new methods for compactly representing spreadsheet data and computing over queries on a spreadsheet, positionally aware indexing structures, and mechanisms for efficiently propagating updates to the user viewport. The project also studies the design of new interaction primitives to replace full-fledged SQL, which is more conducive to a spreadsheet interface, and in support for user-centric transactions. While aiming at developing general techniques to advance the state of the art in data management, the project will be driven by the collaborators' real-world challenges--for use cases, pain points, practical deployment, joint open-source development, and critical evaluation and feedback.