CAREER: Scalable and Flexible Indexing of Compressed Sequences

Suny At Stony Brook, Stony Brook NY

Investigators

Abstract

Lossless data compression is a classical and ubiquitous task that reduces the size of data, leading to a decreased cost of transfer or archival. Modern applications, however, require more than compression. In domains such as computational biology, terabytes of highly compressible data are generated at an increasing rate. To fully take advantage of this data, it needs to be not only stored in a small space but also accessible and searchable. Recent years have witnessed the birth of data structures called "compressed indexes" that can accomplish this task. The research so far, however, has mostly focused on static indexes, leaving out important aspects such as support for updates and efficient construction. This project's overarching goal is to develop powerful, dynamic compressed indexes capable of handling a versatile set of queries and various representations, that can moreover be efficiently constructed. This will make it significantly cheaper, faster, and more energy-efficient to store and share highly compressible datasets such as DNA collections, thereby fully unlocking the potential of advances in DNA sequencing. The advances of this project will also be integrated into research experience for students, as well as outreach to non-computer science students. The main research goals of this project can be broadly classified into the following three directions. First, the project will lead to new efficient algorithms for constructing compressed indexes. The new approach lies in first preprocessing the input using lightly sub-optimal compression and then constructing the final index in compressed time, i.e., time proportional to the precompressed text. Second, the project aims to utilize and improve modern suffix sampling techniques to design new and powerful indexes that are both compressed and able to support powerful queries, including multi-string representations. Finally, the project will study the underdeveloped landscape of lower bounds for compressed indexes. Currently, only the most basic queries, such as random access, are well-understood, and much less is known about lower bounds on more versatile representations or lower bounds for compressed computation. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.