FET: Small: Deep transformers for inferring gene regulatory networks from omics data

University Of Missouri-Columbia, Columbia MO

Investigators

Abstract

Huge volumes of omics data such as genomics, transcriptomics, and proteomics data have been generated by high-throughput sequencing experiments. Extracting biological knowledge regarding gene regulation mechanisms of cells from multiple sources of heterogeneous omics data is a significant computational challenge, which is critical for leveraging the data to address various biological problems such as how genes are regulated under different biological conditions and how genotypes (e.g., genetic mutations) cause phenotypes (e.g., physical features). The overarching goal of this project is to develop cutting-edge artificial intelligence (AI) methods based on transformers to infer gene regulatory relationships (i.e., gene regulatory networks) from omics data more accurately than before. The pretrained transformer tools can be broadly used to study gene regulation in various biological systems such as bacteria, plants, and animals. The publicly released course materials, web sites, videos, databases, software tools, tutorials, user manuals, and the online learning community on social networks will boost gene regulatory network modeling in bioinformatics and AI. The course modules and training activities will enrich both AI and bioinformatics education at middle/high school, undergraduate, and graduate levels. The combination of AI and omics data analysis will promote the diversity in AI, computing, and bioinformatics by attracting and training underrepresented minority and women students. The project will engage the local public and the Missouri State Legislature to advocate for the importance of scientific research for the society and economy. This project aims to achieve three specific objectives: (1) developing transformers to predict transcription factor binding sites on chromosomes/genomes from omics data, including genomics, transcriptomics, epigenomics, and protein sequence and structure data; (2) developing self-supervised graph transformers to infer gene regulatory networks by integrating omics data and transcription factor binding site predictions; and (3) develop graph transformers to infer gene regulatory networks from single-cell omics data via transfer learning. The first self-supervised learning-based transformers to tackle this problem will improve the accuracy of inferring gene regulatory networks over existing unsupervised methods. The self-supervised learning will overcome the weakness of existing supervised methods not effectively dealing with the problem of lacking labelled data. The transformers can predict entire gene regulatory networks as graphs, leveraging the inter-dependence between multiple transcription factor-gene regulations that the current supervised methods of predicting the regulatory relationship for one pair of transcription factor and gene a time cannot. Moreover, the graph transformers use uniform graph representations to integrate multiple modalities of omics data under the same graph framework, which are more scalable and effective than the existing methods that use different ways to process different data. Furthermore, the transformers can predict gene regulatory networks from the data of both single cells and a population of cells (bulk cells) and use transfer learning to fine-tune the models pretrained on bulk-cell data to improve the highly challenging single-cell gene regulatory network inference for the first time, which is likely much more effective than the existing methods of separating the gene regulation inference from bulk-cell and single-cell omics data as two independent problems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.