Parallelization and Interactive Documentation to Maximize the Accessibility of Harmony

Brigham And Women'S Hospital, Boston MA

Investigators

Abstract

PROJECT SUMMARY As the availability of large datasets in biomedicine continues to grow, researchers face the challenge of integrating these datasets to unlock their full potential. However, combining datasets is challenging due to technical confounders, also known as batch effects. Batch effects can emerge in all domains of science and engineering, so the need to correct for technical artefacts is widespread. The Harmony algorithm was originally developed to address correct for batch effects in single cell RNAseq (scRNAseq) data, enabling robust, scalable integration of diverse datasets. Its speed, accuracy, and usability have led to its adoption in platforms like Seurat and ScanPy, with over 5,000 citations and 195,000 CRAN downloads. However, as datasets surpass 100 million cells and new technologies generate even larger datasets, significant updates are needed to scale Harmony for modern use cases, such as multimodal and spatial transcriptomics (ST) data integration. Scaling Harmony requires moving beyond single-core implementations. Parallelization is feasible due to Harmony's modular nature and reliance on matrix operations. In Aim 1 we will Refactor Harmony to scale to 1 billion cells through parallelization using two strategies, multicore parallelization and GPU implementation. The first strategy will allow us to take advantage of multiple cores to achieve a speed-up proportional to the number of available units. For the second approach we will implement GPU support using MAGMA to maximize performance on various hardware configurations, and we expect run times to be sped up by 1-2 orders of magnitude. In Aim 2 we will Interactive web-based documentation for Harmony best practices for both existing and emerging use cases. In our first subaim we will focus on addressing issues that are most commonly encountered by users. In the second subaim we will expand the documentation and the vignettes to cover more advanced use cases. Although Harmony was initially designed for integrating scRNAseq data, the model has proven to be generalizable and the software has been used in many other contexts. Here, we propose to carry out extensive testing for some of these use cases to establish best practices that can be communicated to users. Specific focus will be on the integration of spatial transcriptomics datasets with scRNAseq data. This resource will be made freely available as an interactive guide for troubleshooting batch integration issues. The improvements proposed here will enable researchers to integrate large, complex datasets across diverse modalities, maximizing the impact of Harmony in single-cell genomics and beyond. With a strong track record in algorithm development, benchmarking, and education, our team is well-positioned to implement these enhancements, ensuring Harmony remains accessible and efficient for the growing community of users.