Benchmarking and Computational framework for Optimal Visualization and Interpretability of high-dimensional separable Data

National Institute Of Environmental Health Sciences

Investigators

Linked publications, trials & patents

Abstract

Optimization and benchmarking of data reduction methods for dynamic visualization and interpretation (DVI) present several challenges due to many factors, including data complexity, lack of ground truth, time-dependent metrics, dimensionality bias, different visual maps of the same data by different methods, and scalability. Current benchmarking studies primarily focus on independently static visualization or interpretability metrics, whose performance depends on knowing the ground truth. Developing robust evaluation methodologies that capture the intricacies of dynamic data and interpreting results in a meaningful context after data reduction is crucial for deeper insights into complex biological systems. In this context, we recently submitted a manuscript which is under review on a proposed method called Multivariate Interpretable Benchmarking and COmputational framework for optimal VIsualization of high-dimensional Stochastic data without ground truth (MIBCOVIS). MIBCOVIS optimizes the visualization and interpretability of high-dimensional dynamic data without ground truth. It integrates five robust metrics, including a novel time-ordered Markov-based structure metric, into a semi-supervised hierarchical Bayesian model. The framework evaluates method accuracy performance, considering the interaction effects of all metric features. We apply MIBCOVIS to linear, nonlinear, and artificial neural network dimensionality reduction methods, visualizing distinct dynamic biological processes at the single-cell level. Unlike traditional approaches that rely on single-summary scores, MIBCOVIS compares entire accuracy distributions across data reduction methods. Our findings underscore the importance of jointly evaluating visualization and interpretability, instead of using independent metrics. We show that relying on average performance could maximize performance but obscures individual method feature performance. Additionally, we explore the impact of increasing data complexity on visualization and interpretability performance, accounting for factors like feature dimensionality, unknown cell types, and dynamic differences in three biological processes; EMT, spermatogenesis and induced pluripotent stem cell differentiation. We provide optimal parameter regions, features, and methods, including an optimized Variational Contractive Autoencoder (oVAE), for targeted and untargeted DVI in various data complexity scenarios. We believe that MIBCOVIS holds promise as a tool for evaluating dynamic single-cell atlases and spatiotemporal data reduction models in general.