CAREER: Symmetries and Classical Physics in Machine Learning for Science and Engineering

Johns Hopkins University, Baltimore MD

Investigators

Abstract

The description of physical theories in terms of their symmetries –and the transformation rules of coordinate freedom– played a fundamental role in important developments in physics, including the discovery of general relativity. In modern machine learning, symmetries are key to the design of deep learning architectures: From the translation symmetry of convolutional neural networks; to the permutation symmetry of graph neural networks; to transformers, which are, in principle, permutation equivariant. This project, inspired by physics principles, develops new mathematical and computational techniques to further exploit symmetries and differential geometry in the design of machine learning models. In particular, it will focus on representation learning and physics emulation on point clouds and vector fields. The developed techniques will be applied to problems in cosmology and climate science in collaboration with physicists at New York University. The project involves PhD students from Johns Hopkins and high school student interns from Baltimore City public schools. It also includes activities to promote research in Latin America, and community-building activities for women in math and engineering. The project's first aim is to improve representation learning techniques that embed data such as text or images in a latent space in a self-supervised fashion. Based on recent work that introduced an algebraic structure in the embedding space through approximate group equivariance, the developed methods will enable users to translate interpretable modifications to the input data into linear transformations in the embedding space. This will refine the usability of the learned embeddings by providing a causal structure to the learned representations. We achieve this implicitly, using invariant theory, and explicitly, by learning a special (disentangled) coordinate system with differential geometry techniques. The project's second aim is to develop coordinate-free emulation methods for cosmology and climate science. One approach is to implement algorithms for point clouds that are invariant with respect to permutations and orthogonal (or Lorentz) transformations, on which n-body simulations can be built. In another approach, machine learning methods are built for vector and tensor fields, based on geometric principles from modern classical physics, discretized onto image grids. Success in these projects will lead to more accurate emulation with fewer expensive full-resolution simulations for the training sets. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.