III: Small: Collaborative Research: Learning Active Physics-Based Models from Data

University Of Utah, Salt Lake City UT

Investigators

Abstract

This project explores a novel algorithmic framework for automatic generation of digital models of objects from our natural world, that faithfully reproduce the structure and function of their physical counterparts. We specifically focus on modeling active deformable objects, i.e., objects capable of producing internal forces within their own bodies, such as biological muscles or robotic actuators. Our approach differs from the traditional modeling pipeline by learning the digital models from example data of the mechanism in-action, rather than by manually engineering them from the first principles. We adapt current state-of-the-art deep learning techniques to our problem, in particular artificial neural networks, by endowing them with knowledge about the physics-based behavior of deformable materials. This is expected to significantly upgrade the capabilities of generic neural networks, which would be otherwise forced to learn the laws of physics from data, which is an unnecessary task because fundamental properties of deformable media, such as conservation of energy and rotational invariance, should simply be taken for granted. The proposed algorithmic framework will greatly simplify the creation of digital replicas of objects in our natural world, while enhancing their fidelity. This will empower Virtual and Augmented Reality deployments to deliver life-like experiences in educational and skill-training applications, such as virtual operating rooms or emergency response scenarios. Computer-hosted doubles of functional objects are also a valuable prototyping tool in the design and optimization of physical functional replicas, such as prosthetic devices. To achieve these goals, we hybridize a neural network with a differentiable simulator, which outputs the quasistatic (i.e. equilibrated) shape of an active elastic model as a function of input control parameters, and subject to prescribed (known) boundary conditions. The finite element-based simulator is based on Projective Dynamics and designed with differentiability in mind, which is a key feature that will enable smooth combination with the classical backpropagation algorithm and integration within existing deep learning frameworks, such as PyTorch. The input to the simulator allows the actuation controls to be prescribed at very fine granularity, potentially enabling each finite element to become its own independently controllable actuator. These fine-grained actuation controls will be generated by a convolutional neural network, which creates them using a low-dimensional time-varying control vector and constant (i.e., time-invariant) network weights. We train this aggregate pipeline, jointly inferring both the weights of the control network as well as the values of the latent variables associated with different input configurations, as to best explain the training set as the action of a low-dimensional control space. This core framework will subsequently be extended to 1) allow for processing of contact and collisions, 2) optimization of spatially-varying material parameters, 3) lifting the quasi-statics assumption and simulating time-varying dynamics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.