Exploiting symmetry for Learning Spatiotemporal Dynamics
Abstract
Objectives: The overall objective of this project is to develop a novel deep learning (DL) framework based on symmetry: an integrated approach that combines first-principles with data for learning complex spatiotemporal dynamics. There are two major challenges that we propose to tackle: - Challenge 1: Accommodating imperfect symmetries in real-world dynamics: We propose to design novel approaches that can exploit symmetry as an inductive bias to improve learning. Unlike previous works that assume perfect symmetry, we will work with more realistic settings that allow weak forms of symmetry constraints including partial symmetry, approximate symmetry, and probabilistic symmetry. We will develop equivariant neural networks without stringent symmetry constraints using techniques such as symmetry disentanglement and relaxed weight-sharing. - Challenge 2: Discovering unknown symmetries from spatiotemporal data: Given rich spatiotemporal sensing data with unknown symmetries, we will develop DL frameworks that can automatically discover symmetry inductive biases from the data. Specifically, we will investigate adversarial training algorithms to discover the invariant sets from dynamical systems, Lie algebra convolutional networks to discover symmetry groups, as well as sparse regularization techniques to discover the intrinsic dimensions for efficient representation. We will work with video and trajectory data from cameras, LiDar and navigational devices, using our symmetry-aware DL models as basic building blocks for dynamics learning and decision-making. We will demonstrate the practical values of our framework to solve challenging tasks including trajectory forecasting, motion planning, surrogate modeling and video prediction. Intellectual Merit: Introducing physical constraints into data-driven machine learning is central to enable next generation AI methods that are adaptive, efficient, interpretable, and trustworthy. While most DL methods are designed for static images or discrete texts with little or no physics, our framework will lead to DL innovations for sensing data that are dynamic and continuous, and are governed by physical processes, transforming vast spatiotemporal data into actionable insights in real time. Our work contributes to the emerging field of physics-guided AI, and focuses on the fundamental concept of symmetry. Symmetry as an inductive bias are essential to the success of deep learning. Our method would improve learning performance without stringent constraints. By expanding to imperfect and unknown symmetries, our framework would significantly improve the applicability of symmetry-aware DL to the real world. Moreover, our tools can also discover unknown symmetries from the data, contributing to a deeper understanding of engineering and science problems. Broader Impact: As motivating applications, this research will lead to novel DL approaches that can emulate turbulent flow much faster than the traditional numerical solvers; trajectory and motion forecasting algorithms that are robust and sample efficient; and physically meaningful video prediction models for diverse scenarios. Ultimately, our framework would enable a new generation of AI methods that can efficiently learn, reason, act and adapt in complex, multi-modal, highly dynamic environments. It would contribute to the fundamental knowledge of AI in understanding and reasoning in the spatiotemporal environment, significantly enhancing our ability to derive knowledge from real-time sensing data and strengthening future army capabilities and systems.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 28, 2023
- Source ID
- W911NF2310231
Entities
People
- Rose Yu
Organizations
- Army Contracting Command
- United States Army
- University of California, San Diego