Neural Galerkin Schemes with Active Learning for High-Dimensional Wave Propagation
Abstract
APPROVED FOR PUBLIC RELEASESolutions of partial differential equations (PDEs) are increasingly parametrized by deep networks because, of their approximation capabilities in high dimensions; however, training deep networks, i.e., numerically constructing an approxim,ation rather than just theoretically guaranteeing the existence of one, remains truly challenging in terms of computational costs an,d requirements on training data. The training challenge is further amplified for problems with wavetype dynamics and local support i,n potentially high-dimensional spaces, where an un-informed sampling of training collocation points at which to minimize the PDE res,idual to fit network parameters suffers from the curse of dimensionality; similarly to finding the peak of a concentrated function i,n a high-dimensional space. In stark contrast, this project proposes adaptive Langevin and Stein sampling strategies for selecting t,raining points at which to minimize the PDE residual to fit networks in a self-informed manner that is guided by the dynamics descri,bed by the PDE. The proposed approach builds on Neural Galerkin schemes that parametrize only the spatial domain alone via deep netw,orks and treat time separately and then propagate forward in time the solution in an adaptive fashion via classical time-integration, schemes. In that sense, Neural Galerkin schemes can be seen as merging traditional numerical analysis techniques with machine learn,ing. Propagating forward the network parameters step by step in time enables the proposed adaptive sampling schemes to use informati,on from previous time steps to efficiently estimate the residual for training the network parameters over time, rather than having t,o discover network parameters globally over the time-space domain. Additionally, by using information from previous time steps, the,proposed techniques adapt the network architectures to dynamically increase and decrease the number of neurons depending on the requ,ired expressiveness. The project will show that such an active form of gathering training samples and adapting architectures is key,for numerically realizing the approximation capabilities of networks in high dimensions. Furthermore, the proposed approaches and th,eir analyses will be demonstrated on long-range wave propagation problems and high-dimensional particle systems that exhibit local d,ynamics, which will show the potential of the proposed work to impact a wide range of applications of relevance to the Navy mission.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 08, 2022
- Source ID
- N000142212728
Entities
People
- Benjamin Peherstorfer
Organizations
- New York University
- Office of Naval Research
- United States Navy