GPU Cluster for Physics-Informed Neural Networks (PINNs) to Support MURI Research and Beyond
Abstract
Scientific machine learning is lacking behind the rapid developments in other domains due to the lack of appropriate algorithms for physical applications. To this end, the PIÕs group has pioneered the technique of physics informed neural networks (PINNs). PINNs can blend seamlessly data and simulation, and can solve ill-posed problems otherwise not possible using traditional methods; for example, PINNs can generate flow fields from Schlieren photography only, remove the tyranny of the grid generation, and can readily solve inverse problems regardless of the complexity. KarniadakisÕs group has developed several variants of PINNs for different applications: For example, conservative PINNs (cPINNs) are based on domain decomposition for multiscale problems, enabling large datase efficient parallelism. Parareal PINNs (PPINNs) are suitable for long-time integration and enable parallelism via temporal domain decomposition. Fractional PINNs (fPINNs) solve fractional derivatives for non-local modeling. However, all these algorithms have been developed for a single GPU and they are inefficient as we scale up the problem to bigger domains and longer simulation times. We need urgently a multi-GPU version of PINNs and their variants so that we can tackle open realistic applications of interest to DoD. However, unlike multi-CPU parallel computing, multi-GPU parallelism requires modification of the algorithms, e.g. different learning rates and, in general, different hyperparameter tuning for which very little is currently known. Moreover, this optimization is often application dependent. We have started developing such multi-GPU PINN algorithms albeit on a very small number of GPUs (up to 4) due to the lack of a platform for experimentation. Hence, in this proposal we request funds to purchase a system with more than 100 GPUs to be able to support research in two MURIs that the PI leads and one DARPA project, all of which make use of PINNs for diverse applications. The first MURI, which we will refer to as F-MURI, supports research for Fractional Partial Differential Equations (FPDEs) and we are currently submitting an extension proposal to tackle various applications using fPINNs. The second MURI, which we will refer to as P-MURI, is a new start and will focus on solving PDEs using neural networks and other general networks with applications to non-destructive evaluation of materials and to hypersonics research. The DARPA project aims to build a multiscale/multiphyics general framework, ÒDeepM&M", with specific applications to supersonic transitional boundary layer. We plan to first develop a robust multi-GPU version in collaboration with our NVIDIA collaborators and subsequently to tackle the following problems: (1) direct simulation of turbulence including particulate transport; (2) hypersonic transitional and turbulent boundary layers; (3) fractional largeeddy simulations at high Reynolds number flows; (4) non-destructive evaluation of polycrystaline nickel. In addition, we will explore more risky topics, such as accelerating the training of very deep neural networks via fractional gradients employed in multiphysics applications of PINNs. The impact of the proposed work is tremendous as we will develop the first generation of multi-GPU PINNs, which can be applied across all application domains. PINNs have already been adopted by all national labs and main industry (ANSYS, NVIDIA, SIEMENS), and we expect that a multi-GPU version will propel PINNs and large-scale simulation to the next level of tackling realistic applications at scale. On the education front, in collaboration with our NVIDIA partners, we will organize tutorials and webinars in order to train the next generation of graduate students and postdocs, essentially in GP-GPU programming for neural networks, which is a rare skill in academia but also in industry.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 25, 2021
- Source ID
- W911NF2110089
Entities
People
- George Karniadakis
Organizations
- Army Contracting Command
- Brown University
- United States Army