Improving Neural Networks with (and for) Computational Physics (Tracking number: 21-000000177)
Abstract
Although having a long history of research and publication in computational physics, our group is strategically a member of the Stan ford Artificial Intelligence Laboratory (SAIL) and thus is fully immersed in cutting edge practices in machine learning and artifici al intelligence. It is important to emphasize that much of the progress in machine and deep learning has been in applications areas more germane to computer science disciplines than to traditional computational physics, e.g. robotics, computer vision, natural lang uage processing, etc. There are also valiant efforts to push forward on machine and deep learning in the computational physics (e.g. computational fluid dynamics and computational solid mechanics) communities, but many of these are not deeply rooted in the state-o f-the-art best practices from the computer science disciplines, and few researchers work in both areas as we do. Thus, at a high lev el, our work aims to more fully understand which aspects of deep and machine learning work well, which aspects work less well, and t o improve and adapt these state-of-the-art approaches to computational physics when applicable and appropriate. Broadly speaking , scientists and mathematicians have pushed our understanding of the physical world forward to a large degree; however, there are st ill gaps between what we understand and can predict versus what is observed in experimental data. Notably, machine and deep learning aspirationally provides a connection to bridge that gap (known as the domain gap) from the equations to the real world. This propos ed work will set out in that direction, leveraging what we know from the physics as priors and using neural networks to get the rest of the way there. Notably, some of the most popular neural networks approaches are almost entirely data driven, in large part becau se governing equations are not known for such problems; however, these networks do work well at all (in spite of claims to the contr ary) as can be shown by the ease at which they can be adversarially attacked. On the other hand, given the lack of prior knowledge o f governing equations, these net- works do perform better than the alternatives on the problems they were designed for. The same is not true for computational physics problems, and too many network driven approaches to such problems are either ignoring or devaluin g prior knowledge and the governing equations, instead of standing on their shoulders. Understanding how to leverage the governing e quations and numerical algorithms in a neural network approach that aims to reproduce real world observations lies at the heart of o ur approach. In particular, we aim to alleviate the burden placed on neural networks by leveraging mathematically sound numerica l approaches (woven throughout the network), to evaluate more and less essential aspects of neural network architectures during both training and inference providing a way forward on more accurate and robust network architecture design, and to take a more rigorous and robust approach to neural network training highlighting the flawed assumptions in back-propagation and the avoidable limitation s it induces on not only robustness and accuracy but also on the scope of problems that can be tackled.Approved for public release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 20, 2021
- Source ID
- N000142112771
Entities
People
- Ronald Fedkiw
Organizations
- Office of Naval Research
- Stanford University
- United States Navy