Spectral Characterization of Variable Precision Approximation in the Context of Geophysical Fluid Dynamics
Abstract
APPROVED FOR PUBLIC RELEASEVariable Precision (VP) computing methods aim to approximate the dynamics of a precise mathematical model by intelligently allocating bits to optimize the trade-off between accuracy and costly resource usage (e.g., energy or computing cy cles), where saved resources can be reinvested elsewhere in a larger computation. Despite its importance, currently there is no prin cipled under standing of the nature and quality of the approximation provided by VP methods. The impact of additional bits of nume rical precision in such simulations is unclear, and several important questions remain unanswered:How does pruning or reducing bit precision impact the nature of the approximate dynamics and the quality of the computed solution?How does precision impact approxim ation quality and performance when used in conjunction with machine-learned surrogate models (e.g., neural networks)?How does the i mpact of a adding a bit to the state representation differ from adding abit to the parameter representation in an ODE/PDE system? Ho w should a computational scientist adjudicate between the two alternatives?To answer these questions and provide a rigorous characte rization of VP, we propose to recast VP approximations as truncated basis expansions, with approximations for both the basis coeffic ients (the state representation) and basis function parameters. Thus, the role of each bit of precision in both the problem represen tation and its solution is elucidated.This approach has two main benefits: (i) it unifies different kinds of VP approximations into a single framework, enabling a clearer understanding of the design options and more intelligent budget allocation decisions; and (ii ) it naturally integrates machine-learned neural networks, an emerging class of powerful but poorly understood approximation models, with more established analytical approximation methods. The latter became possible only recently, fueled by recent advances in the understanding of NN approximations (some by the present authors Sahs et al. (2020)). This body of work has started to lift the veil around NNs, demonstrating that they too can be understood as truncated basis expansions. In contrast to the knowledge-directed appro ach, NN basis functions are learned from observational data, rather than derived a priori from domain knowledge.More generally, such a perspective empowers decisions about which basis functions to use, enabling the integration of principled domain knowledge and ob servational data into a single unified framework. We propose to demonstrate the latter by developing a novel Harmonic Analysis techn ique that we call Laplace Series: a principled class of basis function approximations that generalizes Fourier Series and is strongl y informed by both the recent success of NNs and domain knowledge in our application area of interest: numerical weather prediction for geophysical fluid dynamical (GFD) systems.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 07, 2021
- Source ID
- N000142112908
Entities
People
- Ankit Patel
Organizations
- Office of Naval Research
- Rice University
- United States Navy