Active learning of nonlinear operators for forecasting extreme and rare events
Abstract
Our goal is the development of algorithms capable of representing and predicting rare extreme events occurring in complex dynamical systems using only scarce, but carefully chosen, data-points produced by an accurate (and possibly expensive) model or experiment. The problems we have in mind include i) dynamical systems driven by random excitations, such as the development of hot spots in a pandemic driven by social factors and interactions or extreme loads and motions on ships subjected to random waves; and ii) dynamical systems with intrinsic instabilities such as catastrophic precipitation events in a climate system or flow separation events occurring in vortex-induced-vibration problems. For all these systems modern machine- learning methods have very limited capability as the phenomena of interest are typically transient, i.e. they ‘live’ away from the statistical steady state of the chaotic attractor, and therefore have very low frequency of occurrence in an arbitrarily chosen data-set. This feature combined with the fact that the majority of machine-learning schemes have non-guaranteed generalization properties leads to limited applicability to problems related to rare catastrophic events. Our idea is to utilize recent developments on machine learned representations of operators (in contrast to functions) and train those using output-weighted active sampling methods. Specifically, while the machine learning community has made tremendous strides in the past 15 years by capitalizing on the neural network (NN) universal function approximation, and building a plethora of innovative networks with good generalization properties for diverse applications. However, it has ignored an even more powerful theoretical result that neural networks can in fact approximate functionals with arbitrarily good accuracy! A further step to nonlinear functional mapping (from a space of functions into real numbers) is nonlinear operator mapping. This is essential for the short-term prediction of complex dynamical systems, whether these are driven by external functions or intrinsic instabilities (in the second case the input to the prediction operator is the past of the system state, i.e. also a function). On the other hand, while machine-learning operators is an appealing idea, it has to be emphasized that the input space is a functional space, i.e. an infinite-dimensional one. Therefore, when one is interested to achieve good prediction accuracy independently of the frequency of occurrence of a particular state, they have to carefully select or optimize the training data-set. To achieve this objective we will follow the paradigm of active learning, whereby existing samples of a black-box function are utilized to optimize the next most informative sample. However, standard active learning methods rely on acquisition functions based on mutual information that have important limitations both in terms of applicability but also effectiveness. We will utilize a new class of acquisition functions for sample selection that leads to faster convergence in applications related to statistical quantification of rare events. The proposed method aims to take advantage of the fact that some directions of input functional space have a larger impact on the output than others, which is important especially for systems exhibiting rare and extreme events.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 02, 2021
- Source ID
- HR00112110002
Entities
People
- Marissa Clarkson
Organizations
- Defense Advanced Research Projects Agency
- Massachusetts Institute of Technology