Stochastic Algorithms for Data-Driven Constrained Continuous Optimization
Abstract
The aim of this project is to significantly advance the state of the art in terms of the design, analysis, and implementation of algorithms for solving constrained continuous optimization problems with data-driven objective functions and, importantly, data-drivenconstraints involving potentially nonlinear and/or nonconvex functions. Our emphasis on data-driven constraints means that we are focusing on the design of computationally efficient algorithms for solving problems involving so-called hard constraints. This is in contrast to the typical soft-constraint/regularization paradigm that, while popular in machine learning, has several disadvantagesincluding that it (a) necessitates extremely computationally expensive hyperparameter tuning and (b) might, even with such tuning, lead to solutions that fail to satisfy constraints accurately. This latter feature is particularly disadvantageous when the constraints intend to ensure critical properties such as the enforcement of physical laws.Building on our investigators highly successful ONR-funded project on the design of stochastic algorithms for solving data-driven optimization problems with deterministic constraints, this project focuses on the design of adaptive Newton-type algorithms (based on the sequential quadratic optimization and interior-point methodologies) for solving problems with data-driven constraints. The project will focus on three main thrusts as well as additional efforts related to the evaluation and testing of the proposed algorithms on test problems inspired by real-world applications, such as data-driven trajectory prediction. One thrust will focus on the design and analysis of adaptive Newton-type algorithms for solving problems where the objective is defined by an expectation and the constraints are that the expected value of a function satisfies a certain set of equations and/or inequalities. A second thrust will focus on the design and analysis of adaptive and progressive sampling-based algorithms for solving problems involving more robust types of constraint formulations, such as to ensure that a given constraint is satisfied with high probability. The third thrust, inspired by state-of-the-art deterministic algorithmsfor solving constrained continuous optimization problems, will focus on computationally efficient means of moving beyond first-order algorithms for the other two thrusts, and will involve the design and analysis of techniques for employing Lagrange multiplier andHessian-of-the-Lagrangian estimators.This project charters a new course in the design of algorithms for solving next-generation (e.g., physics-informed machine learning) problems with data-driven constraints. Our work addresses various areas of interest of ONR s Mathematics, Computer and Information Sciences (MCIS) Division, specifically within the Mathematical and Resource Optimization program, due to our focus on the design, analysis, and implementation of algorithms for solving large-scale, stochastic, and nonlinear optimization problems.Approved for Public Release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 09, 2024
- Source ID
- N000142412703
Entities
People
- Frank E. Curtis
Organizations
- Lehigh University
- Office of Naval Research
- United States Navy