Using Neural Networks In Constrained Optimization Problems
Abstract
Neural Network (NN) models are typically set up as unconstrained optimization problems with a single objective. However, with NNs seeing widespread adoption across a variety of decision systems, there has been a growth in the need to produce NN models that satisfy important system-specific performance constraints in addition to their primary objective. We consider binary classification in which one must tightly control the false alarm rate while minimizing the false negative rate. Formulating and training NN models in a constrained setting is challenging since most constrained optimization algorithms are not well suited for this setting because the constraint and objective functions are nonconvex, stochastic, and involve potentially millions of parameters. We utilize a new variation of Stochastic Gradient Descent (SGD) called Cooperative-Stochastic Gradient Descent (C-SGD) in an attempt to solve this challenging optimization problem. Application of the C-SGD algorithm is not straightforward, and we explore the effect of its many hyperparameters on performance and efficiency. Overall, we find that C-SGD can be made effective with the right choice of hyperparameters.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 01, 2019
- Accession Number
- AD1080317
Entities
People
- Jan Lim
Organizations
- Naval Postgraduate School