High-performance Computational Methods for Nonlinear Machine Learning Problems
Abstract
Project Summary (Approved for Public Release):With the widespread availability of many computing and sensor technologies, optimization/computation has become the backbone of machine learning, which supplies us with powerful techniques to extract useful knowledge from data. In particular, computation has played a central role in recent advances in artificial intelligence (AI), such as deep learning, reinforcement learning and statistical learning. Nonlinearity is ubiquitous in machine learning, and most nonlinear machine learning problems correspond to non-convex optimization problems. Due to non-convexity, finding the best or correct solution for a nonlinear machine learning problem would need an enormous amount of time using standard optimization techniques. Hence, the AI community has been relying on a variety of heuristic methods to handle non-convexity, which work under strict assumptions and cannot be applied to safety-critical systems and mission-critical systems. Currently, human decisions are considered more reliable than AI-baseddecisions for such systems since even a low-probability error could be catastrophic. By designing efficient computational methods for AI with theoretical guarantees replacing the existing heuristic and ad-hoc techniques, we greatly expand the applications of AI to complex real-world systems and DoD problems.A major open problem in machine learning is on understanding when a non-convex optimization problem modeling a nonlinear machine learning problem can be solved using low-complexity algorithms (e.g., stochastic gradientdescent). There are many recent works on this fundamental problem, but they require strong conditions and only apply to narrow classes of problems. To address this issue, this proposal aims to develop a rich mathematical foundation for solving nonlinear learning problems both fast and with theoretical solution guarantees. By building upon our recent results published in the machine learning and optimization venues, we address six objectives: (1) studying what properties are needed to guarantee that smooth and nonlinear machine learning problems can be solved efficiently using low-complexity local search methods, (2) generalization of the results of Objectives 1 to non-smooth optimization problems in machine learning, (3) studying how to model nonlinear machine learning problems asoptimization problems with favorable properties (such as being free of spurious solutions), (4) studying how to solve high-complexity nonlinear machine learning problems via a sequence of low-complexity non-convex optimization problems, (5) development of a sequential penalized conic optimization technique for nonlinear machine learning problems, (6) performing case studies on real-world systems and benchmark problems in deep learning, statistical learning, reinforcement learning and others, especially those with Navy applications. This project equips AI with reliable computational methods and has a significant impact on the usability of AI for safety-critical systems. This project is interdisciplinary, requiring expertise in machine learning, optimization theory, algorithms, graph theory, and algebraic geometry.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 05, 2021
- Source ID
- N000142112731
Entities
People
- Somayeh Sojoudi
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents