Efficient Algorithms for Solving Large-Scale and Complex Riemannian Optimization Problems Tracking number: 23-000004479
Abstract
Riemannian optimization has attracted a lot of attention due to its wide applications in signal processing,machine learning, communication, and statistics. Representative examples include lowrankmatrix completion, phase retrieval, optimal transport, blind deconvolution, and dictionarylearning. Existing research on Riemannian optimization mainly focuses on problems with three relativelysimple structures: smooth objective, single level, and single agent. Motivated by emergingapplications such as robust subspace recovery, neural architecture search, and model compressionin distributed networks, this project targets to designing new algorithms and developing new toolsfor large-scale Riemannian optimization with complex structures: non-smooth objective, bilevelstructure, and federated learning in distributed networks.For Riemannian optimization with non-smooth objective function, the PI will study how to designRiemannian alteranting direction method of multipliers (ADMM) for solving it. ADMM receivedhuge success in solving structured convex optimization problems. However, it has been not clearhow to design its Riemannian counterpart with convergence guarantees. The PI#s recent workshowed that it is possible to design a provably convergent Riemannian ADMM using the Moreauenvelop smoothing technique. In this project, the PI will address issues related to this approachsuch as how to apply homotopy technique to improve the numerical stability, and how to remove therequirement on Lipschitz smoothness of the function so that it can be applied to K-means clustering.Theultimate goal is to design Riemannian ADMM without using the smoothing technique so thatit can solve non-smooth problems directly. Riemannian optimization with bilevel structure findsapplications in hyperparameter selection of machine learning models such as neural architecturesearch. The bilevel structure brings new challenges due to its complex structure that the constraintinvolves another optimization problem. In this project, we will design new algorithms for solvingbilevel optimization on manifolds. We will start with simpler cases where only the upper levelproblem has a manifold constraint, or the case that lower level problem is geodescially convex.Federated learning has found many important applications due to the fact that smart-phone-APPsand wearable devices generate huge amount of data everyday, which requires to store data locallyand push more network computation to the edge. The main challenge of federated learning onmanifolds is how to make the aggregation step efficient. In this project, the PI will design newalgorithms for federated learning on manifolds. How to reduce the communication cost and howto effectively preserve data privacy will also be investigated in this project.If successful, the outcomes of this project include a set of new algorithms and tools for solvinglarge-scaleand complex Riemannian optimization problems. The new algorithms and tools will beused to solvesignal processing, communication, and machine learning applications arising frompractice that are of future naval relevance.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 09, 2024
- Source ID
- N000142412705
Entities
People
- Shiqian Ma
Organizations
- Office of Naval Research
- Rice University
- United States Navy