Towards Efficient (Deep) Representation Learning Through Nonconvex Optimization and Low-dimensional
Abstract
*APPROVED FOR PUBLIC RELEASE*Office of Naval Research (ONR), Machine Learning, Reasoning & Intelligence.Approved for Public Release., Submitted to the ONR Long Range BAA N00014-22-S-B001 Program under Dr. Behzad Kamgar-Parsi.As data we are collecting today are dram,atically increasing in volume and dimension, the performances of modern machine learning methods crucially rely on the particular ch,oice of data representation. Over the past few decades, we have witnessed the revolution of (deep) representation learning, mainly d,riven by data and computation. However, the underlying principles behind its empirical success largely remain a mystery. One of the,major challenges is due to the nonlinearity of our learning models, resulting in highly nonconvex optimization problems which could,be NP-hard in the worst case. Nonetheless,various empirical evidence suggests that often the symmetric properties of the problemand,intrinsic low-dimensional structures of the data often alleviate the hardness of theseproblems, that simple heuristic nonconvex opti,mization methods often work surprisingly well for learning succinct and even deep hierarchical representations. In this proposal, we, aim to bridge the gap between practice and theory of modern representation learning methods, by studying the geometric properties o,f optimization landscapes and exploiting the lowdimensional structures of the data. The new geometric insights and data structure wi,ll not only explain what kind of representations can be learned through optimization, but also shed light on the principled design o,f better learning models and the development of efficient, globally convergent algorithms. Leveraging this framework, we will study,a wide spectrum of representation learning problems ranging from supervised, self-supervised, to unsupervised learning, and we will,study the robustness and transferability of these learning procedures through understandings of the learned representations.Impact/r,elevance of the Proposed Work to Navy/DoD. The proposed work will contribute to the development of the foundations of machine intell,igence (namely, data acquisition and representation, reasoning, planning), and efficient computational methods with rigorous guarant,ees. This project aims to enhance Navy/DoD capabilities in machine intelligence by developing efficient representation learning meth,ods with improved robustness and efficiency, and with guaranteed correctness.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 13, 2022
- Source ID
- N000142212529
Entities
People
- Qing Qu
Organizations
- Board of Regents of the University of Michigan
- Office of Naval Research
- United States Navy