Optimization, Federated learning, and high dimensional statistics for large-scale machine learning
Abstract
We propose a bold multidisciplinary research plan that aims to establish the foundations for a unified theory of overparametrization and deep learning, with specific focus on efficient optimization and federated implementation of learning algorithms and adversarial robustness. Our investigation will focus on understanding the prediction accuracy of methods that have large capacity to fit the training data, and attempts to explain why in the overparameterized regime that Deep Learning systems operate, data interpolation and zero training loss does not lead to over-fitting. The premise of our proposed effort is that understanding the theoretical aspects of modern machine learning algorithms is a vital step before one can reliably deploy these models in highrisk applications such as designing self-driving cars and building healthcare systems. To this end, our project employs advances in high dimensional statistics, statistical learning theory, and large scale optimization to develop an explanation for unreasonable effectiveness for deep learning systems, and to explain why and how overprameterized learning methods do not suffer from overfitting in certain settings. We also investigate the new emerging topic of federated learning, according to which computations required for learning a statistical model are distributed across multiple machines. Furthermore, we will expand on our recent approach for study of acceleration in non-convex settings. Finally, our project will try to develop a rigorous theory for General Adversarial Networks and robust learning. Our project includes two PIs with a strong track record and a long history of successful collaboration on prior ONR projects. The PIs have coadvised doctoral students and postdocs in the past and this project will be instrumental in training new graduate students and postdoctoral scholarsin optimization theory, statistical learning theory and high dimensional statistics, and the theory of adversarial robustness.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 29, 2020
- Source ID
- N000142012394
Entities
People
- Ali Jadbabaie
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy