Error quantification and complexity limits of deep learning models
Abstract
This project is motivated by the great success of deep learning and the need for a theoretical understanding of the models involved in this field. Generally speaking, the aim of the project is to contribute towards the understanding of deep learning from a theoretical viewpoint.As a first research direction, we propose to characterize the quality of the training step for certain classes of functions in terms of the topology of the energy landscape. The quality of the training step can also be related to the generalization error of deep learning models. A goal of this project is the derivation of worst case and/or average case error bounds. Besides the need for worst case error bounds, it is also relevant to better understand the ability of these networks, as new algorithmic tools, in terms of their computational capability. In particular, we are interested in studying the computational limits of deep learning models, in comparison with classical techniques. Can the use of deep learning reduce the sample complexity of certain inverse problems? How do the computational limits of deep learning compare with classical algorithms like spectral methods, sum of squares or approximate message passing for classical problems like max-cut, minimum bisection and quadratic assignment?This project aims to build on the current understanding of deep learning from the optimization and statistical viewpoints. The majority of the funds requested in the project will be used to hire student interns and train them through two workshops, where top experts in the field will be explaining the state of the art statistical physics techniques that apply to the study of deep learning. The PIs and the students will then work together on the aforementioned research problems and the results will be presented at various conferences in machine learning.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 24, 2019
- Source ID
- FA95501817007
Entities
People
- Augustin Cosse
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- École Normale Supérieure