Toward an actionable theory of deep learning
Abstract
In the last few years, we have witnessed stunning developments in Artificial Intelligence. Problems which had previously seemed farout of reach have suddenly been solved or nearly solved. Latest AI systems can pass Stochastic Aptitude Test (SAT), the bar exam, solve undergraduate level math and physics problems, translate images to LaTeX, or HTML and complete a multitude of other tasks. It is clear that AI will be permanent technology and society of the future. Nevertheless, while modern AI models are relatively simple mathematical formulas, we do not have an understanding of why, when and how they work.The objective of the proposed research is to bridge this gap in our understanding of modern deep learning techniques. Specifically, we aim to develop a theoretical framework that elucidates how contemporary neural networks process and interpret high-dimensional data. Our approach builds on the recent work of the PI Belkin and collaborators on the Neural Feature Ansatz, which provides a new perspective on feature learning in neural networks. By building on this foundation, we intend to explore and devise innovative machine learning methodologies that do not rely on traditional backpropagation techniques.Anticipated outcomes of this research include a deeper theoretical understanding of the mechanisms driving current deep learning models, improvements of the training processes of these models, and the potential development of newmachine learning algorithms that are more efficient in terms of sample usage. Through this work, we expect to make significant contributions to the field of AI, paving the way for more robust, efficient, and understandable AI systems in the future.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 09, 2024
- Source ID
- N000142412631
Entities
People
- Mikhail A. Belkin
Organizations
- Office of Naval Research
- United States Navy
- University of California, San Diego