Experimentation with Massive Computation and Comparison for Understanding Representation Learning
Abstract
We propose to investigate the principles and practices of learning good representations from data, a crucial component in applying machine learning algorithms to real-world applications. To this end, we undertake the proposed research in three components: (1) build hardware and software infrastructure (funding for the necessary equipment is requested through this proposal) that enables intensive computing for large-scale learning problems; (2) massively experiment with a diverse set of learning algorithms on several large-scale benchmark tasks; (3) study systematically and comparatively various representations obtained by those learning algorithms. The main idea is to pry into each other to understand what common properties good representations entail. In turn, those common properties will shed light on the design of generally applicable optimality criteria for deriving representations that are reusable and transferrable. Representations with such properties are essential to cope with new tasks or ne w conditions in a changing environment. As a proof of concept , our preliminary work on scaling up kernel methods to match deep neural networks has shown promising results. Despite being drastically different, the two methods learn strikingly similar representations with only subtle differences. We believe that a more thorough and extensive investigation in this vein, as outlined above, will bear more fruits in helping researchers and practitioners in understanding how to extract knowledge from data and how to use them to build robust and intelligent autonomous systems. Besides advances in representation learning, other potential impacts of the proposed research include faster and more efficient parallel and distributed optimization algorithms that benefit largescale learning, existing and new statistical learning methods that can scale up to large problems, active community participation through organized competitions in representation learning.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 11, 2018
- Source ID
- W911NF1610587
Entities
People
- Fei Sha
Organizations
- Army Contracting Command
- United States Army
- University of Southern California