Tractable Deep Learning: Structure vs. Scale in Data
Abstract
Many well-established problems in data science, such as data classification and clustering, raise unprecedented challenges in the presence of diverse, complex and high dimensional data. We propose to investigate how the representation of data is impacted given its inherent characteristics and how that, in turn, impacts its utilization in applications. Our goal is to build on established understanding of mathematical properties associated with data and more recent accomplishments in Machine Learning and data science, to develop a strategy of analysis trading off structure versus deep scale. Our focus on inference problems is, however not limiting as the proposed representation strategies are equally valid for other applications with minimal modifications. Specifically, we propose a so-called Artificial Neural Fiber (ANF) (i.e., a continuum of neural computational elements) as our basic computational element, and to which we associate a differential model. We subsequently propose to develop a tractable framework of Deep Learning by establishing an equivalence of the new ANF model with an Artificial Neural Network. We maintain that this equivalence to ANN provides a natural transition to the analysis of Deep Learning Networks by suitably extending the depth of ANN. We subsequently propose to develop a full analytical study of Deep Learning by investigating the associated computational/numerical as well as its statistical behavior. To further strengthen the formalism, we put forth for the first time, a strategy to establish a Universal Approximation Property for finite size layers of an asymptotically deep network. The resulting outcome of this analysis is the development of a Deep Learning Network which forgoes the back-propagation, hence promising to lift the problems of convergence which are well known to affect Deep Neural Networks. A Generative Adversarial Network topology is finally proposed for different applications, with a promise of lifting existing challenges and guaranteeing performance.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 01, 2019
- Source ID
- W911NF1910202
Entities
People
- Chau-wai Wong
Organizations
- Army Contracting Command
- North Carolina State University
- United States Army