Closed-Loop Data Transcription via MiniMaxing Rate Reduction
Abstract
In this project, we propose to develop and study a new principled computational framework for automatically learning a closed-loop t,ranscription between a multi-class multi-dimensional data distribution and a linear discriminative representation (LDR) in the featu,re space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding, and decoding mappings sought can be formulated as the equilib-rium point of a closed-loop two-player minimax game between the encod,er and decoder. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for, distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop erro,r feedback from control sys-tems and avoids expensive evaluating and minimizing approximated distances between arbi-trary distributi,ons in either the data space or the feature space.To a large extent, this new formulation unifies the concepts and benefits of Auto-,Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for mult,i-class and multi-dimensional real-world data. Preliminary experiments on many benchmark imagery datasets demonstrate tremendous pot,ential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performa,nce of the encoder is competitive and often better than existing methods based on GAN, VAE, or a combination of both. We notice that, the so learned features of dif-ferent classes are explicitly mapped onto approximately independent subspaces in the feature space;,and the principal components within each subspace seem to automatically disentangle different visual attributes of each class of obj,ects.This project aims to systematically develop a rigorous mathematical theory for this new learning framework including optimality, conditions for theproposed minimax game. In this new framework, deep networks play a natural role as a means to parameterize the ma,ppings between the nonlinear submanifolds and the linear subsapces. We will study principled ap-proach to derive white-box network a,rchitectures from the rate reduction principles for data transcription. Moreover, we aim to extend this new framework to robust infe,rence tasks as well as the incremental learning settings.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 14, 2022
- Source ID
- N000142212102
Entities
People
- Yi Ma
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents