Unsupervised Nonlinear Multimodal Component Analysis:Theory and Optimization
Abstract
Essential to the ArmyÕs Multi-Domain Operations (MOD) are gathering and analyzing intelligence from heterogeneous and complex sources across land, sea, air, and cyberspace. The operation- supporting artificial intelligence (AI) systems are expected to learn and extract actionable information from these domains. Multiview analysis (also known as multimodal learning) is a technique that serves for extracting common and essential information from different views (or modalities) of data samplesÑe.g., data representation acquired bydifferent sensors. Linear multiview analysis tools suchas canonical correlation analysis (CCA) have been the workhorses of unsupervised multiview analysis for decades, but the expressive power of linear tools is very limited. In recent years, deep neural network (NN)-based unsupervised multiview analysis has proven remarkably effective by extensive empirical evidence. However, unlike linear multiview tools whose algebraic and numerical properties have been well understood, nonlinear multiview analysis has been largely intuition-driven, making it difficult to advance this direction with principled design and rigorous analysis. In addition, existing learning algorithms associated with these NN-based approaches heavily rely on off-the-shelf deep learning toolboxes, which are often incapable of flexibly incorporating constraints/regularization to meet the demands of multiview analysis. Most of these algorithms also lack convergence guarantees. To aÀÀÀÀÀÀessthese challenges, this project will leverage the PIÕs recent research on theory-backed unsupervised nonlinear multiview component analysis to develop advanced comprehensive analytical and computational tools for future multimodal learning systems. The project will develop along the following directions: ¥ Thrust (I) Nonlinear Multiview Analysis Theory develops comprehensive component identification theory that supports and guides NN-based multiview learning design under challenging and realistic scenarios. ¥ Thrust (II) Flexible and Scalable Algorithms offers a unified computational framework that facilitates fast, distributed, and flexible unsupervised multiview machine learning tasks. ¥ Thrust (III) Applications and Evaluation evaluates and validates the proposed framework with real-life multiview analysis tasks that are integral to the ArmyÕs missions, e.g., multimodal brain imaging for brain computer interface (BCI) and multiview social network analysis for covert activity discovery.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 25, 2021
- Source ID
- W911NF2110227
Entities
People
- Xiao Fu
Organizations
- Army Contracting Command
- Oregon State University
- United States Army