Mathematics for Learning Nonlinear Generalizations of Subspace Models in High Dimensions
Abstract
A fundamental and predominant modem statistical signal processing challenge is performing low-dimensional modeling of high-dimensional data. A great deal of recent work has demonstrated the applicability of the linear subspace model in this context with sensing constraints and computational constraints. Linear measurement models have been thoroughly studied for similar reasons, as well as statistically independent random measurement models. While these linear models have extensive mathematical foundations, they are both limited in their ability to describe interesting nonlinearities in signals and measurements that are often highly relevant for data-driven decision making. On the other hand, nonlinear generalizations are often problem-specific, each requiring very specific mathematical tools. In this proposal we focus on broad generalizations of the linear subspace model to nonlinear signal and measurement models, developing the foundational mathematics, algorithms, and algorithmic theory for identifying these models in high dimensions. We will specifically consider three novel contexts for low-dimensional modeling of high-dimensional data, all of which generalize the highly successful low-rank linear subspace model but capture interesting nonlinear structure as well. 1) Correlation and nonlinearities in the measurement system: Measurements are often modeled as a linear operator applied to the true signal of interest, with subsampling or compression modeled as random and independent. However, many real measurement systems have nonlinear calibration relationships that are not always known a priori. 2) Algebraic variety models for relationships in the data: The variety model supposes that the data belong to the common zero set of an unknown system of polynomials. If those polynomials were linear this reduces to the subspace model. 3) Atomic-set models with nonlinear measurements: The atomic-set model is a very recent generalization of the sparsity model leveraged widely in compressed sensing and sparse regression. We will study these structured signals when observed through unknown nonlinear monotonic functions. There are many examples of complex systems relevant to the Army for which these issues are central and which motivate the research in this proposal. Video surveillance systems can be used for real-time scene construction and analysis. Linear models can provably handle many of the relevant issues such as failures in feature identification in the video or correspondence across cameras or frames, accurate camera calibration, and memory- and computation-limitations, but their fidelity is coarse and can be improved greatly by allowing for nonlinearities in the system. Another important security task is in monitoring and managing large computer networks, especially as our society relies continually more on the Internet and a network s computing power. Detailed measurements are collected on networks in order to diagnose problems, but temporary link failures lead to missing data, and often the data are only used post-hoc in order to understand what already happened. Again, linear models have limited modeling capacity for such a complex system. All of these measurement systems are not deployed to simply collect measurements but to facilitate inference or to aid in analysis and decision making. Thus the task at hand is to understand, within the nonlinear models that we study, what are the mathematical tradeoffs of collecting more measurements at different quality levels, using processing resources, and getting more accurate results within time constraints and uncertainty specifications. Recent advances in statistical signal processing, computational linear algebra, optimization, and machine learning are allowing us to develop inference algorithms that both have provable guarantees and that use low-dimensional structure to rigorously exploit the redundancy that is nearly inevitable in heavily measured systems.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 14, 2019
- Source ID
- W911NF1910027
Entities
People
- Laura Balzano
Organizations
- Army Contracting Command
- United States Army
- University of Michigan