Learning to Compose Data-Driven Models with Knowledge
Abstract
The overarching goal of the proposed research is to bridge the large gap between exploiting domain knowledge and building data-driven learning models, by developing principled approaches to compose structured and interpretable data-driven models. The project will investigate two composition forms that can capture wide data characteristics and application interests: the plain additive composition, and the hierarchical composition. The former is applicable to signals that can be represented as an additive summation of multiple physically meaningful bases, such as radar, biomedical and physiological signals. The latter is applicable to semantically richer, high-dimensional data such as image, video or multi-media data, whose base concepts may display complicated interconnection and will naturally form semantic hierarchies. Following the compositionality principle, the general methodology will start by constructing a pool of knowledge-based primitives, which preserves domain-specific explainability while having learnable parameters. Those primitives are then selected and composed into a learning model, where the composition strategy is driven by knowledge (e.g., domain-specific logics, or the ontology) and aided by learning. Data-driven updates are eventually performed to optimize the predictive power of the composed model, by tuning both primitives learnable parameters and their weights/coefficients. The interplay between knowledge-driven and data-driven methodologies will naturally motivate novel semi-parametric models and algorithms, whose optimization algorithms and other properties will be thoroughly studied. The technical outcomes will be evaluated using concrete real-world applications, ranging from physiological signal processing to visual object recognition and multi-modal event recognition.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 09, 2020
- Source ID
- W911NF2010240
Entities
People
- Zhangyang Wang
Organizations
- Army Contracting Command
- United States Army
- University of Texas at Austin