Domain-knowledge Hybridized Statistical Machine Learning (DHSML)
Abstract
We propose a novel class of domain-knowledge hybridized statistical machine learning (DHSML) systems, that explicitly encode and leverage domain knowledge. We consider domain knowledge of three types: (a) relational knowledge, ranging from a set of an available set of relations among entities in the system, either explicitly or implicitly via an algorithm, to semantic relationships among entities encoded by knowledge graphs, (b) logical knowledge, ranging over identities, inequalities, invariances over the entities in the system, and more generally, logical rules that encode relationships among entities in the system (that might not always hold), and lastly, (c) scientific knowledge, represented via partial differential equations (PDEs) that describe the dynamical system behaviors of components in the system, typically derived from scientific theories. There are two key ingredients in any SML system: (a) the model architecture, which specifies a family of models that one could learn from data, and (b) the learning algorithm that selects a model from the given model class that best fits the data. In our proposed class of DHSML systems, we propose surgeries to each of these two key ingredients in order to make them amenable to incorporate varied types of domain knowledge. In integrating relational knowledge, we propose a novel deep learning architecture that trains semantic representations based on such relational information in a supervised manner; together with an approach to use such relational knowledge to inform long-term memory in sequence-based models. In integrating logical knowledge, we propose novel deep learning architectures that use regularizations based on logical formulae, together with some modifications to be robust to mis-specified logical knowledge. In integrating scientific knowledge, we use a novel deep learning architecture to incorporate differential equation based constraints on higher order derivatives of the learned function comprising the dynamical system, together with a novel multi-scale stochastic differential equation system, as well as soft invariance incorporating SML system.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 19, 2018
- Source ID
- N000141812861
Entities
People
- Pradeep Ravikumar
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy