Deep Learning: Integrating Domain Knowledge and Interpreting the Network Decisions
Abstract
Problem DescriptionThe proliferation of big and complex data has led to new developments and widespread use of machine learning models, especially the deep learning" models. Their model complexity and expressiveness result in non-linear boundaries. This has lead to huge success of deep networks and their adoption in areas such as computer vision and natural language processing. One major challenge of deep learning is the requirement of large training data to learn the very large numbers of parameters. However, generating enough training data for deep learning models may not be feasible in many applications due to its cost, large collection time, and availability of only small amounts of data in some domains. For example, in a time-sensitive decision-making domains of interest to ONR, a high-performing deep learning model is desired to capture complicated patterns in data, but collecting massive training data can be prohibitive for this specic task. Fortunately, in many learning problems, various types of domain knowledge can be explored and exploited to overcome the challenges due to model complexity and overtting.Proposed ApproachIn this project, we propose a principled approach to integrate domain knowledge in the lifecycle of deep learning and eectively reduce the model complexity and thereby training robust and accurate deep models using the limited amount of training data available. The proposed approach includes three major tasks: (i) integrate data knowledge from auxiliary data sources to revise the formulation of deep learning, in the form of knowledge-dened structural regularization or constraints on the parametric space; (ii) integrate model knowledge, where we exploit the decision surfaces from simpler models on the same task to guide the learning of the deep model, which eectively reduces the model complexity; (iii) integrate optimizerknowledge, which seeks to improve the optimization procedure of the training of deep models. By iden-tifying similar learning tasks and observing their gradient trajectories, the optimizer itself can be trained to provide faster convergence and also avoid poor local optimal solutions. The proposed domain knowledge integration framework is widely applicable to most deep learning tasks, and we plan to evaluate the frame-work on publicly available datasets in healthcare and computer vision. A byproduct of integrating domain knowledge will be to impart interpretability or explainability to the network decision making, a much desired capability which is currently lacking.SignicanceThe proposed research is signicant in the following ways: (i) We provide a principled framework to leverage domain knowledge, which is expected to lead to signicant improvements in the eciency of training deeplearning models, which will also have higher generalization performance on challenging learning tasks. (ii) Besides the training of deep learning, the proposed optimizer knowledge integration can be used in most optimization problems and is expected to advance the broader eld of optimization theory. (iii) The domain knowledge integration will bring interpretability to deep models, which is critical to Navy decision-making problems. (iv) The proposed methodology has a broad range of applications, not only for naval intelligence analysis, but also medical informatics, social network analysis, and recommender systems.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 29, 2020
- Source ID
- N000142012382
Entities
People
- Anil K. Jain
Organizations
- Michigan State University
- Office of Naval Research
- United States Navy