Synergistic Integration of Statistical and Logic-based Reasoning for Adversarial Learning Mechanisms
Abstract
Synergistic Integration of Statistical and Logic-based Reasoning for Adversarial Learning MechanismsProject Summary: We are living in an era of machine learning (ML) and artificial intelligence, with applications in image recognition [47], natural language processing [23], spam detection [24], autonomous vehicles [22, 62] and malware detection [26, 77]. However, ML is increasingly being deployed in adversarial scenarios, where an attacker stands to gain from the failure of a ML system. The question then arises: are ML systems secure in adversarial settings? Previous work has demonstrated and exploited vulnerabilities in ML systems across multiple applications, datasets, and ML classifiers, including deep neural networks. However, attempts to design defenses against such attacks have been heuristic and empirical, focusing on specific classifierfamilies or application domains [34, 37, 39, 53, 73, 89]. Further, the existing defenses provide improved security only against existing attacks in the literature, and it is unclear if the defense mechanisms will be effective against adversaries with knowledge of their existence, i.e. strategic attacks exploiting weaknesses in the defenses. The science of adversarial machine learning is an open problem.Intellectual Merit: In this project, we aim to develop a scientific framework for understanding adversarial learning problems. Our key vision is to combine statistical reasoning with formal reasoning for secure adversarial learning. First, we will establish a formal framework for statistical learning that characterizes adversarial learning scenarios and defines their important performancemetrics. We will search for fundamental limits to learnability in the presence of an adversary using new perspectives from optimization and Bayesian reasoning. Our approach will characterize the adversarial conditions under which learning is still possible and quantify the effects of strategic evasion and poisoning attacks. Second, we will explore multi-layered defenses against strategic adversaries that take advantage of our understanding of the fundamental tradeoffs to build resilience into the training phase. Finally, we will evaluate these defensive strategies against state-of-the-art attacks proposed by the security community and optimal attacks where ever possible.Broader Impacts: Our proposal seeks to develop defenses for adversarial learning scenarios with the potential for widespread application. While the security community has developed a collection of impressive attacks against learning systems, the theoretic groundwork that is required to facilitate effective defenses with rigorous security guarantees does not yet exist. We propose thedevelopment of a scientific/formal framework that connects adversarial learning to the other fundamental problems of statistical learning. Our proposed work can have broad impact on any learning application where data comes from self-interested, intelligent users and has the potential to spur significant academic research, military research, and naval systems development. In summary, ourwork directly furthers the ability of naval forces to operate in untrusted computing environments.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 26, 2018
- Source ID
- N000141812454
Entities
People
- Prateek Mittal
Organizations
- Office of Naval Research
- Trustees of Princeton University
- United States Navy