Adversarial Robustness in Real Environments
Abstract
While deep learning has demonstrated record breaking results in object recognition, skepticism about its robustness against adversarial examples is also becoming a concern. However, majority of the studies today, both theoretical and empirical, are based on analyzing toy problems using known datasets. Adversarial examples in the real physical space are not common. Even for the few ones available in the literature, they are difficult to make but easy to fail: If we change the lighting condition, capture the image at a longer distance, shake the camera, or simply use another classifier, these attacks will no longer work. The goal of this proposal is to challenge a fundamental question made in these studies: If we launch adversarial attack in the real physical environment, how much of the attack is still feasible? The answer to this question will result in two distinctive but important conclusions. (1) If the answer is negative, then adversarial attack could just be an artificial problem. (2) If the answer is positive, then we need to know what causes it to happen and how to defend. In particular, we want to know the roles of the environment, and to find ways to create an environment such that adversarial attacks can be prevented. This proposal outlines a pathway to address adversarial robustness under environmental influence. There are three objectives: (i) Develop a theoretical framework to certify adversarial robustness in the presence of environmental factors. (ii) Provide theoretical and experimental evidence to prove / disprove adversarial attack in real environment. (iii) Formulate a design framework to create an environment that prevents attack. To accomplish these objectives, we introduce a new type of problem called the environmental adversarial attack. Under this problem setting, we analyze how the environment affects the classifier, the attack, and both. Specifically, the analysis includes studying the geometry of the decision boundary, the inter-play between denoising and environment, and the trade-off in accuracy and robustness. In terms of evaluation, the proposal leverages computational imaging techniques to design new experiments and controllable environments to verify the theoretical findings. The intellectual merit of this proposal is the inclusion of environmental factors in studying adversarial robustness. By doing so, we expand the current literature by offering new insights about whether adversarial attack is indeed realistic. The implication is two-fold. On the one hand, the results will help us pin down the conditions under which the attacks will succeed / fail. This will in turn tell us the kinds of environments that would be easy / hard to attack. On the other hand, by treating environmental factors are a design variable we will be able to configure an environment that is robust against attack. The impact of this study is significant, as it informs machine learning practitioners when to trust and how much to trust a machine learning system. The conclusion of project will offer recommendations to designing robust computer vision systems, e.g., for surveillance, navigation, and automation. This abstract is publicly releasable.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 09, 2020
- Source ID
- W911NF2010179
Entities
People
- Stanley H. Chan
Organizations
- Army Contracting Command
- United States Army
- University of Virginia