Computation of Adversarial Manipulations Under Physical Access
Abstract
Neural networks are notoriously susceptible to adversarial perturbations, where small changes to an input cause drastic changes to the networks predictions. Previous work on adversarial perturbations assumes that an adversary can directly manipulate inputs to the neural network. We shift this assumption toone where the adversary can modify inputs only indirectly by manipulating an ambient physical environment. This assumption on an adversary's access to network inputs more realistically models the threat adversarial perturbations pose in many DOD applications. For concreteness, we focus on an application to autonomous sensing for passive sonar. In this setting, the ambient environment is measured with a physical sensor and then passed to a neural network for analysis. We investigate three state-of-the-art neural networks employed in this manner, each of which are able to correctly identify adversarial presence in at least 95% of the cases examined. Assuming the role of the adversary, we find perturbations to the physical environment that drop success rates to less than 50% for the same three networks. In so doing, we demonstrate that the lack of robustness in neural networks can be exploited to compromise physical sensors.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 01, 2023
- Accession Number
- AD1213689
Entities
People
- Austin J. Van Dellen
Organizations
- Naval Postgraduate School