Physical Access to Autonomous Systems: Adversarial Manipulation, Robustness, and Real-Time Computation
Abstract
Approved for Public ReleaseResearch Problem: In many Department of Defense (DoD) applications, autonomous agents operate in environments where opponents seek to undermine their abilities. In other applications, the US and its allies may themselves seek to undermine autonomous agents belonging to an adversary. This problem has generated a flurry of research in recent years, resulting in techniques for adversarial manipulation, which can be used by an attacker to imperceptibly modify the inputs to a predictive model onboardan autonomous system (the defender) such that the model produces erroneous results. Because autonomous systems are often intended to operate in environments outside of human intervention, the extreme lack of robustness demonstrated by models sensitive to adversarial manipulation undermines their usefulness in naval and military operations. Though adversarial manipulations suggest that manypredictive models are sensitive to sabotage by a capable adversary, the actual assumptions on the adversary#s capabilities are prohibitively strong. In adversarial learning, researchers primarily distinguish between #white box# and #black box# adversaries, which captures the adversary#s (potential lack of) knowledge of the predictive model#s details. We introduce a further distinction based on the adversary#s access. The term logical access describes a situation where the adversary possesses the ability to directly modifyinputs to a predictive model. Physical access describes an adversary who possesses the ability to modify the physical environment the defender occupies, but not the predictive model itself. Logical access is often an unrealistic assumption, since it essentially requires an adversary to have complete access to an autonomous system#s onboard data processing. Previous research in adversarial manipulation focuses on the logical access assumption, whereas the physical access assumption more accurately portrays realistic vulnerabilities autonomoussystems possess. The physical access assumption is also more challenging because it requires modeling defender and adversarial agents while incorporating properties of the physical environment through which they inter-act. For example, in acoustic and radio frequency (RF) signal processing, both of which are important applications areas for the US Navy, the received signal depends crucially on the ambient environment as well as the propagated signal, so assessing the vulnerability of autonomous systems to adversarial manipulation should be conducted using a physical access assumption. In response to the challenges presented by understanding and defending against realistic threats in adversarial autonomy, we propose to develop foundational mathematical and statistical tools for adversarial autonomy with a physical access assumption. We will also develop predictive models which are robust against these physical access manipulations, and validate their performance on naval relevant data sets. Our cross-disciplinary researchteam#s combined expertise in statistical learning, operations research, physical modeling, and autonomy, especially with regards tothe operational needs of the DoD, makes us uniquely qualified to accomplish the proposed research.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 15, 2024
- Source ID
- N000142412318
Entities
People
- Johannes Royset
Organizations
- Office of Naval Research
- United States Navy
- University of Southern California