Demonstrating Attacks and Defenses on Autonomous Platforms Driven by Deep Networks
Abstract
This DURIP project builds on an ARO-funded research project, in which the PIs are developing methodologies to defend against vulnerabilities of deep neural networks (DNNs), the state-of-theart machine learning (ML) approach, to both test/inference and training time attacks. Since the training time backdoor (i.e., BadNets) problem formulation considers a stronger adversary (versus the adversarial case), we view the backdoor detection and mitigation problem as a superset of the adversarial input detection problem. This DURIP requests funds for setting up real-world experimental platforms (autonomous and robotic systems) to demonstrate efficacy of defenses that improve resiliency of DNNs. Additionally, these testbeds will help explore adaptive systems learning online reacting to uncertainties, failures, or attacks. To address DNN vulnerabilities, our approach detects presence of adversarial perturbations or triggers (backdoors) and recovers correct output labels for poisoned/adversarial inputs. The approach comprises of offline and online phases. The offline phase learns new models based on the BadNetÕs behavior but using only clean validation inputs and also learns an anomaly detector based on hidden layer activations. The online phase uses the offline trained models to probabilistically detect poisoned/adversarial inputs. The online phase also addresses learning of a classifier to distinguish between validation and quarantined samples and a transformation from validation to quarantined data distribution. This learned transformation is used to fine-tune the DNN online and reduce the adversaryÕs success rate over time. While the current ARO-funded project includes testing of the efficacy of our approach on public data sets with backdoors and other injected adversarial perturbations, this DURIP seeks to demonstrate the adversarial robustness methods being developed on real-world cyber-physical systems (CPS) such as robots and unmanned autonomous vehicles to further validate efficacy of our methodology. In CPS, DNN-based approaches are increasingly popular both for environment perception and end-to-end control. These systems utilize multimodal sensor suites (e.g., cameras, LIDARs) that are fused in real-time using DNN-based approaches. Therefore, investigating DNN vulnerabilities and mitigations is of crucial importance for CPS. Hence, this DURIP proposes an integrated experimental testbed comprising: (i) a stationary 7 degree-of-freedom (DOF) robotic arm supporting direct control of joints; (ii) a mobile wheeled ground robot (with integrated camera and LIDAR) carrying a 7DOF robotic arm identical to the arm mounted on the stationary base (with all sensors); (iii) a high-definition true-color LED monitor to display adversarial images/videos; (iv) a motion capture system for high-accuracy tracking of 6 DOF states of the links of the robotic arms and dynamic objects in the operating environment as a ground truth. The testbed represents a realistic (albeit scaled-down) CPS, for example, smart warehouses and industrial automation. It enables experimental studies of adversarial scenarios (and defenses) via a variety of attack vectors including training-time data poisoning and inference-time adversarial perturbations in single and multi-robot/agent contexts as well as applications involving sensor fusion. Besides validating the methods in the on-going ARO-funded project, the testbed will promote experimental studies on online and active learning. It will be a unique experimental facility for the ML/Robotics community by providing access to (i) state-of-the-art robotic systems with multimodal sensor suites and ML-based end-to-end control and perception systems; (ii) sensor datasets collected in complex dynamic scenarios; (iii) showcase future research in resilient online/adaptive learning systems for autonomous platforms.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 03, 2022
- Source ID
- W911NF2210028
Entities
People
- Farshad Khorrami
Organizations
- Army Contracting Command
- New York University
- United States Army