Integrated Perception and Planning in Resilient Multi-Modal, Multi-Agent Networks
Abstract
Principal Investigator: Amit Roy-Chowdhury, University of California, RiversideThis project will focus on foundational principles to enable integrated sensing and analysis in large multi-modal, multi-agent networks with weak supervision, leading to reliable decision-making in complex, dynamic and uncertain environments while being resilient to adversarial interactions. This will be achieved through two broad research objectives: A) learning multi-agent, multi-modal data association models with limited supervision, and B) reliable decision-making in collaborative, decentralized teams. We will build upon existing low-level feature extraction and classification methods, and focus onthe high-level machine learning and decision making processes. Obj. A will be achieved through three tasks. First, we will show how data association modelsbetween multiple visual sensors can be learned with a small fraction of the labeling cost in current methods by taking into account the global characteristics of the multi-agent network (A1). Associations across modalities can also be learned with limited supervision by leveraging upon side information, possibly from the web; the challenge is to identify the most relevant side information (A2). The models need to be updated over time based on their performance. Information-theoretic methods for selecting the most informative samples will be used for model update, and sparse and low-rank structures inherent in the data will be exploited (A3). Obj. B will build upon the models learned in Obj. A to identify the portions most relevant to a task considering the correlations in the information between various sensors and modalities (B1). B2 will focus on identifying adversarial attacks on the data, first with known dynamical models offeature evolution to set performance bounds, followed by the more practical situation where models are only partially known. B3 closes the loop between perception and planning by acquiring the data that maximizes performance of the analysis modules. The proposed research builds upon recent advances in computer vision, machine learning, signal processing, robotics, and controls. The team of PIs reflects this blend of expertise. All the tasks will involve solid theoretical analysis and algorithm development focusing on optimizationstrategies, computational complexity, and performance bounds, and will be complemented with a rigorous evaluation phase for individual tasks, as well as their integration. Our existing facilities, along with equipment purchased through two new DURIP grants, will enable the experimental evaluation. The results will be presented in top-tier international conferences and journals, as well as in reports and presentations to the sponsor. The project directly addresses the need for understanding an environment from large volumes of heterogeneous streaming data collected by a team of autonomous agents. This has important Naval applications in field operations ranging from national security to disaster response. Our proposed methods will enable continuous and efficient learning of models that represent the data across sensors and modalities without the need for strong supervision, making decisions about the most task-relevant portions of the data along with detection of any adversarial attacks, and collaborative planning to acquire additional data to maximize the decision making accuracy. Overall, the project addresses some of the fundamental limitations of current autonomous systems by requiring only limited supervision, having the ability to identify relevant and reliable portions of large data volumes, and integrating the analysis and acquisition phases
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 25, 2019
- Source ID
- N000141912264
Entities
People
- Amit Roy Chowdhury
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents