Plan-recognition-Driven Attention Models for Supporting Active Perception in Decentralized Multi-Agent Systems
Abstract
Many Naval applications employ multiple autonomous agents (possibly working with human team members) that collaborate to perceive and process task-specific data, overcome environmental obstacles and threats from hostile agents, and accomplish a pre-defined mission. Due to the constraints of the environment, each agent can only partially perceive its surroundings. In general, limited bandwidth for communication would make it unlikely for one agent to access all data streams (especially videos) from other agents. Furthermore, while sensing technologies may provide abundant data streams for the agents, existing methods do not enable each agent to perform sophisticated processing tasks in a timely manner. However, each agent needs to make decision and take action, not only advancing itself towards the goal, but also facilitating other team members in achieving the common goal of the team. Hence many inter-related, sophisticated processing tasks are required. All these pose practical challenges in a multi-agent system. Central to these challenges is the need for new perception and planning methods that work hand-in-hand to allow the collaborating agents to rapidly understand a dynamic environment and take proper actions advancing to the goal.This project aims at addressing several key research challenges in such applications.Specifically, we propose to develop new and critical methodologies for integration of perception and planning/plan-recognition in order to support a multi-agent team navigating through complex environments in accomplishing a challenging mission. Our key approach is to develop new neural attention models for task-aware perception that are driven by planning and plan-recognition (as opposed to by only low-level visual saliency, for example) so as to support efficient processing of sensory data (sifting through many redundant or irrelevant information streams) and active perception (acquiring what is needed for the processing and planning tasks). The task-aware perception models will in turn facilitate joint perception and planning tasks including multi-agent plan recognition from video inputs, a key task in many applications with imaging sensors. The anticipated outcome of the project includes new visual attention models that facilitate high-level reasoning such as planning, new visual representations of video that better support semantic analysis, and planning/plan-recognition approaches grounded on real sensory inputs. These new models have the potential of significantly enhancing and expanding the capabilities of existing multi-agent systems in DoD applications.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 25, 2019
- Source ID
- N000141912119
Entities
People
- Baoxin Li
Organizations
- Arizona State University
- Office of Naval Research
- United States Navy