AI Nets: Predicting Actions and Inferring Intentions of Groups of Targets with a Network of Surveillance Robots
Abstract
In this project we seek to develop algorithms to enable a network of heterogeneous robotic sensing agents to perceive and predict rich, networked interactions among dynamic targets in the environment. This will advance the US Navy s capabilities in situational awareness by enabling a network of surveillance robots to intelligently determine the roles and relationshipsbetween targets in a group, predict their actions, and infer their group intentions. This capability requires advancements in three key areas, comprising the three thrusts of this project, as described below. Thrust I: Hierarchical Activity Map Using Coresets. Firstly, we will deal with the data deluge from the various sources of streaming information on board the sensing agents. We willdevelop algorithms to summarize and compress diverse, high data-rate sensor streams from e.g. cameras, Lidar, and Radar, as well as information from typical sources such as GPS and an IMU. These algorithms will be based on corsets, a tool that enables compression of high volume perception data into only the most salient frames with provably bounded information loss. We will use corsets to maintain a hierarchical target activity map fusing semantic, geometric, and dynamic features of the targets. The map will allow for informationto be autonomously summarized at appropriate levels of granularity for communication to neighboring agents, and for use in algorithms for inference and informative planning. Thrust II: Determining Group Roles, Predicting Groups Actions, Inferring Group Intentions. Secondly, the robots will use this hierarchical activity map to extract relationships among the multiple targets in the environment, including leadership roles, coalitions, rivalries, and overall intentions of the group. We will develop algorithms to learn the correlateddynamics of the group of targets, represented as a combined graphical model and dynamical model. We will learn the relationships between the targets as latent variables in the graphical model. We then will develop algorithms based on these learned models to predict the future actions of the group. Going beyond prediction, we will determine the underlying intentions of the group by inferring the policy they are executing (imitation learning), or the cost function they are optimizing (inverse reinforcement learning). Thrust III: Distributed, Asynchronous Planning for Information Gathering. Thirdly, we will develop distributed planning algorithms for the network of sensing robots to gather more information about the targets to gather information, e.g. to pursue moving targets, to reposition sensors to see occluded targets, and to tack the multiple targets. We will use anonline MPC based paradigm for each sensing robot to plan and re-plan information gathering trajectories based on the information in the hierarchical target activity map from Thrust I and the group dynamical models from Thrust II.The project includes an experimental plan to verify in robotic hardware the algorithms developed in each of the three thrusts, culminating in a demonstration of three thrusts operating together to drive a robot network in a multi-target surveillance application.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 19, 2018
- Source ID
- N000141812830
Entities
People
- Mac Schwager
Organizations
- Office of Naval Research
- Stanford University
- United States Navy