Interpretable End-to-End Streaming Inference in Multi-Agent Environments
Abstract
Imagine our goal is to extract interpretable and actionable information about entities within a dynamic environment. We use a running example of inferring the interaction structure between basketball players, which evolve as plays unfold. We assume observations are collected from a swarm of (possibly mobile) agents that collaborate in collecting data and drawing inferences. For aspects of both perception and planning, deep learning offers significant promises. These methods have incredible expressive power and handle a variety of data sources. Unfortunately, the gains in flexibility often come at a serious loss in interpretability, important to decision making, and likewise require massive training data and compute resources. It is unclear how to use these methods for integrated decision-making based on complex, streaming data, especially in the multi-agent scenario.We propose deep learning models that go beyond only prediction to get at notions of interpretability, cope with limited relevant data, and yield better generalizability to unseen data regimes (i.e., are less brittle). Our models handle nonlinear and non-stationary dynamics observed via massive collections of data streams of varying degrees of relevance, quality, and types. To be used in conjunction with our modeling advances, we devise streaming algorithms to analyze data arriving without bound from multiple sensing agents. These algorithms enable rapid adaptation to changing environments, and allowmultiple agents to simultaneously draw inferences and coordinate their actions.Our proposed work includes scientific developments along three major research threads:1. Interpretable Deep Learning for Complex Data Streams: We develop toolsfor interpretable and flexible structure learning from complex, evolving data streams based on potentially limited measurements of disparate types. A secondary benefit of our approach is better generalizability.2. Distributed Real-Time Inference in Multi-Agent Settings: We devise streaming Bayesian inference algorithms that adapt rapidly to changing environments andcoherently propagate uncertainty. These algorithms apply to a variety of modelingframeworks, including a wide class of expressive deep learning models.3. Collaborative Multi-Agent Sensing and Action Selection We propose distributed algorithms where agents communicate compact summaries of the world and coordinate in data collection; critically, our methods enable integrated perception (inference) and planning (action selection).The resulting methods enable a swarm of mobile agents to simultaneously collaborate in their actions (e.g., data collection) and on-the-fly inferences about complex, dynamic environments. Importantly, our approach is robust to common real-world challenges including the presence of massive, disparate data streams, communication constraints and errors, and agent attrition. As such, this research is an important step towards increasing the impact ofdata collected in surveillance tasks: We are creating tools to effectively collect and extract information from networks of advanced autonomous agents developed and deployed over recent years. Through the proposed technology, we believe we can positively impact the goal of Information Dominance in creating an integrated information and decision-making space.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 19, 2018
- Source ID
- N000141812862
Entities
People
- Emily B. Fox
Organizations
- Office of Naval Research
- United States Navy
- University of Washington