Interpretable End-to-End Streaming Inference in Multi-Agent Environments
Abstract
Imagine our goal is to extract interpretable and actionable information about entities within a dynamic environment. We use a runnin,g example of inferring the interaction structure between basketball players, which evolve as plays unfold. We assume observations ar,e collected from a swarm of (possibly mobile) agents that collaborate in collecting data and drawing inferences. For aspects of both, perception and planning, deep learning offers significant promises. These methods have incredible expressive power and handle a var,ietyof data sources. Unfortunately, the gains in flexibility often come at a serious loss in interpretability, important to decision, making, and likewise require massive training data and compute resources. It is unclear how to use these methods for integrated dec,ision-making based on complex, streaming data, especially in the multi-agent scenario.We propose deep learning models that go beyond, only prediction to get at notions of interpretability, cope with limited relevant data, and yield better generalizability to unseen, data regimes (i.e., are less brittle). Our models handle nonlinear and non-stationary dynamics observed via massive collections of,data streams of varying degrees of relevance, quality, and types. To be used in conjunction with our modeling advances, we devise st,reaming algorithms to analyze data arriving without bound from multiple sensing agents. These algorithms enable rapid adaptation to,changing environments, and allow multiple agents to simultaneously draw inferences and coordinate their actions. Our proposed work i,ncludes scientific developments along three major research threads:1. Interpretable Deep Learning for Complex Data Streams: We devel,op tools for interpretable and flexible structure learning from complex, evolving data streams based on potentially limited measurem,ents of disparate types. A secondary bene?fit of our approach is better generalizability.2. Distributed Real-Time Inference in Multi,-Agent Settings: Wedevise streaming Bayesian inference algorithms that adapt rapidly to changing environments and coherently propaga,te uncertainty. These algorithms apply to a variety of modeling frameworks, including a wide class of expressive deep learning model,s.3. Collaborative Multi-Agent Sensing and Action Selection We propose distributed algorithms where agents communicate compact summa,ries of the world and coordinate in data collection; critically, our methods enable integrated perception (inference) and planning (,action selection).The resulting methods enable a swarm of mobile agents to simultaneously collaborate in their actions (e.g., data c,ollection) and on-the-fly inferences about complex, dynamic environments. Importantly, our approach is robust to common real-world c,hallenges including the presence of massive, disparate data streams, communication constraints and errors, and agent attrition. As s,uch, this research is an important step towards increasing the impact of data collected in surveillance tasks: We are creating tools, to effectively collect and extract information from networks of advanced autonomous agents developed and deployed over recent years,. Through the proposed technology, we believe we can positively impact the goal of Information Dominance in creating an integrated i,nformation and decision-making space.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 01, 2022
- Source ID
- N000142212110
Entities
People
- Emily J Fox
Organizations
- Office of Naval Research
- Stanford University
- United States Navy