Interactive Large-scale Multi-agent Planning with Natural Language Inputs and Explainable AI
Abstract
Approved for Public ReleaseThere is an immense need to build human-interactive multi-agent autonomy, where heterogeneous agents work, closely with humans, comprehend human commands, and execute them promptly, correctly, and safely. However, autonomous agents remain, absent from crucial military and commercial applications due to technical difficulties in effective human-autonomy interaction. In,this proposed project, we aim to fill this gap by developing an explainable human-interactive planning environment for multi-agent a,utonomy. The planning system consists of three major components: 1) a bi-directional nat,t can parse users commands into machine s formal language, and translate the planner s feedback into human language; 2) an automate,d self-explainable multi-agent planner that acts as the brain for autonomous systems to analyze, validate, and execute commands reli,ably, and provides feedback and advice to human users; and 3) an interactive explanation framework and systematic evaluation to alig,n the autonomous agents inferred specifications with the human users intentions, and assess the human users understanding of the,learned specifications. We believe these systems can work synergistically such that they not only enable easy communication between,human users and the planner, but also open up opportunities to carry out more complex and critical tasks.We use Temporal Logic (TL),as a mathematically precise language to specify multi-agent planning tasks. TL can capture the complex spatial, temporal, and logica,l requirements in both human languages and environmental constraints, and can be transformed into actions of executable agents. How,ever, there is no reliable way to perform automated translations between TL and natural language (NL) across application domains, du,e to the ambiguity and variability of human language. We propose to fulfill this critical gap by developing a generalizable NL-TL tr,anslation frameworkenabled by deep transfer learning. More importantly, this translation framework will be seamlessly integrated wit,h the back-end automated planner, as well as an interactive explanation framework for users to comprehend and predict the behavior o,f black-box translation systems. Human users will be able to receive feedback generated at multiple levels of the planning process t,o clear out the ambiguity and potential flaws in human inputs. Our interactive planning and explanation framework and the associated, interface will be tested and iteratively improved for usability through human subject experiments.The proposed project brings toget,her NLU, formal methods, explainable AI, human-robot interaction, human factors, and multi-agent planning, to provide both theoretic,al underpinnings and practical framework of developing a human-interactive planning system for multi-agent autonomy. If successful,,it will accelerate the positive role of multi-agent autonomy in society by creating a computational foundation that will make human-,autonomy teams exceptionally more effective beyond the current state-of-the-art. The proposed project will last 36 months, and the a,nticipated outcomes include datasets, NLU models, human-interactive interface, software implementation, technical reports, and confe,rence paper submissions.Our project aims to advance the foundational science of autonomy and has a strong future naval relevance. If, successful, our system can support naval operations across many sectors, including but not limited to regular military duties, fiel,d operations, combat-ready response deployment, search and rescue, and advanced manufacturing.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 13, 2022
- Source ID
- N000142212578
Entities
People
- Chuchu Fan
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy