Hierarchical Abstractions for Sequential Decision-Making with Applications to Stochastic Games
Abstract
Project AbstractApproved for Public ReleaseThe ability to #abstract,# that is, to focus on the details that are task-relevant, and remove those that are not, is considered as the cornerstone of human intelligence. However, traditionally, this process has always been handled rather heuristically, relying on human experts to provide domain-specific knowledge to guide the construction of these abstractions. The main objective of this research is to develop principled approaches, guided by information theory, for the generation of task-relevant abstractions for decision making, perception, and planning for intelligent autonomous systems, and utilize theseabstractions for solving challenging decision-making problems involving several adversarial agents in uncertainenvironments. Adversarial decision-making will be formulated as a stochastic (e.g., Markov) game, the numerical solution of which is, however, known to be computationally challenging. Realizing this challenge, we will depart from the requirement of exact optimality and will seek instead tractable solutions based on hierarchical decomposition of the original problem, guided by the information theoretic abstractionmechanisms developed as part of this work. By seeking tractable decompositions of the original stochastic game (either spatially ortemporally) to a sequence of smaller games it will be possible to solve them more efficiently. We will also investigate temporal (fast/slow) decoupling of value iterations for the efficient solution of stochastic games within the context of multi-agent RL problems. The theory will be experimentally validated using small robotic ground vehicles of various scales to demonstrate the ability of on-board decision-making with limited resources.Understanding the world environment, including prioritization of what is relevant andwhat is not (i.e., #abstracting# the essential, salient information), is crucial to achieve long term autonomy. When an autonomous agent operates in anadversarial environment, it is also imperative to understand the intentions of the opponent(s) so that not be #outsmarted.# The proposed research will have an immediate impact on adversarial interaction between teams of agents of variable rationality, a common scenario in most military encounters. Coming up with tractable formulations to solve such adversarial decision making problems in the presence of uncertainty will go a long way towards making the theory relevant to Navy applications in the field. The proposed approach also provides a principled manner to better explain how humans make decisions and how to design autonomous systems that can interpret human intent more reliably.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 03, 2023
- Source ID
- N000142312308
Entities
People
- Panagiotis Tsiotras
Organizations
- Georgia Tech Research Corporation
- Office of Naval Research
- United States Navy