ONR White Paper entitled "Embedding Transformation Networks: Toward Modular and Robust Multi-Purpose Agents" (Tracking Number not received)
Abstract
Machine learning (ML) and Artificial Intelligence (AI) have had a dramatic impact in many domains including speech recognition, object recognition, and game play. One significant challenge with modern ML approaches is the failure to learn representations that adapt to continually changing environments. This is a critical need for many military-relevant operations, where it is impossible to anticipate the situations an AI agent will encounter once deployed.These limitations are substantial, but are not shared by humans. We are able to deal with environmental changes to rapidly learn new skills while retaining old ones and to use the same learned representations in a variety of situations. Our research objective is to develop algorithms that model the robustness of humans as they encounter a variety of tasks. Specifically, we will construct algorithms centered around an approach that maps embeddings representing new tasks to previously learned tasks. We refer to the enabling mechanism as an Embedding Transformation Network. Our approach is comprised of two key innovations: (1) transformation networks that map changing inputs to a common representation where existing action policies can then be used, and (2) a metalearning framework that not only learns how to solve tasks, but also learns how to learn new tasks as a blended combination of existing tasks. We believe that these innovations will result in critical improvements in forward transfer of related tasks, allowing a new policy to be initialized to a state that will demand drastically fewer training examples to migrate to maximum performance.We will first demonstrate the efficacy of our approach in a relatively simple, Atari game-like environment. Our ultimate objective, however, is to evaluate our algorithms on Navy-relevant scenarios that represent far more complex situations. For this purpose, we have selected two environments/scenarios that meet this need: maritime platform defense and maritime satellite reconnaissance. Both of these operational scenarios, while being incompatible with frequent retraining, require dynamic adaptation and forward transfer of skills. In the case of platform defense, we intend to use embedding transformation networks to avoid re-training scheduling agents in thepresence of changing fleet or threat characteristics. In the case of satellite reconnaissance, we intend to improve the robustness of search agents to the variations in weather and lighting conditions that currently limit their utility in maritime missions. We anticipate that these two applications will provide a blueprint for transfer and continual learning in Naval systems.ONR White Paper entitled "Embedding Transformation Networks: Toward Modular and Robust Multi-Purpose Agents" (Tracking Number not
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 29, 2020
- Source ID
- N000142012239
Entities
People
- Jared Markowitz
Organizations
- Johns Hopkins University
- Office of Naval Research
- United States Navy