Autonomous Agents that Make Contextual-Based Decision Through Associative Learning
Abstract
The proposed research focuses on the development and evaluation of a biologically inspired architecture that gives autonomous agents the capability of contextual-based decision making through incremental learning of intent from human intelligence officers. This project addresses the DOD’s need for developing autonomous agent (AA) that operates independently across C4ISR applications. The objective of this research is for the AA to initially train with an intelligence officer and after finishing this training, it will function independently completing missions such as intelligence gathering, surveillance and reconnaissance. This type of autonomy requires the AA to be able to observe and interpret the intent of the officer while making decisions based on visual input data. In this research we propose the development of an architecture for associative learning that is inspired by the mirror neuron theory of intent understanding. Associative learning is the process of connecting sensory input (observations of the environment) to an action or behavior. Applied to this research, we will use associative learning as a framework for continuously training AAs on how to make contextual decisions based on observations and interpretations of the intelligence officer’s intent when making decisions. During observation the AA learns the association between sensory input and the action of the officer. To create an appropriate association, a supervised machine learning algorithm, specifically a convolution neural network and long-term short-term neural network, is used to generate textual descriptions of the visual environment in vector form. This vector is then used as input into a hierarchical self-organizing network which learns the spatial-temporal mapping between the action (the officer’s decision) and the visual environment so that it can execute the command independently when presented with a specific context experienced previously. The evaluation of our proposed architecture will occur through a set of experiments where the AA observes humans (intelligence officers) deciding on the emotion/sentiment surrounding a video clip (visual input sequence). After observing (training) the human for a specified amount of time, the agent will then attempt to label video clips with the appropriate emotion.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 30, 2020
- Source ID
- N001741910026
Entities
People
- Jerome Mcclendon
Organizations
- Clemson University
- United States Navy