Architectures for Event Memories using the Value of Information
Abstract
This proposal seeks to innovate Information Sciences and Engineering by extending the concept of system state to include external memories. Currently the system state is an internal variable that brings to the current time a processed version of the past of the input signal relevant for the application. This proposal introduces the concept of events in time which are sequences of states relevant for the application, but that are not necessarily contiguous in time, and that the system learns to store in memory when they occur and recall when necessary for future processing. The proposal studies efficient architectures and compares their performance in environments that are nonstationary, which are relevant for distributed controls and network intelligence. We further propose to implement these event memory architectures in a Reproducing Kernel Hilbert Space which is very efficient to design nonlinear mappers utilizing linear system theory. We propose to utilize the Kalman update to improve the training speed and the accuracy of backpropagation through time, which needs to be utilized because of the recursive nature of the gradients. Finally, the extraction of events will be studied using Stratonovich concept of value of Information, which combines in an optimal way the uncertainty (information) in the data with the utility (external cost) for the application. We expect that this framework will outperform the current methodologies utilized in machine learning.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 25, 2021
- Source ID
- W911NF2110254
Entities
People
- José PrÃncipe
Organizations
- Army Contracting Command
- United States Army
- University of Florida