Resource Constrained Reinforcement Learning for Adaptive Data Analytics in Tactical Networks
Abstract
The success of communication networks has traditionally relied on well-founded analytical models, coupled with simple algorithms that lend themselves to easy implementation. It is the robust design obtained under these design principles that has enabled data aggregation operator such as windowed grouped aggregation a key primitive in most data analytics frameworks as well make their mark in distributed data analytics in tactical networks. However, this approach brings with it severe limitations in terms of low adaptability. Currently, windowed grouped aggregation and its variants typically use a statically fixed window size, regardless of the actual queries, or a dynamic window tuned in an ad hoc way without optimality guarantees. On the other hand, the continuously growing density and availability of sensors in the environment has led to a rapid increase in both volume and geographical diversity of data sources at tactical edges. As a result, achieving low-latency and high-throughput processing of distributed data sources at a low cost (e.g., bandwidth) is key to extract insights in a timely manner. However, ensuring a stable execution of long-running queries in a wide-area environment is very challenging due to the variability and unpredictable nature of both workload and wide-area network bandwidth, which many change in an interval of minutes. Would it be possible to design machine learning (ML) methods to augment the system such that the different nodes can operate together as a team to find the ideal window size for each node to ensure best possible application performance, measured by so-called quality of experience? The goal of this project is a systematic and principled study of methods for adaptive real-time data analytics in resource constrained tactical networks. Achieving this goal will require the conjunction of several mathematical tools to design and analyze learning-augmented algorithms, and strong system deployment skills to make it a reality. This proposal addresses these challenges in three interdependent thrusts: (1) Developing low-complexity timer-based threshold-type policy that simply computes a timer value for each decision variable. Our key insight will be a novel connection between data aggregation and traditional caching framework. (2) Proposing resource constrained reinforcement learning (RL) based algorithms. Our key insight is that RL can leverage the structure of the timer-based policy so as to reduce the computational complexity in resource constrained tactical networks. (3) Evaluation and implementation on open-sourced projects and public testbed such as Apache Flink and POWDER. Our research will constitute a significant advance in the development of theories, algorithms and systems for designing and analyzing data analytics in resource constrained tactical networks using ML. At the same time, the project develops fundamental theories that pertain to the area of ML, specifically to reinforcement learning. This project includes an education plan focusing on machine learning and data analytics, and outreach in the form of summer camps, seminars and demonstrations to high-school students based on the project results.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 01, 2023
- Source ID
- W911NF2310072
Entities
People
- Jian Li
Organizations
- Army Contracting Command
- Research Foundation for the State University of New York
- United States Army