The Military Inventory Routing Problem: Utilizing Heuristics Within a Least Squares Temporal Differences Algorithm to Solve a Multiclass Stochastic Inventory Routing Problem with Vehicle Loss
Abstract
Military commanders currently resupply forward operating bases (FOBs) from a central location within an area of operations mainly via convoy operations in a way that closely resembles vendor managed inventory practices. Commanders must decide when and how much inventory to distribute throughout their area of operations while minimizing soldier risk. Technology currently exists that makes utilizing unmanned cargo aerial vehicles (CUAVs) for resupply an attractive alternative due to the dangers of utilizing convoy operations. Enemy actions in wartime environments pose a significant risk to a CUAV's ability to safely deliver supplies to a FOB. We develop a Markov decision process (MDP) model to examine this military inventory routing problem (MILIRP). In our first paper we examine the structure of the MILIRP by considering a small problem instance and prove value function monotonicity when a sufficient penalty is applied. Moreover, we develop a monotone least squares temporal differences (MLSTD) algorithm that exploits this structure and demonstrate its efficacy for approximately solving this problem class. We compare MLSTD to least squares temporal differences (LSTD), a similar ADP algorithm that does not exploit monotonicity. MLSTD attains a 3:05% optimality gap for a baseline scenario and outperforms LSTD by 31:86% on average in our computational experiments. Our second paper expands the problem complexity with additional FOBs. We generate two new algorithms, Index and Rollout, for the routing portion and implement an LSTD algorithm that utilized these to produce solutions 22% better than myopic generated solutions on average. Our third paper greatly increases problem complexity with the addition of supply classes. We formulate an MDP model to handle the increased complexity and implement our LSTD-Index and LSTD-Rollout algorithms to solve this larger problem instance and perform 21% better on average than a myopic policy.
Document Details
- Document Type
- Technical Report
- Publication Date
- Sep 16, 2018
- Accession Number
- AD1063682
Entities
People
- Ethan L. Salgado
Organizations
- Air Force Institute of Technology