Adaptive Cyber Defense Against Multi-Stage Attacks Using Learning-Based POMDP
Abstract
Growing multi-stage attacks in computer networks impose significant security risks and necessitate the development of effective defense schemes that are able to autonomously respond to intrusions during vulnerability windows. However, the defender faces several real-world challenges, e.g., unknown likelihoods and unknown impacts of successful exploits. In this article, we leverage reinforcement learning to develop an innovative adaptive cyber defense to maximize the cost-effectiveness subject to the aforementioned challenges. In particular, we use Bayesian attack graphs to model the interactions between the attacker and networks. Then we formulate the defense problem of interest as a partially observable Markov decision process problem where the defender maintains belief states to estimate system states, leverages Thompson sampling to estimate transition probabilities, and utilizes reinforcement learning to choose optimal defense actions using measured utility values. The algorithm performance is verified via numerical simulations based on real-world attacks.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Nov 08, 2020
- Source ID
- 10.1145/3418897
Entities
People
- Minghui Zhu
- Peng Liu
- Zhisheng Hu
Organizations
- Army Research Office
- Baidu
- Division of Computer and Network Systems
- Division of Electrical, Communications & Cyber Systems
- National Security Agency
- Pennsylvania State University