Competitively Learned Attention Mechanism Prototypes for Network Intrusion Detection
Abstract
Maintaining secure computer networks and information systems is critical to the administrative functions and operations carried out by the Department of Defense (DOD). Ensuring the security and functionality of these networks is vital to US National Security, however, previous work on applying neural networks to intrusion detection has focused on recurrent and convolutional neural networks but has yet to explore attention-mechanism-based architectures. Inspired by the Transformer in Vaswani et al. (2017), These attention-based models rely predominantly on layers that produce rich contextualized representations through learning pairwise interactions within sequences, enabling significant advances in computer vision and natural language processing over the last five years. This research investigated the performance of attention-based neural network architectures compared to traditional models on the University of New Brunswick's CSE-CIC-IDS2018 dataset. Reporting precision, recall, and the Area Under the Receiver Operating Characteristic Curve (AUROC), results show that models leveraging attention mechanisms performed demonstrably better than a tuned feed-forward network on the infiltration attack class. Additionally, this work explores a novel efficient attention mechanism for deep neural networks by learning a compressed representation of the data through competitively-learned memory prototypes, showing competitive performance against an alternative efficient attention architecture that utilizes gradient descent.
Document Details
- Document Type
- Technical Report
- Publication Date
- May 16, 2022
- Accession Number
- AD1171846
Entities
People
- Kade M. Heckel
Organizations
- United States Naval Academy