Assessing Cyber Training by Cyber Operation Mistakes
Abstract
Cybersecurity has become one of the most critical threats to mission-critical operations and national security as the nation#s critical infrastructures increasingly depend on programs, computers, and networks. Cyber operators play an essential role in protecting cyber systems. As the keeper in cyberspace, their competence determines the defense posture of a system. Despite the importance of cyber operators to the nation#s and military#s infrastructures and systems, the training for cyber operators is insufficient for practical needs. Serious security incidents remain rampant, from data breaches, and ransomeware, to Advanced Persistent Threats. Prior work also found that 37.5% cyber operating analysts consider current training is #not working,# and they #would love to change them#.Therefore, it is imperative to develop more effective training that will improve the skills of cyber operators. Having measurable and quantifiable metrics of cyber operation training is key to evaluating thequality and determining the outcomes. The PIs identifiedan understudied aspect of cyber operations#cyber operation mistakes, which are mistakes operators make during cyber operations. While mistakes greatly vary among operators and operations, an effective cyber operation training should have observable impacts on various measurable aspects of cyber operation mistakes that cyber operators make before and after the training. In preliminary work, the PIs observe that different cyber operation mistakes# whether novices or experts make them # share similar cognitive patterns. The PIs found that experts and novice operators follow the same cognitive process of committing mistakes, observing unexpected behaviors, realizing mistakes, and addressing mistakes. Measurable metrics in each step of this cognitive process can act as indicators of cyber training outcomes and effectiveness. This proposal aims to tackle the research problem of how to systematically assess and improve cyber training by cyber operation mistakes. The objective of this proposal is to study cyber operation mistakes, create automatedand semi-automated solutions to measure mistake-related aspects, use the measured data to improve an existing cyber operation training event, and, eventually, verify the findings and theories through human subject study. If successful, this proposal will harvest techniques for evaluating and improving cyber operation training and enable the iterable and constant evolution of cyber operation training programs against a quickly changing cyber threat ecosystem.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 29, 2023
- Source ID
- N000142312563
Entities
People
- Youzhi Bao
Organizations
- Arizona State University
- Office of Naval Research
- United States Navy