Detecting Hardware-assisted Hypervisor Rootkits within Nested Virtualized Environments

Abstract

Virtual machine introspection (VMI) is intended to provide a secure and trusted platform from which forensic information can be gathered about the true behavior of malware within a guest. However, it is possible for malware to escape a guest into the host and for hypervisor rootkits, such as BluePill, to stealthily transition a native OS into a virtualized environment. This research examines the effectiveness of selected detection mechanisms against hardware-assisted virtualization rootkits (HAV-R) within a nested virtualized environment. It presents the design, implementation, analysis, and evaluation of a hypervisor rootkit detection system which exploits both processor and translation lookaside buffer-based mechanisms to detect hypervisor rootkits within a variety of nested virtualized systems. It evaluates the effects of different types of virtualization on hypervisor rootkit detection and explores the effectiveness in-guest HAV-R obfuscation efforts. The results provide convincing evidence that the HAV-Rs are detectable in all SVMI scenarios examined, regardless of HAV-R or virtualization type. Also, that the selected detection techniques are effective at detection of HAV-R within nested virtualized environments, and that the type of virtualization implemented in a VMI system has minimal to no effect on HAV-R detection. Finally, it is determined that in-guest obfuscation does not successfully obfuscate the existence of HAV-R.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jun 14, 2012
Accession Number
ADA563168

Entities

People

  • Daniel B. Morabito

Organizations

  • Air Force Institute of Technology

Tags

DTIC Thesaurus Topics

  • Air Force
  • Computational Science
  • Computer Programming
  • Computer Science
  • Computers
  • Cybersecurity
  • Debugging
  • Detection
  • Information Science
  • Instruction Set Architecture
  • Intrusion Detection
  • Intrusion Detectors
  • Kernels (Operating System)
  • Network Science
  • Operating Systems
  • Statistical Analysis
  • Test And Evaluation

Fields of Study

  • Computer science

Readers

  • Cybersecurity.
  • Parallel and Distributed Computing.

Technology Areas

  • Cyber