Securing Operating Systems using Hardware-Enforced Compartmentalization

Abstract

Monolithic kernels have been the traditional design choice of many modern operating systems for practical and historical reasons. Though monolithic systems excel in performance, they suffer from exposure to security vulnerabilities. The past 6 years of published Linux CVE data has revealed hundreds of security vulnerabilities that can potentially be exploited by an attacker to escalate privileges and leak sensitive user data. Though some of these vulnerabilities can be mitigated with proper memory safety enforcement, others require privilege separation to ensure code only accesses data that is explicitly granted by a developer. We present Hardware-Assisted Kernel Compartments (HAKC), a solution that mitigates exposure to security vulnerabilities by leveraging modern commodity Arm hardware and automatic LLVM instrumentation to enforce compartmentalization in an effective manner without requiring significant developer effort. Using Arm Pointer Authentication Codes (PAC) and Arm Memory Tagging Extensions (MTE), HAKC enforces a two-tier compartmentalization scheme that is performant and provides flexibility for up to 4 * 1015 compartments, which, when compared to prior works, is orders of magnitude more compartments afforded to developers. To test HAKC, we implemented a compartmentalization policy for nf_tables, a commonly used packet filtering LKM. LKMs are prime targets for compartmentalization because CVE analysis has shown that most kernel vulnerabilities reside in LKMs, and the HAKC two-tiered compartmentalization scheme easily adapts to LKM logical groupings of kernel subsystem functionality. Evaluations show that we are able to achieve strong security enforcement without adding significant overhead.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 01, 2021
Accession Number
AD1204448

Entities

People

  • Yianni Giannaris

Organizations

  • MIT Lincoln Laboratory
  • Massachusetts Institute of Technology

Tags

Communities of Interest

  • Cyber
  • Space

DTIC Thesaurus Topics

  • Authentication
  • Code Injection
  • Computer Access Control
  • Computer Programming
  • Computer Programs
  • Computer Science
  • Computers
  • Cybersecurity
  • Data Storage Systems
  • Device Drivers
  • Electrical Engineering
  • Instruction Set Architecture
  • Network Protocols
  • New York
  • Operating Systems
  • Programming Languages
  • Systems Engineering
  • Web Browsers

Fields of Study

  • Computer science

Readers

  • Cybersecurity.
  • Parallel and Distributed Computing.