Calibrating Trust in Autonomy

Abstract

HAVE R2D2 was a Technical Management Project executed by USAF Test Pilot School students of class 21B in March 2022, demonstrating a framework for how to test and measure calibrated trust in tactical autonomy in a live-flight environment. During flight test, two L-29s engaged in one and two-circle dogfights, with one L-29 being controlled by an AI agent through steering commands. This paper highlights the framework development, test procedures, human-machine interfaces, and lessons learned from the 93 engagements conducted. Testing determined that helmet mounted cuing proved to be very effective at altering operator behavior and allowed a means to communicate real-time-trust in the AI agent. The team also discovered that in order to effectively calibrate appropriate trust, a complex risk model is required, with established mission priorities being key to success. Ultimately, this project seeks to establish the necessary building blocks for the future of testing human trust in autonomy.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 23, 2022
Accession Number
AD1183184

Entities

People

  • Joseph S Brown

Organizations

  • Air Force Test Center

Tags

DTIC Thesaurus Topics

  • Abstracts
  • Aircrafts
  • Artificial Intelligence
  • Aspect Angle
  • Autonomy
  • Battle Management
  • Battles
  • Biological Phenomena
  • Computer Programs
  • Department Of Defense
  • Ecological And Environmental Phenomena
  • Environment
  • Fighter Aircraft
  • Human-Machine Interfaces
  • Knowledge Management
  • Lessons Learned
  • Steering
  • Targets
  • Video Games

Readers

  • Aerial Unmanned Vehicle Swarm Micro Periodontal Dentistry.
  • Aerospace Engineering
  • Systems Analysis and Design