2022 ATEAS Workshop: Evaluating HSI with AI-Enabled Systems: What should you consider in a TEMP?

Abstract

The Institute for Defense Analyses and the Joint AI Center (JAIC) are developing a framework to support testers as they evaluate Human Systems Integration (HSI) with AI-enabled systems. This product is a short briefing that gives an overview of the framework, conveys a sense of what the fuller framework looks like, and ends with an example of how increased autonomy in aviation led to the loss of Air France 447. We are creating this framework because HSI expertise in the DOD Test and Evaluation workforce is rare and increasingly important with more automation in systems. This briefing focuses on the portion of the framework which introduces non-experts to core HSI concerns to help testers write and assess TEMPs. This briefing covers a subset of HSI topics included in the framework, including mental models, trust, workload, and function allocation. HSI will continue to be a vital component of creating and adopting effective, suitable, and trustworthy AI-enabled systems.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Apr 14, 2022
Accession Number
AD1211270

Entities

People

  • Brian D. Vickers
  • Daniel J. Porter
  • Rachel A. Haga

Organizations

  • Institute for Defense Analyses

Tags

Communities of Interest

  • Human Systems

DTIC Thesaurus Topics

  • Abstracts
  • Artificial Intelligence
  • Atlantic Ocean
  • Automation
  • Autonomy
  • Classification
  • Complex Systems
  • Contracts
  • Department Of Defense
  • Engineering
  • Governments
  • Human Systems Integration
  • Human-Computer Interaction
  • Instructions
  • Joints
  • Management Personnel
  • Materials
  • National Governments
  • Security
  • Test And Evaluation
  • Virginia
  • Workload
  • Workshops

Fields of Study

  • Computer science

Readers

  • Aerospace Engineering
  • Geospatial Intelligence and Artificial Intelligence Analytics
  • Systems Analysis and Design