The Development and Inter-Rater Reliability of the Department of Defense Human Factors Analysis and Classification System, Version 7.0

Abstract

This report describes improvements in the Department of Defense Human Factors Analysis and Classification System (DoD HFACS). Military services had a unanimous desire to improve DoD HFACS inter-rater reliability. A working group, composed of representatives from all services, met several times with the goal of improving inter-rater reliability while retaining the value of the tool. Additional requirements included preserving compatibility with existing databases and having inter-operability across the services. The steps involved included determining which nanocodes were rarely or never used and collapsing nanocodes and rewriting definitions to arrive at a total of 109 nanocodes, reduced from 147 nanocodes. A table, included in this report, allowing for the ready translation of old codes into new codes permits continued analysis of data already collected. The authors, in collaboration with the DoD HFACS Working Group, then developed a stepwise checklist to systematically guide investigators through consideration of nanocodes. Researchers tested several iterations of the technique using students in Air Force mishap investigation courses to gauge inter-rater reliability. Student investigators were also invited to offer constructive criticism to hone checklist questions. While inter-rater reliability results are encouraging, the DoD HFACS Working Group has additional work to accomplish to realize the goal of an optimally reliable human factors taxonomy. This report contains the complete DoD HFACS version 7.0 taxonomy as well as the checklist that was generated and tested with groups of mishap investigation students. Suggestions for future efforts are offered, to include an on-going research program.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Apr 09, 2015
Accession Number
ADA623729

Entities

People

  • Erik W. Kuhlmann
  • Jeffrey Lawson
  • Raymond E. King
  • Timothy Strongin

Organizations

  • United States Air Force School of Aerospace Medicine

Tags

Communities of Interest

  • Air Platforms
  • Biomedical
  • Human Systems
  • Space

DTIC Thesaurus Topics

  • Accidents
  • Aerospace Medicine
  • Air Force
  • Air Force Research Laboratories
  • Aircrafts
  • Classification
  • Computer Programming
  • Control Systems
  • Data Analysis
  • Databases
  • Department Of Defense
  • Governments
  • Medical Personnel
  • Reliability
  • Students
  • Taxonomy
  • Training

Fields of Study

  • Engineering

Readers

  • Aviation Safety Risk Assessment.
  • Instructional Design and Training Evaluation.
  • Software Engineering.