Statistical Design and Analysis Challenges in Defense Testing

Abstract

Before the DoD acquires any major new capability, that system must undergo realistic testing in its intended environment with military users. The complex, data-limited, highly variable nature of the test environment presents many unique statistical challenges. The set of conditions in which a system will operate is typically large, and important variables are often uncontrollable during test, making rigorous experimental design a challenge. Data sets obtained from tests are almost always messy. Issues such as lurking variables, small and unbalanced sample sizes, and ordinal responses necessitate creative and sometimes sophisticated data analysis approaches. This document examines some of these defense-testing situations in detail and discusses how statisticians in the test and evaluation community have approached associated design and analysis challenges.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2019
Accession Number
AD1122393

Entities

People

  • Heather Wojtan
  • Kelly Avery

Organizations

  • Institute for Defense Analyses

Tags

DTIC Thesaurus Topics

  • Air Force
  • Aircrafts
  • Change Detection
  • Communities
  • Computing-Related Activities
  • Congress
  • Control Systems
  • Data Analysis
  • Data Science
  • Data Sets
  • Department Of Defense
  • Detection
  • Environment
  • Experimental Design
  • Fighter Aircraft
  • Information Science
  • Personnel Management
  • Reliability Engineering
  • Simulations
  • Statistics
  • Test And Evaluation
  • Unmanned Aerial Vehicles
  • Validation

Fields of Study

  • Computer science

Readers

  • Regression Analysis.
  • Systems Analysis and Design