Knowing When You Don't Know: AI Engineering in an Uncertain World Research Review

Abstract

Final Thoughts. Machine-learned models are are able to express uncertainty in their predictions that can lead to more informative, robust AI systems by: 1. allowing humans to reason about when the model is likely to be incorrect, 2. allowing components in a larger system to take different actions based on model confidence. In this project we research methods to evaluate, characterize, articulate and rectify uncertainty. Next steps: - Develop a demonstration highlighting the utility of accurately expressing uncertainty. Create techniques to characterize the cause of uncertainty for a ML model. For the audience: We are always looking for motivating real-world uses for our work. If you have a need for AI Systems that are able to express and reason under uncertainty, do not hesitate to reach out.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Nov 01, 2021
Accession Number
AD1150262

Entities

People

  • Eric T. Heim

Organizations

  • Carnegie Mellon University

Tags

Communities of Interest

  • Autonomy

DTIC Thesaurus Topics

  • Artificial Intelligence
  • Calibration
  • Data Sets
  • Department Of Defense
  • Engineering
  • Guarantees
  • Intervals
  • Learning
  • Machine Learning
  • Materials
  • National Guard
  • Probability
  • Software Development
  • South Carolina
  • Uncertainty
  • Universities

Readers

  • Neural Network Machine Learning.
  • Systems Analysis and Design

Technology Areas

  • AI & ML
  • AI & ML - DoD AI Strategy
  • AI & ML - Neural Networks