Evaluation of an Integrated Multi-Task Machine Learning System with Humans in the Loop

Abstract

Performance of a cognitive personal assistant RADAR, consisting of multiple machine learning components natural language processing, and optimization was examined with a test explicitly developed to measure the impact of integrated machine learning when used by a human user in a real world setting. Three conditions (conventional tools, Radar without learning, and Radar with learning) were evaluated in a large-scale, between-subjects study. The study revealed that integrated machine learning does produce a positive impact on overall performance. This paper also discusses how specific machine learning components contributed to human-system performance.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2007
Accession Number
ADA534467

Entities

People

  • Aaron Steinfield
  • Dan Siewiorek
  • Django Wexler
  • Jordan Hayes
  • Julie Fitzgerald
  • Kyle Cunningham
  • Matt Lahut
  • Pablo-alejandro Quinones
  • Paul Cohen
  • S. R. Bennett

Organizations

  • Carnegie Mellon University

Tags

DTIC Thesaurus Topics

  • Algorithms
  • Computer Languages
  • Computer Science
  • Electronic Mail
  • Engineering
  • Institutional Review Board
  • Machine Learning
  • Materials
  • Measurement
  • Natural Language Processing
  • Optimization
  • Personality
  • Radar
  • Radar Components
  • Test And Evaluation
  • Test Methods
  • Word Processors

Fields of Study

  • Computer science

Readers

  • Computational Linguistics
  • Team-Based Human-Centered Cognitive Task Decision Making and Information Performance.

Technology Areas

  • AI & ML
  • AI & ML - Neural Networks