Beyond the Turing Test: Performance Metrics for Evaluating a Computer Simulation of the Human Mind

Abstract

Performance metrics for machine intelligence (e.g., the Turing test) have traditionally consisted of pass/fail tests. Because the tests devised by psychologists have been aimed at revealing unobservable processes of human cognition, they are similarly capable of revealing how a computer accomplishes a task, not simply its success or failure. Here we propose the adaptation of a set of tests of abilities previously measured in humans to be used as a benchmark for simulation of human cognition. Our premise is that if a machine cannot pass these tests, it is unlikely to be able to engage in the more complex cognition routinely exhibited by animals and humans. If it cannot pass these sorts of tests, it will lack fundamental capabilities underlying such performance.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Aug 01, 2002
Accession Number
ADA511091

Entities

People

  • Craig Latta
  • Nancy Alvarado
  • Sam S. Adams
  • Steve Burbeck

Organizations

  • IBM Thomas J. Watson Research Center

Tags

Communities of Interest

  • Autonomy

DTIC Thesaurus Topics

  • Acquisition
  • Artificial Intelligence
  • Cognition
  • Computational Science
  • Computer Simulations
  • Computers
  • Defense Mechanisms
  • Engineering
  • Environment
  • Grammars
  • Human Behavior
  • Intelligent Systems
  • Language
  • Learning
  • Psychology
  • Reasoning
  • Simulations

Readers

  • Neural Network Machine Learning.
  • Psychometric Testing or Psychological Assessment.
  • Systems Analysis and Design