Responsive Behavior of a Listening Agent

Abstract

The purpose of this assignment is twofold. First the possibility of generating real time responsive behavior is evaluated in order to create a more human-like agent. Second, the effect of the behavior of the agent on the human interactor is evaluated. The main motivation for the focus on responsive gestures is because much research has been done already on gestures that accompany the speaker, and nothing on gesture that accompany the listener, although responsiveness is a crucial part of a conversation. The responsive behavior of a virtual agent consists of performing gestures during the time a human is speaking to the agent. To generate the correct gestures, first a literature research is carried out, from which is concluded that with the current of the current Natural Language Understanding technology, it is not possible to extract semantic features of the human speech in real time. Thus, other features have to be considered. The result of the literature research is a basic mapping between real time obtainable features and their correct responsive behavior: - if the speech contains a relatively long period of low pitch then perform a head nod. - if the speech contains relatively high intensity then perform a head nod - if the speech contains disfluency then perform a posture shift, gazing behavior or a frown - if the human performs a posture shift then mirror this posture shift - if the human performs a head shake then mirror this head shake - if the human performs major gazing behavior then mimic this behavior A design has been made to implement this mapping into the behavior of a virtual agent and this design has been implemented which results in two programs. One to mirror the physical features of the human and one to extract the speech features from the voice of the human. The two programs are combined and the effect of the resulting behavior on the human interactor has been tested.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 01, 2005
Accession Number
AD1159554

Entities

People

  • Jonathan Gratch
  • R. M. Maatman
  • Stacy C. Marsella

Organizations

  • University of Southern California
  • University of Twente

Tags

DTIC Thesaurus Topics

  • Algorithms
  • Automated Speech Recognition
  • Background Noise
  • California
  • Cognitive Science
  • Computational Science
  • Computer Programming
  • Computer Programs
  • Computer Science
  • Computers
  • Data Acquisition
  • Fast Fourier Transforms
  • Feature Extraction
  • Language
  • Natural Language Understanding
  • Natural Languages
  • Operating Systems
  • Signal Processing
  • Virtual Reality

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Speech Processing/Speech Recognition.
  • Systems Analysis and Design