Learning Models of Speaker Head Nods with Affective Information

Abstract

During face-to-face conversation, the speaker's head is continually in motion. These movements serve a variety of important communicative functions, and may also be influenced by our emotions. The goal for this work is to build a domain-independent model of speaker's head movements and investigate the effect of using affective information during the learning process. Once the model is learned, it can later be used to generate head movements for virtual agents. In this paper, we describe our machine-learning approach to predict speaker's head nods using an annotated corpora of face-to-face human interaction and emotion labels generated by an affect recognition model. We describe the feature selection process, training process, and the comparison of results of the learned models under varying conditions. The results show that using affective information can help predict head nods better than when no affective information is used.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2009
Accession Number
AD1171167

Entities

People

  • Alena Neviarouskaya
  • Helmut Prendinger
  • Jina Lee
  • Stacy C. Marsella

Organizations

  • University of Southern California

Tags

Communities of Interest

  • Autonomy

DTIC Thesaurus Topics

  • Accuracy
  • Artificial Intelligence
  • Artificial Intelligence Software
  • Automated Speech Recognition
  • Autonomous Agents
  • Computational Linguistics
  • Computational Science
  • Feature Selection
  • Hidden Markov Models
  • Language
  • Linguistics
  • Machine Learning
  • Markov Models
  • Models
  • Multiagent Systems
  • Probabilistic Models
  • Probability
  • Recognition

Fields of Study

  • Computer science

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Computational Modeling and Simulation

Technology Areas

  • AI & ML
  • AI & ML - Neural Networks