A sequence of tools for multimodal analysis of verbal and nonverbal behaviors related to interactional rapport

Abstract

We describe a method for annotation and analysis of multimodal video data using a sequence of specialized tools that we have made interoperable. We present initial, test-sample results derived with this method. This report details the building blocks of cross-language, multi-modal analyses planned for a large corpus of audio-videotaped, dyadic, conversation data comprising elicitations from Gulf-region Arabic speakers, Mexican Spanish speakers, and American English speakers. We discuss how our approach meets the challenge of readying audio, video, and transcription text data from these three diverse languages for annotation and comparative analysis of multimodal language behaviors related to maintenance of interactional rapport.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2009
Accession Number
AD1108273

Entities

People

  • Dan Loehr
  • Gina-anne Levow
  • Sue Duncan

Organizations

  • MITRE Corporation

Tags

Communities of Interest

  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Arabic Language
  • Automated Speech Recognition
  • Best Practices
  • Cognitive Systems Engineering
  • Colorado
  • Corporations
  • Data Set
  • Data Sets
  • Dictionaries
  • Digital Data
  • Hard Copy
  • Human-Computer Interaction
  • Information Systems
  • Intervals
  • Language
  • Linguistics
  • Maintenance
  • Modal Analysis
  • Sequences
  • Standards
  • Universities
  • User Interface
  • Video

Fields of Study

  • Linguistics

Readers

  • Computational Linguistics
  • Distributed Systems and Data Platform Development
  • Speech Processing/Speech Recognition.