Exploring User Satisfaction in a Tutorial Dialogue System

Abstract

User satisfaction is a common evaluation metric in task-oriented dialogue systems, whereas tutorial dialogue systems are often evaluated in terms of student learning gain. However, user satisfaction is also important for such systems, since it may predict technology acceptance. We present a detailed satisfaction questionnaire used in evaluating the BEETLE II system (REVU-NL), and explore the underlying components of user satisfaction using factor analysis. We demonstrate interesting patterns of interaction between interpretation quality, satisfaction and the dialogue policy, highlighting the importance of more fine-grained evaluation of user satisfaction.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jun 01, 2011
Accession Number
ADA542507

Entities

People

  • Gwendolyn Campbell
  • Johanna D. Moore
  • Myroslava O. Dzikovska
  • Natalie Steinhauser

Organizations

  • Naval Air Warfare Center

Tags

Communities of Interest

  • C4I

DTIC Thesaurus Topics

  • Artificial Intelligence
  • Command And Control
  • Computational Linguistics
  • Data Analysis
  • Data Science
  • Dialogue Systems
  • Factor Analysis
  • Graphical User Interface
  • Information Science
  • Language
  • Learning
  • Linguistics
  • Natural Languages
  • Questionnaires
  • Standards
  • Students
  • Surveys

Fields of Study

  • Computer science

Readers

  • Artificial Intelligence
  • Systems Analysis and Design