CMU OAQA at TREC 2015 LiveQA: Discovering the Right Answer with Clues

Abstract

In this paper, we present CMUs automatic, web-based, real-time question answering (QA) system that was evaluated in the TREC 2015 LiveQA Challenge.This system answers real-user questions freshly submitted to the Yahoo! Answers website that have not been previously answered by humans. Given the title and body of the question, we generated multiple sets of keyword queries and retrieved a collection of web pages based on those queries. Then we extracted answer candidates from web pages in the form of answer passages and their associated clue. Finally, we combined both IR- and NLP-based relevance models to rank and select answer candidates. In the TREC 2015 LiveQA evaluations, human assessors gave our system an average score of 1.081 on a three-point scale, the highest average score achieved by a system in the competition (the second-best score was .677, and the average score was .465 for the 21 systems evaluated).

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Nov 20, 2015
Accession Number
AD1004807

Entities

People

  • Di Wang
  • Eric Nyberg

Organizations

  • Carnegie Mellon University

Tags

DTIC Thesaurus Topics

  • Algorithms
  • Artificial Intelligence Computing
  • Artificial Intelligence Software
  • Computer Languages
  • Estimators
  • Intelligent Agents
  • Language
  • Natural Language Processing
  • Networks
  • Neural Networks
  • Personality
  • Recurrent Neural Networks
  • Supervised Machine Learning
  • Test And Evaluation
  • Training
  • Web Service

Fields of Study

  • Computer science

Readers

  • Information Retrieval