TREC 2014 Web Track Overview

Abstract

The goal of the TREC Web track over the past few years has been to explore and evaluate innovative retrieval approaches over large-scale subsets of the Web--currently using ClueWeb12, on the order of one billion pages. For TREC 2014, the sixth year of the Web track, we implemented the following significant updates compared to 2013. First, the risk-sensitive retrieval task was modified to assess the ability of systems to adaptively perform risk-sensitive retrieval against multiple baselines, including an optional self- provided baseline. In general, the risk-sensitive task explores the tradeoffs that systems can achieve between effectiveness (overall gains across queries) and robustness (minimizing the probability of significant failure, relative to a particular provided baseline). Second, we added query performance pre- diction as an optional aspect of the risk-sensitive task. The Adhoc task continued as for TREC 2013, evaluated using both adhoc and diversity relevance criteria.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Feb 18, 2015
Accession Number
ADA618628

Entities

People

  • Craig Macdonald
  • Ellen M. Voorhees
  • Fernando Diaz
  • Kevyn Collins-thompson
  • Paul Bennett

Organizations

  • University of Michigan

Tags

Communities of Interest

  • Biomedical
  • Cyber

DTIC Thesaurus Topics

  • Abstracts
  • Algorithms
  • Altitude Sickness
  • Arachnid Bites And Stings
  • Automatic
  • Base Lines
  • Body Weight
  • Data Sets
  • Factor Analysis
  • Information Operations
  • Judgment
  • Language
  • Social Networking Services
  • Spiders
  • Standards
  • Test And Evaluation
  • Universities

Fields of Study

  • Computer science

Readers

  • Adaptive Control and Estimation with Uncertainty in Dynamic Systems.
  • Information Retrieval
  • Instructional Design and Training Evaluation.