Of Ivory and Smurfs: Loxodontan MapReduce Experiments for Web Search

Abstract

This paper describes Ivory, an attempt to build a distributed retrieval system around the open-source Hadoop implementation of MapReduce. We focus on three noteworthy aspects of our work: a retrieval architecture built directly on the Hadoop Distributed File System (HDFS), a scalable Map-Reduce algorithm for inverted indexing, and webpage classification to enhance retrieval effectiveness.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Nov 01, 2009
Accession Number
ADA517816

Entities

People

  • Donald Metzler
  • Jimmy Lin
  • Lidan Wang
  • Tamer Elsayed

Organizations

  • University of Maryland

Tags

DTIC Thesaurus Topics

  • Abstracts
  • Algorithms
  • Batch Processing
  • Cloud Computing
  • Computations
  • Computer Programming
  • Computer Science
  • Computers
  • Content Addressable Memory
  • Data Management
  • Data Transmission
  • Elephants
  • Frequency
  • Information Retrieval
  • Operating Systems
  • Standards
  • Test And Evaluation

Fields of Study

  • Computer science

Readers

  • Computer Science/Computer Engineering/Data Science/Digital Signal Processing.
  • Distributed Systems and Data Platform Development
  • Maritime Security/Maritime Homeland Security