Statistical Language Modeling for Information Retrieval

Abstract

This chapter reviews research and applications in statistical language modeling for information retrieval (IR) that has emerged within the past several years as a new probabilistic framework for describing information retrieval processes. Generally speaking, statistical language modeling, or more simply, language modeling (LM), refers to the task of estimating a probability distribution that captures statistical regularities of natural language use. Applied to information retrieval, language modeling refers to the problem of estimating the likelihood that a query and a document could have been generated by the same language model, given the language model of the document and with or without a language model of the query.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2005
Accession Number
ADA440321

Entities

People

  • W. Bruce Croft
  • Xiaoyong Liu

Organizations

  • University of Massachusetts Amherst

Tags

Communities of Interest

  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Automated Speech Recognition
  • Computational Science
  • Hidden Markov Models
  • Information Processing
  • Information Retrieval
  • Information Science
  • Information Systems
  • Knowledge Management
  • Language
  • Linguistics
  • Markov Models
  • Natural Language Processing
  • Natural Languages
  • Probabilistic Models
  • Probability
  • Probability Distributions
  • Signal Processing

Fields of Study

  • Computer science

Readers

  • Adaptive Control and Estimation with Uncertainty in Dynamic Systems.
  • Computational Modeling and Simulation
  • Computer Science.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Information Retrieval
  • AI & ML - Machine Translation