The Comparison and Evaluation of Forecasters.
Abstract
In this paper, we present methods for comparing and evaluating forecasters whose predictions are presented as their subjective probability distributions of various random variables that will be observed in the future, e.g. weather forecasters who each day must specify their own probabilities that it will rain in a particular location. We begin by reviewing the concepts of calibration and refinement, and describing the relationship between this notion of refinement and the notion of sufficiency in the comparison of statistical experiments. We also consider the question of interrelationships among forecasters and discuss methods by which an observer should combine the predictions from two or more different forecasters. Then we turn our attention to the concept of a proper scoring rule for evaluating forecasters, relating it to the concepts of calibration and refinement. Finally, we discuss conditions under which one forecaster can exploit the predictions of another forecaster to obtain a better score. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- May 01, 1982
- Accession Number
- ADA121924
Entities
People
- Morris H. Degroot
- Stephen E. Fienberg
Organizations
- Carnegie Mellon University