Probabilistic Contingency Tables: An Improvement to Verify Probability Forecasts
Abstract
The 45th Weather Squadron (45 WS) records daily rain and lightning probabilistic forecasts and the associated binary event outcomes. Subsequently, they evaluate forecast performance and determine necessary adjustments with an established verification process. For deterministic outcomes, weather forecast analysis typically utilizes a traditional contingency table (TCT) for verification; however, the 45 WS uses an alternative tool, the probabilistic contingency table (PCT). Using the TCT for verification requires a threshold, typically at 50%, to dichotomize probabilistic forecasts. The PCT maintains the valuable information in probabilities and verifies the true forecasts being reported. Simulated forecasts and outcomes as well as 2015–18 45 WS data are utilized to compare forecast performance metrics produced from the TCT and PCT to determine which verification tool better reflects the quality of forecasts. Comparisons of frequency bias and other statistical metrics computed from both dichotomized and continuous forecasts reveal misrepresentative performance metrics from the TCT as well as a loss of information necessary for verification. PCT bias better reflects forecast verification in contrast to that of TCT bias, which suggests suboptimal forecasts when in fact the forecasts are accurate.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Apr 01, 2019
- Source ID
- 10.1175/waf-d-19-0116.1
Entities
People
- Christine Schubert Kabban
- Darryl Ahner
- Edward L White
- Mike Mcaleenan
- Sarah Gold
- William Roeder
Organizations
- 45th Weather Squadron
- Air Force Institute of Technology
- United States Air Force