Strictly Proper Scoring Rules, Prediction, and Estimation

Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the forecast and on the event or value that materializes. A scoring rule is strictly proper if the forecaster maximizes the expected score for an observation drawn from the distribution F if she issues the probabilistic forecast F, rather than any G � = F. In prediction problems, strictly proper scoring rules encourage the forecaster to make careful assessments and to be honest. In estimation problems, strictly proper scoring rules provide attractive loss and utility functions that can be tailored to the scientific problem at hand. This paper characterizes strictly proper scoring rules on general probability spaces, and proposes and discusses examples of such. In the case of categorical and binary variables, a rigorous version of the Savage representation is established. Examples of scoring rules for probabilistic forecasts in the form of predictive densities include the spherical, pseudospherical, logarithmic and quadratic score. The continuous ranked probability score applies to probabilistic forecasts that take the form of predictive cumulative distribution