Quantifying Forecast Uncertainty

Government sponsors are often interested in predicting attrition rates for service members at future time points. Point forecasts are most commonly considered in isolation. While point forecasts are useful, they only provide partial information regarding the future distribution of the quantity of interest and no information regarding the uncertainty in the forecast. Point estimates for future values of interest can be close to the truth, but they are always subject to uncertainty. Use of distributional forecasts or prediction intervals around point estimates improves understanding of the uncertainty associated with these predictions. We discuss several methods for quantifying forecast uncertainty, specifically in survival forecasts, including both generic and learner-specific methods and implement these approaches in the Finite Interval Forecasting Engine (FIFE), developed by Institute for Defense Analyses research. We find that generic approaches can often be too conservative and use-specific methods can be misleading when used naively. We discuss the performance of these methods and suggest improvements.