DATAWorks 2023 Development of a Wald-Type Statistical Test to Compare Live Test Data and M&S Predictions

April, 2023
IDA document: D-33406
FFRDC: Systems and Analyses Center
Type: Documents
Division: Operational Evaluation Division
Elliot AJ Bartis, Carrington A. Metts, Curtis G. Miller See more authors
This work describes the development of a statistical test created in support of ongoing verification, validation, and accreditation (VV&A) efforts for modeling and simulation (M&S) environments. The test decides between a null hypothesis of agreement between the simulation and reality, and an alternative hypothesis stating the simulation and reality do not agree. To do so, it generates a Wald-type statistic that compares the coefficients of two generalized linear models that are estimated on live test data and analogous simulated data, then determines whether any of the coefficient pairs are statistically different.