Applications of Modern Statistical Techniques to Operational Testing

Test data from the Department of Defense present numerous opportunities for statistical research on applied problems. Operational testing and evaluation is an integral part of the DoD acquisition process. Since conducting realistic tests with representative users in a full operational environment is costly in terms of time, money, and resources, it is imperative that we do it right. That means designing efficient tests, extracting all the information we can out of available test data, and making use of validated models and simulations. The field of statistics offers a variety of tools for addressing these problems and a framework for finding new solutions when existing techniques don’t quite fit the unique challenges presented by operational testing. Over the past 5 years, the Institute for Defense Analyses has worked with the Director, Operational Test and Evaluation to make modern statistics the norm. This talk discusses some recent challenges in operational testing relevant to modern statistical approaches and the work that IDA has undertaken to address them.