The current fiscal climate demands now, more than ever, that test and evaluation (T&E) provide relevant and credible characterization of system capabilities and shortfalls across all relevant operational conditions as efficiently as possible. In determining the answer to the question, “How much testing is enough?” it is imperative that we use a scientifically defensible methodology. Design of Experiments (DOE) has a proven track record in Operational Test and Evaluation (OT&E) of not only quantifying how much testing is enough, but also where in the operational space the test points should be placed. Over the last few years, the T&E community has made great strides in the application of DOE to OT&E, but there is still work to be done in ensuring that the scientific community’s full toolset is utilized. In particular, many test programs have yet to capitalize on the power of the test design when conducting the data analysis. Employing empirical statistical models (e.g., regression techniques, analysis of variance (ANOVA)) allows us to maximize the information from every data point, resulting in defensible analyses that provide crucial information about system performance that decision-makers and warfighters need to know. DOT&E will continue to work to ensure the highest technical caliber in every DOT&E evaluation, and that Test and Evaluation Master Plans (TEMPs) are adequate to support these robust evaluations. As we improve in our use of these test designs and analysis methods, we need to ensure these practices are institutionalized across the entire T&E community and applied across all phases of DoD testing.
