Scientific Test and Analysis Techniques Statistical Measures of Merit

November, 2013
IDA document: D-5070
FFRDC: Systems and Analyses Center
Type: Documents
Division: Operational Evaluation Division
Authors:
Authors
Laura J. Freeman See more authors
Design of Experiments (DOE) provides a rigorous methodology for developing and evaluating test plans. Design excellence consists of having enough test points placed in the right locations in the operational envelope to answer the questions of interest for the test. The key aspects of a well-designed experiment include: the goal of the test, the response variables, the factors and levels, a method for strategically varying the factors across the operational envelope, and statistical measures of merit. Currently, the majority of test plans utilize statistical measures of merit based on confidence and power. Although important, confidence and power are not the only measure of the adequacy and merit of a test design. The type of method that is appropriate is dependent on the goal of the test and the experimental design methodology used. There is no one-size-fits-all solution; rather there is a collection of useful tools that apply in various combinations for different test goals and designs. This talk outlines different statistical measures of merit that should be used when planning an operational test.