Test Science Website Videos—Test Planning and Design

Understanding the test design process is fundamental to conducting efficient and effective tests of systems. Planning is a critical element of this process and requires collaboration and iteration among subject matter experts. Test planning involves identifying the test objectives, determining the appropriate outcome measure(s), and identifying what factors may influence those outcomes. The first set of website videos introduces the overarching test design process and provides best practices and examples for each of the three planning steps. Test design is the process of actually creating and evaluating a set of experimental runs. It entails careful consideration of the test goals to ensure objectives can be met by the data collected, building the corresponding type of design, and then evaluating it across multiple criteria. The test design should have a large enough sample size to be useful, be executable in the real world, and support the desired analysis. The second set of videos describes and provides recommendations for each element of the test design process, and also introduces considerations for how to manage and store data collected from tests.