Assured Dependability for Autonomous Systems

May, 2019
IDA document: D-10669
FFRDC: Systems and Analyses Center
Type: Research Summary/Research Insights , Documents
Sequential diagnostic, operational, and acceptance testing is the normal approach to assuring system developers and users that any kind of system will perform its mission safely, effectively, and reliably. The dependability of an autonomous system, however, depends on the system’s decision processes, which interpret sensor data, model the environment, consider mission goals and priorities, choose courses of action, observe outcomes, and potentially modify the system’s own logic over time through post-fielding learning. The normal testing approach cannot possibly effectively test, evaluate, verify, and validate system behavior in every decision context an autonomous system could face. A different approach is needed to assure developers, operators, and commanders that autonomous systems will perform dependably in situations that may differ significantly from any that were tested explicitly prior to fielding.