A Framework for Evidence-Based Licensure of Adaptive Autonomous Systems

March, 2016
IDA document: P-5325
FFRDC: Systems and Analyses Center
Type: Documents
Division: Science and Technology Division

IDA was tasked to assess potential benefits and challenges of applying an evidencebased licensing scheme to autonomous systems, and provide any insights gleaned to DoD. The autonomy community has identified significant challenges associated with test, evaluation verification and validation of autonomous systems. These challenges are exacerbated for adaptive and/or learning systems whose behavior on given inputs can change over time. The result is that a traditional comprehensive testing approach is impractical or impossible to implement. This report provides support for evidence-based licensure as a test, evaluation, verification, and validation (TEVV) framework that can address these challenges.

IDA found that traditional approaches to developmental and operational testing will not generally be sufficient to establish the safety and effectiveness of autonomous systems. A licensure approach does have the potential to support confident fielding of autonomous systems. To be successful, a licensure approach would require novel or modified activities throughout the life cycle of the system.These include:

• Explicit statement of safety and effectiveness requirements
• Formal methods during development
• Normative models of intended system behavior
• Modular and scalable virtual development test bed
• Run-time monitoring • Periodic recertification of learning systems

Existing software and systems engineering techniques, if rigorously applied, provide a good start. Furthermore, there are ongoing research programs in most of these novel areas. That said, the approach must be supported by S&T investments aimed at overcoming distinct challenges of adaptive or learning autonomous systems. Candidate investment areas include:

• Mapping high level natural language requirements to testable (preferably machine testable) specifications
• Design of architectures that treat development and verification of capabilities on an equal footing
• Post-fielding testing of adaptive/learning systems iii
• Run-time monitoring and multiple-agent systems
• Coactive Design of human-machine teaming approaches
• Communication and Interaction among autonomous systems and between autonomous systems and humans.