A System-Theoretic Approach to Engineering Assurance for Artificial Intelligence Systems
Abstract
Predictive systems that incorporate neural network models have been deployed in both non-safety-critical and highly safety-critical domains. When the models fail to perform as desired, it is often difficult to identify a root cause. In domains where failures might cause irreparable damage, or harm to life or property, steps must be taken to assure those using these systems that risks have been mitigated by thoughtful analysis during design. This thesis demonstrates the use of system-theoretic process analysis (STPA) as a repeatable approach for selecting and calibrating machine learning development actions to provide assurance during the machine learning development life cycle (MDLC). STPA is a system analysis method that identifies component hazards arising from component-level interactions in safety-critical systems. In this research, STPA is used to assess machine learning development safety, with respect to responsible artificial intelligence (AI) principles, for a system that utilizes a classification model to detect maritime vessels based upon audio signatures. As an outcome of the analysis, recommendations are made that can proactively guide the AI design process so that decisions within each phase of the life cycle are explainable. It is demonstrated that by applying this approach, AI systems are more reliable and safer to deploy.
Document Details
- Document Type
- Technical Report
- Publication Date
- Sep 01, 2023
- Accession Number
- AD1224376
Entities
People
- Eugene D. Williams
Organizations
- Naval Postgraduate School