Defining the Meaning of a Major Modeling and Simulation Change as Applied to Accreditation
Abstract
Increasingly, decisions are being made based partially or entirely on models. These decisions may be quite important, with the potential of serious or unacceptable consequences if an incorrect decision is made. The models used to support the decisions may have been newly developed or modified versions of existing models. In the former case, it seems clear that a new model should undergo validation before it is used for any significant application. In the latter case, the matter may be less clear; must the modified model, which presumably was validated when it was initially developed, be subject to another round of validation due to the modifications? Several existing methods address the re-validation question, including methods developed by the Johns Hopkins University Applied Physics Laboratory, the Institute of Electrical and Electronics Engineers, the Joint Accreditation Support Agency, and the project's sponsoring agency. These existing methods vary widely in several respects including level of detail, specificity with respect to modeling and simulation, degree of quantitativeness, ease of use, and applicability to the sponsoring agency. All include some form of the notion of risk, which is conventionally defined as the product of the likelihood of an incorrect decision and the consequences of such a decision. A new method, the Quantitative-to-Qualitative Risk-based (QQR) method, was developed to make a quantitative recommendation regarding the re-validation of a modified model. The QQR method was developed with these goals in mind: to be quantitative, repeatable, and transparent to consider both model modifications and model use risk; to focus on model types and simulation applications of interest to the sponsoring agency; and to be simple and accessible so as to encourage its use in practical applications. The QQR method estimates the probability that not re-validating a modified model will lead to unacceptable consequences, given the modifications made to it.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 12, 2012
- Accession Number
- ADA582534
Entities
People
- J. C. Beach
- Mikel D. Petty
- Philip W. Alldredge
- Wesley N. Colley
Organizations
- Systems Engineering Research Center