Robustness Testing of Software-Intensive Systems: Explanation and Guide
Abstract
Many Department of Defense (DoD) programs engage in what has been called "happy-path testing" (that is, testing that is only meant to show that the system meets its functional requirements). While testing to ensure that requirements are met is necessary, often tests aimed at ensuring that the system handles errors and failures appropriately are neglected. Robustness has been defined by the Food and Drug Administration as "the degree to which a software system or component can function correctly in the presence of invalid inputs or stressful environmental conditions." This technical note provides guidance and procedures for performing robustness testing as part of DoD or federal acquisition programs that have a software component. It includes background on the need for robustness testing and describes how robustness testing fits into DoD acquisition, including source selection issues, development issues, and developmental and operational testing issues. Section 1 provides general background and definitions. Section 2 provides information on testing guidance in relation to robustness testing. In addition, DoD, federal, and commercial testing sources are discussed. Section 3 describes the actual robustness testing procedures. Section 4 describes how robustness testing fits into DoD acquisition, including source selection issues, development issues, as well as developmental and operational testing issues. Appendix A provides a table of the specific tests called out in Section 3. Appendix B contains additional references to software testing in commercial standards. Appendix C contains links to resources that may be useful when dealing with robustness testing. Appendix D contains an acronym list.
Document Details
- Document Type
- Technical Report
- Publication Date
- Apr 01, 2005
- Accession Number
- ADA441308
Entities
People
- Dan Plakosh
- Julie Cohen
- Kristi Keeler
Organizations
- Carnegie Mellon University