Software Testing using Algebraic Specification Based Test Oracles.
Abstract
In TAV4, the first author presented a paper describing an algorithm to perform run-time consistency checking of abstract data types specified using algebraic specifications. This algorithm has subsequently been incorporated into a run-time consistency checking tool for the Anna specification language for Ada, and works on a subset of all possible algebraic specifications. The algorithm implementation can be considered a test oracle for algebraic specifications that performs its activities while the formally specified program is running. This paper presents empirical results on the use of this test oracle on a real life symbol table implementation. Various issues that arise due to the use of algebraic specifications and the test oracle are discussed. 50 different errors were introduced into the symbol table implementation. On testing using the oracle, 60% of the errors were detected by the oracle, 35% of the errors caused Ada exceptions to be raised, and the remaining 5% went undetected. These results are remarkable, especially since the test input was simply one sequence of symbol table operations periormed by a typical client. The cases that went undetected contained errors that required very specific boundary conditions to be met - an indication that white box test data generation techniques may be required to detect them. Hence, a combination of white-box test data generation along with a specification based test oracle may be an extremely versatile combination in detecting errors. This paper does not address test data generation, rather it illustrates the usefulness of algebraic specification based test oracles during runtime consistency checking. Runtime consistency checking should be considered a complementary approach to unit testing using generated test data.
Document Details
- Document Type
- Technical Report
- Publication Date
- Apr 01, 1993
- Accession Number
- ADA314233
Entities
People
- Anoop Goyal
- Prakash Sikchi
- Sriram Sankar
Organizations
- Stanford University