Serial Experimentation for the Management and Evaluation of Communications Systems,
Abstract
When there is a decision to mount a demonstration of new communications systems in the field, controversy about the role of evaluation often results. Those who have operational responsibility for the system become its advocates, believing it to be sufficiently robust to succeed. They resent research that would divert resources from the central purpose of demonstrating success. In contrast, evaluators anticipate the chance of failure. They maintain that it is essential to construct systematic and rigorous research on a project so that future projects can learn from past experiences. The conflict is likely to intensify when the demonstration is underway and something starts going wrong. Project personnel will quickly want to make changes, trying to correct the difficulty. Such action is clearly detrimental to the evaluator's efforts since the more rigorous their design, the more irrelevant the data usually will be if the program has been continually altered. But try, as an evaluator, to persuade a program manager that he or she should leave a failing field program unaltered so that you can rigorously establish the reason for failure in a final report. One approach to this problem which serves the purposes of both program advocates and evaluators is to use a research design that consists of a series of short experiments that can mutate through successive generations.
Document Details
- Document Type
- Technical Report
- Publication Date
- Sep 01, 1977
- Accession Number
- ADA054184
Entities
People
- Suzanne G. Quick
- William A. Lucas
Organizations
- RAND Corporation