Principles for Testing a Data Fusion System
Abstract
The testing of a data fusing system presents a conceptual challenge. What seem to be simple and direct approaches to testing may be tantamount to making a judgment on the basis of the flip of a coin. In this paper we dissect the intelligence system and identify precisely the role of the fusion engine in it, with particular emphasis on distinguishing between the deterministic parts of the system and those that are sources of noise or randomness. It is emphasized that the role of the fusion engine is to perform a deterministic calculation of a probability distribution through a combining of more elementary probability distributions characterizing the sources of noise or randomness in the overall system. While such a deterministic calculator can be verified by comparing the results of the fusion engine s calculation to results that are computed by some independent means, it is recognized that many users would be more comfortable with a test in which the response of the fusion engine is compared in some fashion to the ground truth (or simulation thereof) that gave rise to the response. Toward this end we present a valid and practicable testing procedure which results in a chi-square comparison of the output distribution of the fusion engine with an appropriately derived histogram of instantiations of ground truth (or simulations thereof). It is shown that this test would be equivalent to estimating independently the desired output of the fusion engine by means of a Monte Carlo calculation. Finally, some other considerations and judgment criteria applicable to the fusion engine are mentioned.
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 31, 1998
- Accession Number
- ADA399143
Entities
People
- R. N. Dewitt