A CASE STUDY IN THE APPLICATION OF CRANFIELD SYSTEM EVALUATION TECHNIQUES,
Abstract
An account is given of a project performed on behalf of the U.S. Navy Bureau of Ships Technical Library to evaluate the effectiveness of a computerized information retrieval system based on the Engineers Joint Council system of role indicators and links. The evaluation method used was that developed by Cleverdon in the ASLIB Cranfield Project. The search program involved all major combinations of appropriate descriptors and role indicators. Once the searches were performed, the documents produced were submitted to compilers of the original questions. A basis was obtained for computation of relevance ratios. Recall ratios were determined by actually doing a total check of the specimen collection to locate all documents that had any possible relevance to 10 of the test questions. The greatest cause of noise in the search results was searching errors, and the most frequently occuring type of searching error was the failure to select crucial terms or concepts in translating the questions into the language of the system. The question was different on analysis for reasons for failure to retrieve documents that were relevant to search questions. Here, indexing errors contributed more heavily to the cause of failure.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 1964
- Accession Number
- AD0608743
Entities
People
- F. W. Lancaster
- Saul Herner
- Walter F. Johanningsmeier