The Detection of Duplicates in Document Image Databases
Abstract
Document imaging technology has developed to the point where it is not uncommon for organizations to scan large numbers of documents into databases with little or no index information. This may be done for archival purposes, in which case the necessary index may be as simple as a case number, or with the ultimate goal of automatically extracting index information for content-based queries. Maintaining the integrity of such a database is difficult, especially in a distributed environment where copies of documents with different physical histories may be scanned at different times. In this paper we present a novel approach to detecting duplicate documents in very large databases using only features extracted from the image. The method is based on a robust "signature" extracted from each document image which is used to index into a table of previously processed documents. The system is able to deal robustly with differences between scanned documents with respect to such factors as resolution, skew and image quality. The approach has a number of advantages over OCR or other recognition-based methods including speed and robustness to imaging distortions. To justify the approach and demonstrate its scalability, we have developed a simulator which allows us to change parameters of the system and examine performance while processing millions of document signatures. A complete system has been implemented and tested on a collection of technical articles and memos.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 1997
- Accession Number
- ADA458638
Entities
People
- David S. Doermann
- Huiping Li
- Kemal Kilic
- Omid Kia
Organizations
- University of Maryland