Results from Evaluation of Three Commercial Off the shelf Face Recognition Systems on Chokepoint Dataset
Abstract
This report is a supplement to Division Report 2014-27 (TR) Evaluation methodology for face recognition technology in video surveillance applications (by E. Granger and D. Gorodnichy) [1]. It presents complete evaluation results of three Commercial Off-The-Shelf (COTS) Face Recognition (FR) systems: Cognitec, PittPatt and Neurotechnology - obtained on the Chokepoint public data-set using the multi-level evaluation methodology introduced in the previous report. Four levels of details are examined according to the methodology for each target person from the Checkpoint data-set: Level 0 or score-based analysis illustrates the probability distribution of the genuine scores against that of the impostors at different face resolutions, which visually illustrates the discrimination power of the system for each target individual. Level 1 or transaction-based analysis provides the averaged description of the system in terms of false positive and false negative rates aggregated over all transactions, expressed using Receiver Operative Curves (ROC), Detection Error Trade-off (DET), and Precision-Recall Operating Characteristic (PROC) curves. Level 2 or subject-based analysis describes the performance of the system using the-so-called "Doddingtons Zoo" categorization of individuals, which detects whether an individual belongs to an easier or a harder classes of people that the system is able to recognize [2, 3]. Finally, Level 3 or temporal analysis allows one to examine the overall discrimination power of the system by accumulating the positive predictions while tracking a person over time and computing the recognition confidence based thereon. Two-page Report Cards summarizing the performance of the system for each target individual are published, thus providing an exhaustive report of the systems performance on a variety of different target subjects.
Document Details
- Document Type
- Technical Report
- Publication Date
- Sep 01, 2014
- Accession Number
- AD1018024
Entities
People
- D. Bissessar
- D. Gorodnichy
- E. Choy
- E. Granger
- J. Bergeron
- P. Radtke
- W. Khreich
Organizations
- Canada Border Services Agency