Rank-Ordering of Subjects Involved in the Evaluation of Automatic Speech Recognisers.
Abstract
It is well known that the performance of current automatic speech recognisers varies significantly between talkers. Hence it is essential for anyone involved in the evaluation of speech recognition system, or the assessment of the acceptability of such systems in a particular application, to be able to 'calibrate' potential talkers. The purpose of this memorandum is to provide measurements of the expected performance of a typical speech recoginser on a group of forty subjects who have provided recordings for speech recognister evaluation. The group includes all but one of the speakers from the RSRE speech database, all of the speakers who contributed towards the UK part of the NATO RSG10 spoken digit database, and a group of pilots involved in the assessment of automatic speech recognition in avionics applications at the royal aircraft establishment of Bedford and Farnborough. A method ranking of the subjects based on simpler measurements was also considered. This method potentially requires much less computation and was found to correlate well (rank correlation 0.687) with the ranking by expected error-rate.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 1986
- Accession Number
- ADA169546
Entities
People
- D. C. Smith
- M. J. Russell
- M. J. Tomlinson
Organizations
- Royal Signals and Radar Establishment