American Sign Language Recognition and Translation Feasibility Study
Abstract
The development of a system for automatically and robustly translating between American Sign Language (ASL) and spoken English in real time on mobile devices holds the promise of enabling natural and spontaneous communication between Deaf ASL signers and English speakers anywhere and anytime. One key component of such a system is the automatic recognition of ASL signs, which is an active area of research in the academic community. A number of other system challenges remain in order to support deploying this technology on mobile devices that include addressing compute limitations and recognition robustness for acquired signals that are highly variable (e.g., sign variation, apparent pose angle) and poorly matched to existing training corpora. While several commercial companies have pursued the development of mobile translation systems, none of them has successfully commercialized such a system to date. This report investigates the technical feasibility of performing real-time translation between ASL and spoken English on mobile devices. An important aspect of this investigation is the identification of the key technical challenges in developing such a system and the development of a roadmap for addressing these challenges. In order to support the feasibility study detailed in this report, an extensive literature search has been completed, a number of rigorous experiments have been performed to characterize state of the art performance, and a prototype system has been developed.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 20, 2018
- Accession Number
- AD1098892
Entities
People
- E. Salesky
- J. T. Melot
- Jim Williams
- K. Brady
- M. S. Brandstein
- M. T. Chan
- N. Malyska
- P. R. Khorrami
- Y. L. Gwon