HandSight: Supporting Everyday Activities Through Touch-Vision
Abstract
We built and evaluated a new system called HandSight to support activities of daily living (ADLs) for blind and low vision people by sensing and feeding back non-tactile information about the physical world as it is touched. HandSight consists of tiny cameras and micro-haptic actuators integrated into one or more fingers, computer vision algorithms to support inference and recognition, and a smartwatch for processing, power, and speech output. The two high-level goals were: first, to develop the basic building blocks of an extensible platform that could support a range of ADL applications. Second, to explore and demonstrate the potential of HandSight through three proof-of-concept applications: reading, dressing, and technology access. Research contributions include functional prototypes, computer vision algorithms to extract attributes of the physical world (e.g., text, color, texture) and to support on-body interaction for mobile technology accessibility, haptic feedback techniques to guide the users finger/hand, empirical data from user studies with blind and low vision users of all of the aforementioned contributions, and design recommendations for mobile and wearable computer-vision-augmented touch systems.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 2019
- Accession Number
- AD1149589
Entities
People
- Jon E. Froehlich
- Leah Findlater
- Rama Chellappa
Organizations
- University of Maryland