3D Human Gesture and Activity Recognition in the presence of degraded environments: Algorithms, Mathematical Models, and Performance Evaluation
Abstract
Automated recognition of human activity from visual sources using machine learning has received significant attention from the computer vision communities. Various visual representations and classification approaches have been proposed to address challenging and practical issues in human activity recognition. While human biological sensing and computation is excellent at identifying human thre"ats, gestures and activities, machine learning approaches so far, in spite of all the progress made, have not come up with effective"", robust, scalable, and efficient solutions, particularly when humans are in complex and degraded environments. Thus, reliable algor"ithms and computational methods for human activity recognition have been elusive thus far for complex and unpredictable environments such those of interest to Navy war fighters.Human gestures and activities in cooperative and simple environments that are not deg"raded, humans with closeproximity to the sensors (high SNR), or in uncluttered backgrounds and without obscurations are simpler to"" identify. On the other hand, human activity and gesture recognition in complex and dynamically changing degraded environments, such"" as humans in the presence of occlusion and obscurations, dust, fog, smoke, low light levels, etc., and humans in very cluttered bac"kgrounds and crowded scenes where the gestures or activities are a small fraction of the entire scene are substantially more difficu"lt to detect, and there have not been any effective solutions so far. Also, for visual source data, most of the past machine intelli"gence approaches for algorithm development on human action recognition have focused on 2D images.A comprehensive investigation of" action recognition in the presence of degraded environments such as occlusion and obscurations, low light levels, low visibility (s""moke, fog, dust), etc. has not be carried out to the best of our knowledge. Although the action recognition community widely acknowl""edges the importance of the robustness against occlusion, it is an issue which is rarely investigated in practice, and it is conside""red an open issue due to the complexity of formalizing obscurations and low visibility conditions, and lack of datasets to promote a"" systematic study. While some forms of 3D imaging have been applied to gesture recognition, they were usually with RGB images plus d""epth map sensors such as Kinect which are limited to objects within a very short range (a few meters) of the sensor, and do not perf""orm well in the presence of occlusion and/or complex environments that may be of interest to Navy war fighters.In this project, we" propose to investigate machine intelligence approaches for algorithm development andcomputational methods on human activity and gestures recognition that are optimum with 3D visual image source data which can perform well for complex and dynamically changing deg"raded environments such as humans in the presence of occlusions and obscurations, dust, fog, smoke, low light levels, etc., and also"" for human activity embedded in very cluttered, dynamic, and unpredictable background. The main focus of the project will be on deve"loping computational methods and algorithms for automated human activity recognition with 3D data. We shall investigate a comprehensive computational method based solution using 3D data including algorithms for detection of human targets in complex backgrounds and" degraded environments, followed by human activity and gesture recognition of the areas of interest. To the best of our knowledge, t"his will be the first comprehensive attempt to investigate and develop algorithms and computational methods based on 3D data for human action recognition in degraded environments. This investigation can substantially contribute to human activity recognition and be potentially transformative because of the unique capabilities that optimized algorithms based on 3D data can provide for action recognition in comple
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 09, 2017
- Source ID
- N000141712561
Entities
People
- Bahram Javidi
Organizations
- Office of Naval Research
- United States Navy
- University of Connecticut