Scene Understanding and Reasoning for Robot Autonomy
Abstract
We will develop a software platform which will be deployed on a mobile robot, such as the SquadBot or the Atlas humanoid robot, to enable high-level robot autonomy. The software platform will have three sets of capabilities, summarizes.~ 3D scene parsing from images and video, including i) 3D scene reconstruction and 3D object recognition and localization; ii) 3D human poses estimation and 4D human-object interaction & action recognition; and iii) Recognizing doors, pathways, door knobs etc.~ Cognitive reasoning for scene commonsense understanding, including: i) inferring functionality and affordance maps in the 3D scenes; ii) inferring some physical relations between objects for estimating stability and risks; and iii) reasoning about potential hiding places for humans in the scene, for example, reasoning about containers.~ Robot navigation and manipulations based on additional sensory input, including: i) task and plan planning based on the computed affordance maps; ii) self-localization using various sensors; and iii) manipulating objects, such as twisting various types of door knobs and opening doors.We will leverage technology developed under the MURI visual common reasoning project, including:Spatial, temporal and causal And-Or Graphs for representing scenes and events and for predicting human intents and behaviors;4D human-object interaction models to represent functions and affordance;Models for physical and causal relations for reasoning; andHuman utility learning in daily scenes.We will first test the software on a mobile robot at UCLA which has manipulators complex indoor scene for the aforementioned tasks; and then we will deploy the software on a Squadbot robot at IHMC or other mobile robot(s) platform directed by ONR project manager. Potential Impacts. This project will extend traditional technologies used in robotics (e.g. SLAM for point cloud, navigation, and manipulation) by utilizing recent advancements in computer vision and cognitive reasoning. The new technology, demonstrated in MURI project, will bring deeperphysical and social scene understanding, such as functionality, affordance, human intent and utility etc. to robots, and thus enable higher level autonomy which are important for DoD tasks, such as autonomous robots, search and rescue missions.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 25, 2019
- Source ID
- N000141912153
Entities
People
- Song-Chun Zhu
Organizations
- Office of Naval Research
- United States Navy
- University of California, Los Angeles