Camera Relocalization using 3D Point Clouds for Enhanced Underwater Situational Awareness (white paper tracking # 20-000001053)
Abstract
Objective: The goal of this project is to transfer findings in uncertainty aware camera relocalization methods from terrestrial to underwater scenes. Enabling such self-localization capabilities is an initial step in the grand scheme of vehicles autonomously navigating in deep seas. Scope: We consider self-localizing a submarine against a known, static 3D reconstruction of an underwater environment. To this end, we propose to utilize the existing underwater SLAM datasets as well as deep neural networks designed for indoor/outdoor environments such as PoseNet. We will conduct research to adapt these techniques in the underwater environments. Schedule: This project is designed to last for one year with one leading PhD student. The milestones are: Data preprocessing, compiling previous works for relocalization on land, adapting these methods to work in underwater environments, and testing/deployment as well as reporting. Background: Within the past couple of years, starting with the well known PointNet and PointNet++ networks, our Geometric Computing Group has been a dominant player in 3D geometric deep learning, and especially object detection and segmentation. Recently, we have pioneered research on uncertainty-aware methods for estimating camera as well as object poses in scenarios that contain ambiguities. As in the setting of this project, the ambiguities in terrestrial scans can arise due to self-similar scene structures, self-occlusions, object symmetries or unforeseen noise. We have gathered our efforts on these fronts in a project website: http://multimodal3dvision.github.io. We have also worked on camera relocalization in changing environments using multiple approaches. Finally, our group has broad experience in 3D scene reconstruction, that is a relevant prerequisite for 6D relocalization. Tasks/Scientific Goals: Our project in five stages as follows: (1) We begin with a data processing stage in which we restructure two different public datasets on underwater environments into a common format that is well suitable for localization. At this point we may also apply certain input refinement methods which can improve the quality of the ground truth data in learning tasks; (2) We gather together the state of the art methods in relocalization from terrestrial data and prepare a unifying framework; (3) We then assess the performance of both correspondence-based and direct methods on the underwater datasets and identify advantages as well as disadvantages of different algorithms; (4) We incorporate a suitable probabilistic regression module which could result in a diverse set of modes, one for each potential solution, as well as a confidence estimate; (5) We finally conclude with a careful study of the failure modes to motivate future directions. Outcome & Reporting: We plan to make our code public and publish the findings in the top journals/conferences. We shall also report the research progress semi-annually. Eventually, we envision this project as helping create a visual self-localization software stack for autonomous underwater vehicles.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 06, 2021
- Source ID
- N000142112082
Entities
People
- Leonidas J. Guibas
Organizations
- Office of Naval Research
- Stanford University
- United States Navy