Underwater Computer Vision
Abstract
Project Summary/AbstractBackground. The rapid advances in deep learning over the last two decades have revolutionized the field ofcomputer vision. However, translating that remarkable success from traditional computer-vision tasks to theunderwater setting has proved elusive. The way light propagates in an aquatic environment is fundamentallydifferent from in air, resulting in two major challenges to the knowledge- and code-base we have built fortraditional computer vision: 1) in the underwater regime, colors and visibility are lost in a distance- andwavelength-dependent manner, and therefore image formation is governed by a different, and more complexequation that requires more features in images to be considered, and 2) it is impossible to obtain ground-truthdata for underwater scenes. These challenges call for a new way of thinking. The proposed approach. The new collaboration between our labs (PI Akkaynak#sLaboratory for #Computational Optics and Light in the Ocean Realm#, aka COLOR lab at the University of Haifa, and PI Freifeld#s #Vision, Inference, and Learning# group at Ben-Gurion University) is aimed at addressing such challenges in underwater computer vision via a unique approach that combines physics-driven methods (grounded in ocean optics and physical laws/constraints), data-driven learning-based computer-vision methods, and biological insights from the visual systems of marine creatures. We have two main objectives: 1) render underwater scenes as if they were imaged in air (including synthesizing novel views), and establish a framework for multi-object tracking/segmentation in underwater videos that results in performance as robust as traditional computer-vision methods. Scope. While the research proposed in this document focuses on the challenging tasks of (i) rendering underwater scenes without water, including underwater novel view synthesis and (ii) underwater multiple object tracking/segmentation, we emphasize that the scope of our new joint-research endeavor is larger than these particular tasks. We seek to make significant and measurable progress in decreasing the gap between traditional computer vision and its underwater counterpart. For this purpose, our focus is on the broader topics of underwater 3D reconstruction and underwater video analysis. Task (i) is an example of the former, while task (ii) is an example of the latter. Relevance to ONR and other organizations within DoD. In the context of underwater exploration, the potential of optical imaging has not yet been realized to its full potential. This is mainly because acoustic sensing provides much larger range inthe water than optical sensing, and consequently allows for exploring larger areas with less investment, and that generally most underwater optical images come out dull, with washed-out colors, low contrast and low visibility. Extending robust and high-performingimage analysis and computer-vision methods from the land domain to the underwater domain simplydoes not work. The end result is that objects in underwater images cannot be identified, segmented, classified, or tracked easily in an automated manner, and almost in every case, a human expert has to extend tedious, expensive effort to manually inspect the images and video, making them less preferred and less useful. Yet, there is a lot of information contained in underwater optical images that cannot be extracted from acoustic images, and with the right tools, this information can be exploited on its own, or a multi-modal way with acoustic sensing or withspecialized sensors like lidar. With our proposed project, we hope to pave the way so any underwater exploration mission, search and rescue operation, monitoring effort, or outreach activity benefits maximally from the use of cameras. We believe that our vision is highly relevant to ONR, as well as other Navy-related bodies within the DoD, and, potentially, civilian US Government agencies, and K-12 or higher education in
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 13, 2023
- Source ID
- N629092312104
Entities
People
- Oren Freifeld
Organizations
- Ben-Gurion University of the Negev
- Office of Naval Research
- United States Navy