Hardware for Light Field & 3D Geometric Learning in Scene Understanding
Abstract
We request support for a large-scale GPU and CPU cluster for machine learning on highdimensional visual datasets such as 4D light fields, and 3D geometry. This cluster will support the research of PI Ramamoorthi, funded by ONR # N000141712687 on Computer Vision and Scene Understanding with Light Field Imaging. The project addresses key goals of computer vision and scene understanding to reconstruct and understand the visual world in 3D. This is relevant in many pivotal applications in national defense such as navigation, localization, and mapping for unmanned ground and aerial vehicles, and maritime surveillance for battlefield intelligence. We are increasingly seeing a revolution where new types of multi-camera imaging sensors are available, along with the computational power to take advantage of them. For example, currentmobile devices such as the iPhone 7+ include two cameras (dual lenses), with images combined using computational photography. Mobile camera arrays have been proposed by vendors such as Pelican Imaging. Consumer light field cameras have become available from Lytro and Raytrix. These cameras capture the full 4D light field, c orresponding n ot j ust t o 2D p ixel i ntensity, but also capturing the angular distribution of light. Moreover, almost every major camera vendor has released a virtual reality camera, which in general can be viewed as a 4D light field sensor. The PI~s ONR-funded research leverages his groundbreaking results on physics-based computer vision (recently honored by elevation to IEEE and ACM Fellow, and earlier by an ONR Young Investigator and PECASE awards), with advances in deep learning, which have recently revolutionized image understanding tasks. However, light fields are 4D quantities, involving substantial datasets. As such, significant computational hardware is needed for machine learning on large 4D light field datasets. Moreover, we are increasingly exploring the use of synthetic datasetscoupled with real images to train neural networks. The computer clusters requested will greatly assist in fast rendering of thousands of synthetic images.Along with 4D light field d ata, we are s eeing an explosion in 3D geometric models ( in fact light field cameras are a prime passive acquisition device for 3D geometric data). In collaboration with co-PI Su, we will investigate large-scale 3D machine learning. The hardware proposed here will be pivotal to planned ONR MURI proposal (Active Perception and Knowledge Exploitation in Navigation and Spatial Awareness) and ARO proposal (Foundations of Image and MultimodalData Analysis). We will build and deploy 20 GPU-based servers, each containing 8 GPUs (NVIDIA GeForce GTX 1080 Ti). Each server is expected to cost around $18.3K. We will further include 10 CPU servers for applications in which we need to generate large synthetic datasets or require significant CPU cycles for other purposes. We also include 4 storage servers for serving data and archivingpurposes. The plan leads to a total equipment cost of approximately ?$495K. This equipment will be actively used by some 20 Ph.D. students within the UC San Diego Center for Visual Computing, and will also enable collaborations on federally funded research with other computer vision faculty like Profs. Chandraker and Tu. Finally, the computer cluster will enable tight integrationof research and education for Ph.D. students, as well as MS and undergraduate students taking our advanced graphics, vision, and machine learning courses.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 25, 2019
- Source ID
- N000141912293
Entities
People
- Ravi Ramamoorthi
Organizations
- Office of Naval Research
- United States Navy
- University of California, San Diego