Geometry-Aware Distillation for Indoor Semantic Segmentation
Abstract
It has been shown that jointly reasoning the 2D appearance and 3D information from RGB-D domains is beneficial to indoor scene semantic segmentation. However, most existing approaches require accurate depth map as input to segment the scene which severely limits their applications. In this paper, we propose to jointly infer the semantic and depth information by distilling geometry-aware embedding to eliminate such strong constraint while still exploiting the helpful depth domain information. In addition, we use this learned embedding to improve the quality of semantic segmentation, through a proposed geometry-aware propagation framework followed by several multi-level skip feature fusion blocks. By decoupling the single task prediction network into two joint tasks of semantic segmentation and geometry embedding learning, together with the proposed information propagation and feature fusion architecture, our method is shown to perform favorably against state-of-the-art methods for semantic segmentation on publicly available challenging indoor datasets.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 16, 2019
- Accession Number
- AD1152128
Entities
People
- Honghui Shi
- Jianbo Jiao
- Rynson Lau
- Thomas Huang
- Yunchao Wei
- Zequn Jie
Organizations
- City University of Hong Kong
- Department of Engineering Science, University of Oxford
- IBM Research
- University of Illinois Urbana–Champaign