Rapid Face Asset Acquisition with Recurrent Feature Alignment
Abstract
We present Re current F eature A lignment (ReFA), an end-to-end neural network for the very rapid creation of production-grade face assets from multi-view images. ReFA is on par with the industrial pipelines in quality for producing accurate, complete, registered, and textured assets directly applicable to physically-based rendering, but produces the asset end-to-end, fully automatically at a significantly faster speed at 4.5 FPS, which is unprecedented among neural-based techniques. Our method represents face geometry as a position map in the UV space. The network first extracts per-pixel features in both the multi-view image space and the UV space. A recurrent module then iteratively optimizes the geometry by projecting the image-space features to the UV space and comparing them with a reference UV-space feature. The optimized geometry then provides pixel-aligned signals for the inference of high-resolution textures. Experiments have validated that ReFA achieves a median error of 0.603 mm in geometry reconstruction, is robust to extreme pose and expression, and excels in sparse-view settings. We believe that the progress achieved by our network enables lightweight, fast face assets acquisition that significantly boosts the downstream applications, such as avatar creation and facial performance capture. It will also enable massive database capturing for deep learning purposes.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Nov 30, 2022
- Source ID
- 10.1145/3550454.3555509
Entities
People
- Haiwei Chen
- Shichen Liu
- Yajie Zhao
- Yichao Zhou
- Yunxuan Cai
Organizations
- United States Army Research Laboratory
- University of California, Berkeley
- University of Southern California